Feedstock

Feedstock is a TypeScript library for web crawling and scraping that runs on Bun and leverages Playwright for browser automation. It provides a native TypeScript API without additional build steps, offering developers the ability to control page navigation, resource blocking, DOM down‑sampling, and metadata extraction directly from code. The library returns a structured CrawlResult containing raw HTML, cleaned HTML, Markdown conversion, classified links, media assets, and extracted metadata such as titles, descriptions, and Open Graph tags.

The tool supports deep crawling with configurable traversal strategies (BFS, DFS, Best‑First) and allows filtering, scoring, and custom extraction via CSS selectors, regular expressions, XPath, or composite rules. Users can choose between Playwright’s full browsers or Lightpanda backends, and benefit from built‑in caching, rate limiting, robots.txt handling, and anti‑bot detection features.

Feedstock is aimed at developers who need to programmatically collect and transform web content into structured formats, such as Markdown or JSON, for downstream processing, data pipelines, or site analysis. Its design emphasizes speed, extensibility through strategy patterns, and a seamless TypeScript development experience.