Skip to content
The Daily Repo

lightfeed / extractor

Use LLMs to robustly extract web data

Why this job: matched by what the repo talks about: web, data, llm, extract

What it does

  • LLM-ready Markdown. Convert HTML to LLM-ready markdown, with options to extract only main content and clean URLs by removing tracking parameters.
  • LLM Extraction. Use LLMs in JSON mode to extract structured data according to input Zod schema. Token usage limit and tracking included.
  • JSON Recovery. Sanitize and recover failed JSON output. This makes complex schema extraction much more robust, especially with deeply nested objects and arrays.
  • URL Validation. Handle relative URLs, remove invalid ones, and repair markdown-escaped links.
  • Works with Playwright. Use Playwright to load pages, then extract structured data from the HTML content.

Get started

npm install @lightfeed/extractor @langchain/core
Maintenance
Active
Last commit 5 months ago
Stars
321
On GitHub
Licence
Apache-2.0
Allows business use
Downloads for
None listed
No release files
Latest version
v0.5.4
Released 5 months ago
Built with
TypeScript
100% of the code

Is it looked after?

Contributors
0
Last 90 days
Issues
None
No issues filed
Pull requests
69 merged
2 waiting
Releases
21
10 in the last year

Similar repos

Repos similar to this one
RepoStarsLast commitLicenceDownloads for
firecrawl / firecrawl200,979todayAGPL-3.0No downloads
ScrapeGraphAI / Scrapegraph-ai32,72837 days agoMITNo downloads
itsOwen / CyberScraper-20773,30031 days agoMITNo downloads
More: languages, topics and project record

Languages

  • TypeScript 100%
  • JavaScript <1%

Topics

  • ai-agents
  • article-extractor
  • crawler
  • data-engineering
  • data-pipeline
  • ecommerce-scraping
  • etl
  • google-gemini
  • html-parser
  • html-to-markdown
  • llm
  • llm-extraction

Project record

  • JobAutomation and scraping
  • StatusListed
  • Commits76
  • Forks10
  • Watching0
  • Stars in 28 days+1
  • CreatedMay 2, 2025
  • First seen by usNov 7, 2026
  • CheckedNov 7, 2026

Type to search.

Try