lightfeed / extractor
Use LLMs to robustly extract web data
Why this job: matched by what the repo talks about: web, data, llm, extract
What it does
- LLM-ready Markdown. Convert HTML to LLM-ready markdown, with options to extract only main content and clean URLs by removing tracking parameters.
- LLM Extraction. Use LLMs in JSON mode to extract structured data according to input Zod schema. Token usage limit and tracking included.
- JSON Recovery. Sanitize and recover failed JSON output. This makes complex schema extraction much more robust, especially with deeply nested objects and arrays.
- URL Validation. Handle relative URLs, remove invalid ones, and repair markdown-escaped links.
- Works with Playwright. Use Playwright to load pages, then extract structured data from the HTML content.
Get started
npm install @lightfeed/extractor @langchain/core - Maintenance
- Active
- Last commit 5 months ago
- Stars
- 321
- On GitHub
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.5.4
- Released 5 months ago
- Built with
- TypeScript
- 100% of the code
Is it looked after?
- Contributors
- 0
- Last 90 days
- Issues
- None
- No issues filed
- Pull requests
- 69 merged
- 2 waiting
- Releases
- 21
- 10 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| firecrawl / firecrawl | 200,979 | today | AGPL-3.0 | No downloads |
| ScrapeGraphAI / Scrapegraph-ai | 32,728 | 37 days ago | MIT | No downloads |
| itsOwen / CyberScraper-2077 | 3,300 | 31 days ago | MIT | No downloads |
More: languages, topics and project record
Languages
- TypeScript 100%
- JavaScript <1%
Topics
- ai-agents
- article-extractor
- crawler
- data-engineering
- data-pipeline
- ecommerce-scraping
- etl
- google-gemini
- html-parser
- html-to-markdown
- llm
- llm-extraction
Project record
- JobAutomation and scraping
- StatusListed
- Commits76
- Forks10
- Watching0
- Stars in 28 days+1
- CreatedMay 2, 2025
- First seen by usNov 7, 2026
- CheckedNov 7, 2026
