samber / the-great-gpt-firewall
🤖 A curated list of websites that restrict access to AI Agents, AI crawlers and GPTs
Why this job: says: curated list
What it does
This collection is a curated list of websites that employ the robots.txt file to restrict access to AI Agents, AI crawlers and GPTs.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 8 days ago
- Stars
- 101
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- 2026.11
- Released 8 days ago
- Built with
- Python
- 100% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 0% closed
- 1 still open
- Pull requests
- 7 merged
- 1 waiting
- Releases
- 12
- 8 in the last year
- No contributing guide
- Security policy (opens in new tab)
- Code of conduct (opens in new tab)
More: languages, topics and project record
Languages
- Python 100%
Topics
- agent
- anthropic
- blocklist
- censorship
- crawler
- firewall
- genai
- generative-ai
- gpt
- gpt-4
- llm
- openai
Project record
- JobLists and guides
- StatusListed
- Commits66
- Forks7
- Watching5
- Stars in 28 days+1
- CreatedFeb 13, 2024
- First seen by usNov 7, 2026
- CheckedNov 7, 2026