Other●●Solid
The Crawl Times – Newspaper-style front pages for tech news sites
Cloudflare /crawl API powers nostalgic newspaper layouts for Hacker News and friends.
Eye CandyCozy
g_br_l
503mo ago
Back to browse
GitHub Repository
Lightweight CLI for crawling documentation sites into Markdown with defuddle
3 starsTypeScript
CLI for crawling documentation sites into Markdown with defuddle
by nistuley·Jun 3, 2026·5 points·0 comments
AI Analysis
●●SolidShip ItSolve My Problem
No-browser docs crawler using defuddle when Firecrawl and JinaAI already exist.
Strengths
- •No browser dependency means lighter weight than Puppeteer-based scrapers
- •Defuddle integration handles messy docs HTML cleanly
- •Manifest.json with content hashes enables incremental updates
Weaknesses
- •Doesn't execute JavaScript, so dynamic docs sites won't render
- •RAG docs ingestion already served by Firecrawl, JinaAI Reader
Category
Target Audience
Developers building RAG pipelines or local knowledge bases
Similar To
Firecrawl · JinaAI Reader · Crawlee
Similar Projects
Developer Tools●●Solid
Dw2md – Compile all DeepWiki pages into a single, LLM-friendly file
DeepWiki scraper for Claude, but Jina and Firecrawl already do this better.
Solve My ProblemNiche Gem
nwyin
104mo ago
Developer Tools●●●Banger
DocMCP – Index any docs site locally, search it from Claude via MCP
Hybrid BM25 + vector search via MCP beats pure keyword or pure semantic for API docs.
Big BrainSolve My Problem
pieeee
203mo ago
Developer Tools●Mid
Qapir – Generate API tests automatically from docs
Auto-generates API tests from OpenAPI specs when Schemathesis and Postman already exist.
Ship ItSolve My Problem
gradnichkovski
321mo ago
Developer Tools●Mid
DocOCR – Convert document images to Markdown locally on macOS
Local Markdown OCR via CLI and HTTP, though macOS Live Text overlaps heavily.
Solve My ProblemCozy
riddleling
4016d ago
Developer Tools●Mid
Crawl API
Clean API design but JinaAI and Firecrawl already dominate this crowded scraping category.
Slick
mddanishyusuf
103mo ago