GitHub Release Tracker
All JS React Ruby Go Postgres Frontend Node
24h 7d 14d 30d | Newest Best Latest version only Popular

crawler

Past 30d, sorted by best first, all versions
7 results Markdown version
05/15 8
firecrawl/Firecrawl v2.9.1
API service that crawls URLs and converts web content into clean markdown or structured data without requiring sitemaps.
TypeScript 122962☆ 767d old #javascript #typescript #data #ai #markdown
05/04 8
projectdiscovery/katana v1.6.0
Next-generation crawling and spidering framework designed for automation pipelines.
Go 16681☆ 1956d old #golang #cli #go #crawler #web-spider
05/15 7
firecrawl/Firecrawl v2.10
API service that crawls URLs and converts web content into clean markdown or structured data without requiring sitemaps.
TypeScript 122962☆ 767d old #javascript #typescript #data #ai #markdown
05/05 7
projectdiscovery/katana v1.6.1
Next-generation crawling and spidering framework designed for automation pipelines.
Go 16681☆ 1956d old #golang #cli #go #crawler #web-spider
05/23 5
chushuai/wscan 1.0.41
Web security scanner that uses machine learning to automate and personalize web penetration testing.
Go 706☆ 1479d old #golang #go #vulnerability-scanner #sql-injection #crawler
05/10 5
seantomburke/Sitemapper 4.1.6
XML sitemap parser for Node.js supporting sitemap indexes plus image and video formats.
TypeScript 134☆ 4283d old #javascript #typescript #parse #crawler #sitemap
05/05 5
pzaino/The CROWler v1.2.3
Self-hosted, event-driven platform for browser-based web crawling, scraping, detection, and automation with rulesets, plugins, agents, and search API.
Go 54☆ 908d old #golang #go #crawler #crawling #indexing