All
JS
React
Ruby
Go
Postgres
Frontend
Node
crawler
Past 30d, sorted by best first
| 05/15 |
8
|
API service that crawls URLs and converts web content into clean markdown or structured data without requiring sitemaps.
|
| 05/04 |
8
|
projectdiscovery/katana
v1.6.0
Next-generation crawling and spidering framework designed for automation pipelines.
|
| 05/23 |
5
|
Web security scanner that uses machine learning to automate and personalize web penetration testing.
|
| 05/10 |
5
|
XML sitemap parser for Node.js supporting sitemap indexes plus image and video formats.
|
| 05/05 |
5
|
Self-hosted, event-driven platform for browser-based web crawling, scraping, detection, and automation with rulesets, plugins, agents, and search API.
|