* [Firecrawl v2.9.1](https://github.com/firecrawl/firecrawl) – API service that crawls URLs and converts web content into clean markdown or structured data without requiring sitemaps.

* [katana v1.6.0](https://github.com/projectdiscovery/katana) – Next-generation crawling and spidering framework designed for automation pipelines.

* [wscan 1.0.41](https://github.com/chushuai/wscan) – Web security scanner that uses machine learning to automate and personalize web penetration testing.

* [Sitemapper 4.1.6](https://github.com/seantomburke/sitemapper) – XML sitemap parser for Node.js supporting sitemap indexes plus image and video formats.

* [The CROWler v1.2.3](https://github.com/pzaino/thecrowler) – Self-hosted, event-driven platform for browser-based web crawling, scraping, detection, and automation with rulesets, plugins, agents, and search API.