GitHub Release Tracker
All JS React Ruby Go Postgres Frontend Node

crawler

Past 30d, sorted by best first
8 results Markdown version
10/05 8
projectdiscovery/katana v1.8.0
Next-generation crawling and spidering framework designed for automation pipelines.
Go 17639☆ 2106d old #golang #cli #go #crawler #web-spider
10/06 7
cv-cat/catbus v0.1.0
Command-line tool that gathers item search results from multiple social platforms and outputs normalized JSON for users and AI agents.
TypeScript 224☆ 11d old #javascript #typescript #cli #bilibili #crawler
10/06 7
he-yufeng/FindJobs-Agent v1.1.0
Full-stack job search assistant that crawls postings, uses LLMs to analyze skills, matches them to parsed resumes, and runs AI mock interviews.
Python 239☆ 326d old #python #llm #crawler #interview #ai-agent
10/01 7
pzaino/The CROWler v2.2.0
Self-hosted, event-driven platform for browser-based web crawling, scraping, detection, and automation with rulesets, plugins, agents, and search API.
Go 62☆ 1062d old #golang #go #crawler #crawling #indexing
10/09 6
scrapfly/Scrapfly Anti-bot Detector v2.8.4
Manifest V3 Chrome extension detecting CAPTCHAs, anti-bot systems, and browser fingerprinting in real time with confidence scores and capture tools.
JavaScript 603☆ 402d old #javascript #captcha #detection #scraper #crawler
10/05 6
loadkpi/CrawlerDetect v1.2.18
Ruby gem for detecting bots, crawlers, and spiders based on user agent and HTTP headers.
10/05 5
microlinkhq/top-user-agents v2.1.139
Always up-to-date list of the top 100 HTTP user-agent strings used on the Internet.
JavaScript 378☆ 2421d old #javascript #browser #desktop #user-agent #crawler
09/21 5
chushuai/wscan 1.0.59
Web security scanner that uses machine learning to automate and personalize web penetration testing.