kabegame

Kabegame — An anime image crawler client with pluggable crawlers (from a GitHub plugin repo), wallpaper rotation, local…

Rust★ 67

androidanimecrawlerlinuxmacosopen-sourceotakutauri

k@kabegame++

Tor_Spider

Python project to crawl and scrap the lesser known deep web or one can say dark web. Just provide the onion link and ge…

Python★ 70

crawlerfile-manageriocpython3scraperscrapingsocksstem

a@absingh31++

tir

Have time.ir in shell!

Python★ 155

calendarcrawlerdateiraniranianiranian-calendarshellterminal

p@pouriya++

lightnovel_epub

🍭 epub generator for (light)novels (轻)小说 epub 生成器，支持站点：轻之国度、轻小说文库

Python★ 344

clicrawlerebookepublightnovellknovelopencv

j@JeffersonQin++

EasySpider

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的…

JavaScript★ 44k

batch-processingbatch-scriptcode-freecrawlerdata-collectionfrontendguihtml

n@NaiboWang++

seo-audits-toolkit

SEO & Security Audit for Websites. Lighthouse & Security Headers crawler, Sitemap/Keywords/Images Extractor, Summarize…

Python★ 804

analysisauditscrawlerdashboardextractorheadersinternal-linkslighthouse

s@StanGirard++

diskover-community

Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by…

PHP★ 1.8k

crawlerdisk-spacedisk-space-analyzerdisk-usageduplicate-filesduplicatefilefinderelasticsearchfile-indexing

d@diskoverdata++

js-cookie-monitor-debugger-hook

js cookie逆向利器：js cookie变动监控可视化工具 & js cookie hook打条件断点

TypeScript★ 783

crawlerjs-reversered-teamreverse-engineeringuserscriptweb-security-research

j@JSREI++

Pxer

A tool for pixiv.net. 人人可用的P站爬虫

JavaScript★ 1k

add-onbatchcrawlerpixivtampermonkeyuserscript

p@pea3nut++

fredy

❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in German…

JavaScript★ 1.4k

appartmentappartmentscrawlerebay-kleinanzeigenfredyimmobilienimmobilienscout24immonet

o@orangecoding++

pydoll

Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.

Python★ 7k

anti-detectionautomationbrowser-automationcdpchromiumcrawlere2e-testsfingerprinting

a@autoscrape-labs++

autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Python★ 7.8k

aiartificial-intelligenceautomationcrawlermachine-learningpythonscrapescraper

a@alirezamika++

crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs,…

Python★ 9.4k

apifyautomationbeautifulsoupcrawlercrawlingheadlessheadless-chromeparsel

a@apify++

maxun

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into stru…

TypeScript★ 17k

agentsapiautomationbrowser-automationcrawlercrawlingdata-extractionno-code

g@getmaxun++

katana

A next-generation crawling and spidering framework.

Go★ 17k

clicrawlergocrawlerhacktoberfestheadlessspider-frameworkweb-spider

p@projectdiscovery++

crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScr…

TypeScript★ 25k

apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejs

a@apify++

Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Python★ 72k

aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extraction

d@D4Vinci++