Reverse Dependencies of Scrapy

The following projects have a declared dependency on Scrapy:

• 2adif — Convert table to ADIF format.
• Advertools — Digital Marketing productivity and analysis tools.
• AdyanUtils — Special package
• aha-scrapyd — Pre-built of Scrapyd
• airscrapy — Scrapy contrib for Airflow
• allusgov — This project attempts to map the organization of the US Federal Government by gathering and consolidating information from various directories.
• angeltools — personal python small tools collection
• anikore — Anikore API for Python
• apify — Apify SDK for Python
• arachnado — Scrapy-based Web Crawler with an UI
• ArachneScrapy — API for Scrapy spiders
• aroay-cloudscraper — scrapy的一个下载中间件，绕过cloudflare检测
• aroay-pyppeteer — scrapy的一个下载中间件，无缝对接pyppeteer
• aroay-translate — scrapy的异步翻译，最准确的翻译器
• article-extract — Article extractor can extract title, time, author, article content, etc. according to article URL.
• athlinks-races — Web scraper for race results hosted on Athlinks.
• ayugespidertools — scrapy 扩展库：用于扩展 Scrapy 功能来解放双手。
• baotool — BaoTool (宝图), 个人积累的 python 工具库
• bibscrap — Semi-automated tools for systematic literature reviews.
• board-game-scraper — Board games data scraping and processing from BoardGameGeek and more!
• bocfx — Easy API to get foreign exchange rate from Bank of China.
• braise — Lunch without #lunch
• bwt-scrapy-proxy — My short description for my project.
• c4v-py — Code for Venezuela python library.
• camcops-server — CamCOPS server
• canvasrobot — Library which uses Canvasapi (see https://canvasapi.readthedocs.io) to provide a CanvasRobot class.
• catlyst — Extracting Data using scrapy framework
• cewler — Custom Word List generator Redefined
• chady — A package for ML libraries
• chronos_ai — no summary
• city-scrapers-core — Core functionality for City Scrapers projects
• city-scrapers-sentry — Scrapy extension that logs errors to Sentry
• ClappScrapers — Clappform Python scraper
• cloudacademy-crawler — A simple cloudacademy course crawling & downloading tool
• clutch.co-scraper — clutch.co-scraper is a command-line application written in Python that scrapes and saves information about firms according to the user-defined filters.
• cohospider — A package to scrape Companies House data
• coupang — Python wrapper for coupang open api
• crau — Easy-to-use Web archiver
• crawlab-ai — SDK for Crawlab AI
• crawlab-demo — Demo for Crawlab
• crawlab-sdk-ssl — Python SDK for Crawlab
• crawler-test — This is a web application that extracts images URLs from web pages.
• crawlib — tool set for crawler project.
• crawltools — Simple crawlers
• crawlutils — Base Scrapy project
• crwutils — Base Scrapy project
• data-engineering-job-market — A data engineering batch pipeline to analyse job postings.
• data-job-crawler — no summary
• data-prep-connector — Scalable and Compliant Web Crawler
• datalad-crawler — DataLad extension package for crawling external web resources into an automated data distribution
• datasource-contributor — A CUI tool that automatically crawls website data and contributes to http://www.citybrain.org
• deepctrl-scrapy-redis — Redis-based components for Scrapy.
• dig-spider — NO CODE!!! Base on Scrapy, crawl websites with simple configuration.
• dj-kaos-webclipper — Clip web pages html and store in Django
• django-covid19 — A django app of covid-19 API in countries around the world, provinces and cities in China, and states in the USA.
• django-generic-search — A Django app to conduct generic search on a django web application.
• docrawl — Do automated crawling of pages using scrapy
• docscraper — A web crawler to scrape documents from websites
• dootle — A Library of extensions for Doot task running
• dr-zoidberg — Are you ready to operate, Doctor? - I'd love to, but first I have to perform surgery.
• dsbundle — Streamline your data science setup with dsbundle in one effortless install.
• DtCrawlEngine — 一个使用http api提供爬虫脚本服务的引擎
• duplicate-url-discarder — Discarding duplicate URLs based on rules.
• e-models — Tools for helping build of extraction models with scrapy spiders.
• eagle-kaist — Stock Extractor library
• easydown — Downloader by scrapy
• ecoindex-cli — `ecoindex-cli` is a CLI tool that let you make ecoindex tests on given pages
• eintf — no summary
• elcrawlsdk — Python SDK for elcrawl
• energy-trading-api — API Wrappers for the Energy Markets
• ensembl-rest — An interface to the Ensembl REST APIs, biological data at your fingertips.
• esgf-scraper — Keeps a local data repository in syncronisation with ESGF
• favorites-crawler — Crawl your personal favorite images, photo albums, comics from website. Support pixiv, yande.re for now.
• finance-cn — python for wps
• finscraper — Web scraping API for Finnish websites
• friday-cli — AI-powered testing agent
• frontera-seedloader-mongodb — no summary
• fxportia — Convert portia spider definitions to python scrapy spiders
• fzutils — A Python utils for spider
• gather-vision — Obtain, extract, organise, and store information.
• genutility — A collection of various Python utilities
• geo-spider — Crawling all GEO metadata.
• gerapy — Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.
• gerapy-item-pipeline — Item Pipeline Components for Scrapy & Gerapy
• gerapy-playwright — Playwright Components for Scrapy & Gerapy
• gerapy-prometheus-exporter — Prometheus Exporter Components for Scrapy & Gerapy
• gerapy-proxy — Proxy Components for Scrapy & Gerapy
• gerapy-pyppeteer — Pyppeteer Components for Scrapy & Gerapy
• gerapy-redis — Distribution Support for Scrapy & Gerapy using Redis
• gerapy-selenium — Selenium Components for Scrapy & Gerapy
• gesp — convenient scraping of german court decisions
• GFP-TEST — no summary
• gggspider — 通用采集框架。
• google-chad — Not another Google Dorking tool.
• gpt-web-crawler — A web crawler for GPTs to build knowledge bases
• Greek-scraper — Ultra-fast and efficient web scraper with GPU utilization for text cleaning and JSON output. Supports generic and language-specific scraping.
• gsoft-py-utils — python常用工具类
• gzSpiderTools — 魔改使用工具库
• habra-favorites — Sort your favorites posts from Habrahabr.ru
• haipproxy2 — High aviariable proxy pool client for crawlers.