AlexPavlov 

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

Declarative high performance web scraper in C#. Easily crawl any web site and parse the data, save structed result to a file, DB, etc.

Azure Service Bus scheduler for WebReaper: a distributed job queue backed by an Azure Service Bus queue, for sharing crawl state across workers and serverless functions. Adds... More information

Azure Cosmos DB sink for WebReaper. Adds ScraperEngineBuilder.WriteToCosmosDb(...). Satellite package (ADR-0009) so the WebReaper core stays dependency-light and AOT-clean.

MongoDB adapters for WebReaper: the MongoDbSink, plus MongoDB-backed scraper-config and cookie storage. Adds ScraperEngineBuilder.WriteToMongoDb / WithMongoDbConfigStorage / WithMongoDbCookieStorage. Satellite... More information

Headless-browser (Puppeteer/Chromium) page-load transport for WebReaper, for scraping JavaScript-rendered pages (GetWithBrowser / FollowWithBrowser / PaginateWithBrowser). Adds... More information

Redis adapters for WebReaper: the Redis scheduler, visited-link tracker, sink, and Redis-backed scraper-config and cookie storage, all sharing one connection pool (ADR-0005). Adds... More information