Web-scraper Libraries

The primary goal of Scrapple is to abstract the process of designing web content extractors. The focus is laid on what to extract, rather than how to do it. The user-specified configuration file cont…

Tests are (so far) only run on static html files. One of which is a linkedin profile, the other is just used to test some utility functions. Because of Linkedin's anti-scraping measures, you must mak…

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex. Python is arguably…

If you don't want to deal with installing and configuring software, I've set up this repository so that a GitHub Codespace can do all of that for you. Select Code -> Codespaces Tab -> The + Ico…

🚀 Swiss Army Knife for Bot Development 🤖 👋 Hello, I've made A LOT of bots in my life. I have written messaging bots to message people on fiverr and linkedin promoting my services. Bots's that scraped…

Scrapy tutorial to build a Craigslist crawler that scrapes engineering jobs in New York. Check this blog post for the full Scrapy tutorial: In Terminal or CMD, navigate to the main Scrapy project fol…

The repository is set up in a modular fashion so you can easily substitute in your own email address and report contents. I have personally found this to be a fun use for custom web scrapers that gra…