Crawling Libraries

Scrapy is a BSD-licensed fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, fro…

I myself have not used Grab for many years. I am not sure it is being used by anybody at present time. Nonetheless I decided to refactor the project, just for fun. I have annotated whole code base wi…

Ruia is still under developing, feel free to open issues and pull requests:

Sorry for this. Faster updates The following dependencies are required and must be installed separately. upgrade: (If a YouTube video has any closed captions, they will be downloaded together with th…

🔹 Main menu 🔹 Download posts 🔹 Like posts 🔹 Bulk unfollow 🔹 Bulk commenting 🔹 Bulk comment deletion

🚀 Swiss Army Knife for Bot Development 🤖 👋 Hello, I've made A LOT of bots in my life. I have written messaging bots to message people on fiverr and linkedin promoting my services. Bots's that scraped…

The project consists of three main modules: Each of these modules works independently of the rest and can be changed. Each module has a more detailed description in its corresponding directory.

This is a Webcrawler based on Scrapy and Selenium frameworks This spider crawls thru the directferries.com website in order to generate a json file with all tickets available for one of these directi…

These instructions will get you a copy of ARGUS up and running on your local machine. Follow these easy steps, which are described in more detail below, to make a successful ARGUS scraping run: ARGUS…

Scrapy tutorial to build a Craigslist crawler that scrapes engineering jobs in New York. Check this blog post for the full Scrapy tutorial: In Terminal or CMD, navigate to the main Scrapy project fol…

Features: Pomp makes no attempt to accomodate: Pomp is written and maintained by Evgeniy Tatarkin and is licensed under the BSD license.

Web scraping and automation using python

A HTML page with curated list of Python frameworks! That's it! Add a Bootstrap card element, like this: If you need a new category for Python frameworks, try this: And, update the navbar links,

A HTML page with curated list of Python frameworks! Fork and clone this repository, and run Add a Bootstrap card element, like this: If you need a new category for a Python framework, try this: