Agora

Agora

About the project

The project's purpose is to search for ads on websites. We scrape ads from all websites and create a hash of ad images in order to check check for identical images later. Then we scrape title, text, link of each ad. 

We get text of ads using the library, allowing retrieving only the main text from website. The web application was created to display information of scraped ads. There you can filter ads by date and other criteria.

Duration2 years
ClientUnder NDA
Categorybig data & analytics
TypeWeb Application
visit website

Technologies

Technologies

Python, Django, Scrappy

We used Scrappy to get the data and Django admin panel to manage it for each customer.