Scope of Work:
We are looking for an experienced web scraper & database developer to build a system that:
- Scrapes StubHub to find front-row tickets for concerts in low-cost cities.
- Scrapes Orbitz to check for cheap flights and hotel prices in those cities.
- Stores the collected data in a PostgreSQL database for analysis and recommendations.
Requirements:
Strong experience in web scraping (Selenium, Scrapy, BeautifulSoup, etc.)
Ability to bypass bot detection (CAPTCHAs, proxies, headers, etc.)
Experience with APIs (if applicable for StubHub & Orbitz)
Database expertise in PostgreSQL (schema design, efficient storage)
Ability to process and analyze pricing trends for future recommendations
Strong debugging & documentation skills
Deliverables:
Fully functional scraping scripts (StubHub & Orbitz)
PostgreSQL database setup and data storage
Basic analysis queries/reports for insights
Documentation & setup instructions
Please provide examples of past scraping projects and estimated completion time.
Looking forward to working with a skilled developer!