At AX data technology we automate the collection of large amounts of web data. We write our own web crawling bots that run thousands of instances in parallel. We maintain accounts with a number of proxy providers that allow us to scrape data local to specific countries.
We maintain a number of bots which we use to gather data from ecommerce sites. We can collect data such as price, promotions, availability and stock levels. We can collect data with a daily frequency if required.
We maintain bots which gather search engine data at scale, bots running in parallel continuosly gathering data regarding businesses and people of interest.
Alex Kendrick, Data scientist
alex@axdata.tech