Scraping Product Prices and Reviews

Success Story -

This case study reviews the success of delivering product prices and reviews from various e-commerce sites using customized product scraper developed by iWeb Scraping.

Client’s Background

One of the leading business firms in Europe, contacted iWeb Scraping for extracting product prices and reviews from various e-commerce websites.

Requirements

As company is the largest firm in its industry, it sells a variety of products online through e-commerce retailers. Because their items are distributed on a variety of platforms, they need a record of how different e-commerce customer sites price their products, as well as user/customer feedback.

Solution Delivered

Issue: The biggest issue was that client did not provide us with any URLs of the products they wanted to scrape on various e-commerce portals but they had provided SKU IDs along with the product names.

Solution: After asking for sample SKU IDs and product names, our team proposed an idea to search for the relevant products using site-wise SKU IDs which will return the desired results.

However, the client wanted authority over the input data, hence we developed a crawler based on the client’s requirement to extract product prices, reviews, and delivered this data from a shared location. Although some of these websites had a few blocking methods, obtaining both product-related data and reviews data was fairly straightforward, after we found the product pages.

Final Results

We managed to set up the crawlers as soon as the crawl parameters were finalized, and we delivered data in no time. We got the full setup, up and operating in about 7 working days for 20 websites, which equals to 40 site installations

The data is merged into a single file per website and then sent to the client's FTP folder, where it is imported into their internal operations, and automatically tracks price differences, if any, across numerous vendors.

Our client has built sentiment analysis technology to explore how people perceive their items based on the feedback received. They find this necessary in figuring out what works for them and what needs to be changed.

By employing our web crawling service, the client ensured that no manual element was involved in the data gathering process, saving a significant amount of labor hours, infrastructure costs, human capital costs of getting a dedicated staff, and crawler maintenance if it had been built in-house.