How to get data without getting blocked in Listly
You’ve probably got blocked while scraping websites at least once. This tutorial will cover Listly’s advanced features: Group extraction and Screenshot, which allows you to check for the error type by yourself. If you’ve ever got blocked while scraping websites, you may want to check this out!
Today’s website example is Shopbop, one of the largest online shopping platforms. As always, click Listly Whole and scrape the website.
There you go, you’d be able to get all data with just a single click.
If you’re using the free plan…
If you go back to the website, you’d notice that the website displays product info by splitting its content into multiple 1, 2, 3… pages. The simple way to collect the information about fashion products is to repeat clicking Listly Whole on each webpage. It has 11 pages at best, so you can get the job done fairly easily. Listly offers unlimited 1 web page-1 Excel conversion for free, it would be accessible to anyone looking to collect product info from websites!
If you’re using the business plan…
Here’s a better way to get it done in just one minute — psst, this will make your work life much easier.
Click Group and paste the respective product list page URLs to the ADD URL section at the bottom of the page.
Except for the first page, each list page includes the following parameters: ?offset=NUMBER, with numerical values in ascending order. You can simply copy and paste the URLs or adjust the numeric values of the addresses.
Once you click Submit, Listly will automatically extract data from multiple product list pages.
Refresh your browser, then your extraction status would change as below.
Just wait for a minute until your data extraction is done.
If you encountered an error…
Aw, there’s one page that Listly seemed to fail extracting data. No worries, Listly provides the screenshot feature that allows you to check what types of errors occurred during the extraction with a user-friendly interface.
Click Screenshot and check for the error.
The error message literally displays the screenshot of the error that occurred during the scraping. Many websites create a promotional pop-up as below, and it blocks scrapping data behind the pop-up window. As the promotional messages pop up at random times, the easiest solution for this would be re-doing the extraction.
Select the URL that didn’t extract the data you wanted and click Refresh.
There you go, you can get all the data without re-extracting all the web pages! Now, you can save data in different formats.
