Hands-Free Web Scraping for Hands-On Marketers
Tips for dealing with “Load More”
Quick Tips
1. Visit your target website
2. Click Listly’s extension icon and select LISTLY WHOLE or LISTLY PART
3. Copy CSS Selector of “Load More” button
4. Click edit setting on Databoard
5. Paste CSS Selector to auto-click field
6. Refresh data sources
Marketers spend hours on routine tasks from market research to competitor price monitoring. There’s no getting around the need for marketing automation. Above all, we cannot talk about it without web scraping. By the end of this read, you’ll be ready to gather data from any websites using “Load More” buttons.
In this tutorial, we’ll collect women’s new product information from the H&M online market.
The website displays only 36 products at a time, and you can load more items once you click the “LOAD MORE PRODUCTS” button.
Normally you should repeat mouse clicks until there are no new products left, but you can automate this process in just 10 seconds with Listly.
Click LISTLY Whole and you’ll see that only 36 items were collected.
Move your mouse to the “LOAD MORE PRODUCTS” button on the product page, right-click, and select Inspect.
You’ll be able to see some of the code is highlighted in blue, right-click again and select Copy and Copy selector.
Go back to your Databoard and click add setting.
There you go, paste the element CSS selector to the following field. You can also change the number of times to repeat the auto-click action.
Here, we set the auto-click action to repeat 23 times and changed the wait time to 5 seconds.
- Repeat N Times: The number of times to repeat the auto-click action
- Wait after clicking: The amount of time that is given to Listly’s web scraping service after clicking a “Load More” button
All right, now it’s time to hit the refresh data button!
What do you say? Every single new item on the H&M website was collected in just one page.
You know what? You can also download the data by simply clicking the EXCEL button on your Databoard!
