Geek Culture
Published in

Geek Culture

Web Scraping

Web Scraping With Just One Click

Accessible, structured data at your fingertips.

Photo by charlesdeluvio on Unsplash

If you’ve ever tried to scrape data from websites, you will know there are many things that can go wrong — a website may block your IP address or change its structure entirely. And all this can happen without warning!

So what do you do if you want to get data from a website? Well, you can either hire someone with enough coding knowledge to set up a scraper for you or you can use web scraping tools.

With the rise of awesome no-code web scraping tools, you should definitely go for the latter.

That said, even though there are many web scraping tools out there, only a few of them get it right.

Some of them require advanced skills and knowledge in programming, and others might be too confusing and hard to use.

Are you looking for a simple and straightforward scraping tool?

Look no further.

Listly

Listly Homepage

Listly is an easy-to-use no-code web scraping tool with a very affordable pricing model.

With Listly, scraping web data has never been easier!

Scraping with Listly is just a 3 step process

  1. Install the Chrome Extension
  2. Scrape the website
  3. Download data as an excel file.
How Listly Works

Listly is also used by thousands of people every day worldwide.

Listly’s Features

Listly also comes packing with tons of features, notably the API integration which allows you to scrape the same website by calling the same API on Listly.

Still not convinced? Watch it in action!

Listly in Action

To show you the capabilities of Listly, we’ll apply it to 4 use-cases

  1. YouTube Videos
  2. Medium
  3. Adidas Sneakers
  4. OpenSea NFTs

1. Scraping Youtube Videos

Let’s say you’re a YouTuber, and you want to analyze MrBeast’s Youtube videos.

With Listly, you can do that with one click.

Click on Listly Whole

Then, you’ll be taken to Listly’s data extraction page.

Notice a couple of tabs were extracted, the 18 cards tab doesn’t look right, let’s check out the one with 120 cards.

And there you have it! 120 video details all scraped.

However, MrBeast definitely has more than 120 videos, let’s use Listly’s auto-scroll feature!

Go to Databoard, and hit settings.

Find the auto-scroll section, and set it to the settings you want. Let’s try 20 times.

Then, click refresh and Listly will scrape the website again for more data.

Now we wait.

Once it’s done, click Latest

And voila! We have 721 of his video details all scraped!

Uncheck arrange cell, and download the data with Excel.

Here are the first 100 rows of the data we got!

You can also opt for exporting to Google Sheets, and even the API (in beta).

Let’s check out the API!

Next time you want to scrape MrBeast’s video again, just call the API!

Instead of a particular channel, you could be interested in a particular topic, i.e. Data Science.

Want to scrape all the results instantly? Listly’s got your back!

You can even scrape YouTube comments.

You get the idea.

2. Scraping Medium.com

Here’s another scenario.

Let’s say you’re a medium writer who wants a list of all the articles you’ve published.

With Listly, it’s just a matter of clicks.

Click Listly Whole, and you come to this page like usual.

Here’s the data.

I could also scrape my article stats.

Another incredibly useful use case of Listly is to scrape trending articles on Medium.

Doing so is a great inspiration for future content writing and brainstorming.

For example, this is the trending section for the topic Data Science

We can use Listly to scrape all the titles, the authors, the subtitle, etc.

Since this is an infinite scroll, you can similarly implement the auto-scroll feature to grab more articles.

Below are the 50 articles that we scraped.

3. Scraping Adidas Sneakers

Let’s say you’re a sneakerhead, and you want to scrape all the sneakers that Adidas is selling.

adidas.com

Click Listly Whole.

We scraped 48 sneakers from Adidas!

But scrolling down, you see this.

There are 18 pages in total, you might be thinking, can Listly also scrape them?

The answer is yes.

Here’s how.

As we switch to page 2, the URL is:

https://www.adidas.com/us/men-athletic_sneakers?start=48

Save this.

Now click the red Group button.

What we can do now is paste the previous URL to the Add URL section in the settings.

Notice how you only need to change the start number? Let’s do it for the next few pages.

Then, hit the submit button.

Listly will then start scraping.

Click on it for more details about the scraping process.

After it’s done, click on Group Excel

Here’s the final data!

4. OpenSea NFTs

Now for a fun project.

Let’s scrape some cool NFTs!

The Moonbirds collection looks cool, let’s try scraping that.

OpenSea Moonbird NFTs

As always, we click on Listly Whole.

Congratulations, you know own NFTs.

Jokes aside, this can be inspiration for NFT artists for creating their own collections.

I hope these examples showed you what Listly is capable of and inspired you to use more web scraping in your life!

Convinced? Use my referral link for a whopping 10% discount!

To fully make use of Listly, below are a couple of resources to help you master it.

--

--

--

A new tech publication by Start it up (https://medium.com/swlh).

Recommended from Medium

Google Interview Question — LeetCode 1074

A Simple Guide to EDA in Python

Some comments on AB testing implementation

Graphing With Seaborn Basics

How Data Analysis Helps Unveil the Truth of Coronavirus

Text Similarity with (and without) AI

Benedict Neo

Benedict Neo

+1.5M views | Connect 👉 https://linkedin.com/in/benedictneo/

More from Medium

Data Analysis — Find a good book using Goodreads data and Python

Analyzing played track on Spotify with Python

PyScript — The power of Python in your browser! 📈