Web scraping using C#, HTTPClient, And HTML Agility Pack

Web scraping is a technique that scrapes information from other online sources. This is a great way to combine different sources into one source. To create a web scraper you need a few things: online sources, some code that can access the internet, and a GUI. In this article, I am going to show you how web scraping using C# works.

Goals Of This Article

In this article, I will show you how you can scrape information from the internet by using C# and .NET. We will use the HTTPClient for the online connection and the HTML Agility Pack to read the information we get from the HTTPClient response.

At the end of this article, you know how to retrieve information from a website and put that information in C# objects for you to use.

I will be using a console application called WebScraping.Console in a solution called WebScraping.

What Is Web Scraping?

I love to travel and I get my information from several sources. Some websites, some social media, and more. It can be a hassle to keep up to date with all these different online sources, so why not combine them all in one? And that is what web scraping can do.