Adventures With Js : Crawling Website Of Nigerian Newspapers.

Out of boredom and the need to always stay updated on local news, I started working on curating news from some reliable newspapers in the country. I choose 5 news categories they are:

  • Politics
  • Business
  • Entertainment
  • Editorial
  • Sports

Three years ago when I used to assist my friends dad to deliver newspapers around 5am in the mornings, I learnt that some papers are more expensive due to their quality and their demand is not as high as the other ones, this inspired me to limit the scrapping to few websites.

  • Guardian
  • Vanguard
  • Punch
  • Sun
  • The Nation

The Code.

The websites are built with wordpress, this means all the pages for each website have a unique layout but different content. For each website there is a function to extract the content of the website based on the layout.

Thanks for reading, If you enjoy this post, please click on the ♡ button, leave comments that to help improve the code, it will be greatly appreciated.