The fastest web scraper API on Node.js — GoldenGate
Published in
27 min readJul 21, 2020
Author: Emin Muhammadi (https://muemin.me). You can contact here for ideas and suggestions.
Nowadays, the main problem is the lack of information. Think of an environment where your microservice generates data on its own. Yes, this is possible, but the main problem is that the data is too cumbersome.
The above scheme shows the working principle of GoldenGate and the relationship of the elements inside. Yes, this is an open-source project and you can fork from the link below:
Okay, this can be tedious for an introduction, let me explain the purpose of the project. Although it sounds as inexperienced as possible, the technologies used for GoldenGate’s interior elements are: Firebase Realtime Database, Google Cloud Storage, Docker, PM2, Puppeteer, JWT, Nodejs and npm packages… One of the most important features of the project is that the components can be easily changed.
Installing
You can fork the project from the repository on emigatech/goldengate or
git clone https://github.com/emigatech/goldengate.gitAfter downloading from the repository, you need to make some adjustments. You must first edit the documents in the config folder. Since a variable called secret is irreplaceable, you have to change its value with a longer expression.
Second, you must put the service keys in json format in the google-cloud folder to run Google Cloud Storage and Firebase. For example,
How it works
curl -X POST "https://127.0.0.1:5000/api/auth/login" \
-H "accept: application/json" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "email=myemail@fakemail.com&password=mypassword"You must have JWT tokens to use the service endpoints. So you have to get one using the expression above. If you do not have user data, create a new user. For detailed information about API, please visit to read Swagger UI docs.
{
"status": 200,
"time": 0,
"message": "authenticated",
"data": {
"id": "88bef8463206d4bdd46be031829c09e3",
"email": "myemail@fakemail.com",
"token": "token.token.token",
"firstname": "GoldenGate",
"lastname": "User"
}
}Once you have the token, you need to add it to the header Authorization: Bearer token.token.token. After doing all this, you can now use authorized endpoints. I would like to remind you that short-term Public APIs can be used. To do this, click here
As an example, I scrape the https://bbc.com website using the scrape endpoint
curl -X POST "https://127.0.0.1:5000/api/scrape" \
-H "accept: application/json" \
-H "Authorization: Bearer token.token.token" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "url=https%3A%2F%2Fbbc.com"For reponse is :
{
"status": 201,
"time": 0,
"message": "success",
"url": "https://bbc.com",
"key": "e375c041-dd06-4caf-982e-34515e10bfc9",
"data": {
"seo": {
"title": {
"meta": [
"BBC - Homepage"
],
"og": [
"BBC - Homepage"
],
"twitter": []
},
"description": {
"meta": [
"Breaking news, sport, TV, radio and a whole lot more.\n The BBC informs, educates and entertains - wherever you are, whatever your age."
],
"og": [],
"twitter": []
},
"canonical": {
"meta": [
"https://www.bbc.com/"
],
"og": [
"https://www.bbc.com/"
]
},
"image": {
"og": [],
"twitter": []
}
},
"url": [
{
"text": "Homepage",
"link": "https://www.bbc.co.uk"
},
{
"text": "Skip to content",
"link": "https://bbc.com/#orb-modules"
},
{
"text": "Accessibility Help",
"link": "https://www.bbc.co.uk/accessibility/"
},
{
"text": "BBC Account",
"link": "https://account.bbc.com/account"
},
{
"text": "Notifications",
"link": "https://bbc.com/#"
},
{
"text": "Home",
"link": "https://www.bbc.co.uk"
},
{
"text": "News",
"link": "https://www.bbc.co.uk/news"
},
{
"text": "Sport",
"link": "https://www.bbc.co.uk/sport"
},
{
"text": "Weather",
"link": "https://www.bbc.co.uk/weather"
},
{
"text": "iPlayer",
"link": "https://www.bbc.co.uk/iplayer"
},
{
"text": "Sounds",
"link": "https://www.bbc.co.uk/sounds"
},
{
"text": "CBBC",
"link": "https://www.bbc.co.uk/cbbc"
},
{
"text": "CBeebies",
"link": "https://www.bbc.co.uk/cbeebies"
},
{
"text": "Food",
"link": "https://www.bbc.co.uk/food"
},
{
"text": "Bitesize",
"link": "https://www.bbc.co.uk/bitesize"
},
{
"text": "Arts",
"link": "https://www.bbc.co.uk/arts"
},
{
"text": "Taster",
"link": "https://www.bbc.co.uk/taster"
},
{
"text": "Local",
"link": "https://www.bbc.co.uk/news/localnews"
},
{
"text": "Three",
"link": "https://www.bbc.co.uk/bbcthree"
},
{
"text": "Menu",
"link": "https://bbc.com/#orb-footer"
},
{
"text": "Search",
"link": "https://search.bbc.co.uk/search"
},
{
"text": "UK 'badly underestimated' Russian threat",
"link": "https://bbc.com/news/uk-politics-53484344"
},
{
"text": "UK Politics",
"link": "https://bbc.com/news/politics"
},
{
"text": "UK 'badly underestimated' Russian threat",
"link": "https://bbc.com/news/uk-politics-53484344"
},
{
"text": "Why are Americans so angry about masks?",
"link": "https://www.bbc.com/news/world-us-canada-53477121"
},
{
"text": "US",
"link": "https://bbc.com/news/world/us_and_canada"
},
{
"text": "Why are Americans so angry about masks?",
"link": "https://www.bbc.com/news/world-us-canada-53477121"
},
{
"text": "Netflix cancels Turkish show in gay character row",
"link": "https://www.bbc.com/news/entertainment-arts-53489296"
},
{
"text": "Entertainment & Arts",
"link": "https://bbc.com/news/entertainment_and_arts"
},
{
"text": "Netflix cancels Turkish show in gay character row",
"link": "https://www.bbc.com/news/entertainment-arts-53489296"
},
{
"text": "The near-death experience of a genius",
"link": "https://www.bbc.com/culture/article/20200720-beethoven-250-the-ultimate-song-of-health-after-illness"
},
{
"text": "Culture",
"link": "http://www.bbc.com/culture"
},
{
"text": "The near-death experience of a genius",
"link": "https://www.bbc.com/culture/article/20200720-beethoven-250-the-ultimate-song-of-health-after-illness"
},
{
"text": "Why some people never get bored",
"link": "https://www.bbc.com/future/article/20200720-what-the-mysterious-boredom-divide-teaches-us"
},
{
"text": "Future",
"link": "http://www.bbc.com/future"
},
{
"text": "Why some people never get bored",
"link": "https://www.bbc.com/future/article/20200720-what-the-mysterious-boredom-divide-teaches-us"
},
{
"text": "News",
"link": "https://bbc.com/news"
},
{
"text": "Ukraine police free hostages in bus stand-off",
"link": "https://bbc.com/news/world-europe-53489527"
},
{
"text": "Europe",
"link": "https://bbc.com/news/world/europe"
},
{
"text": "Ukraine police free hostages in bus stand-off",
"link": "https://bbc.com/news/world-europe-53489527"
},
{
"text": "Ex-Trump lawyer sues government over 'gag order'",
"link": "https://bbc.com/news/world-us-canada-53489390"
},
{
"text": "US & Canada",
"link": "https://bbc.com/news/world/us_and_canada"
},
{
"text": "Ex-Trump lawyer sues government over 'gag order'",
"link": "https://bbc.com/news/world-us-canada-53489390"
},
{
"text": "Woman takes selfie as wild bear sniffs her hair",
"link": "https://bbc.com/news/world-us-canada-53482294"
},
{
"text": "Latin America & Caribbean",
"link": "https://bbc.com/news/world/latin_america"
},
{
"text": "Woman takes selfie as wild bear sniffs her hair",
"link": "https://bbc.com/news/world-us-canada-53482294"
},
{
"text": "Sport",
"link": "https://bbc.com/sport"
},
{
"text": "Premier League: Aston Villa 1-0 Arsenal - Trezeguet with huge goal",
"link": "https://bbc.com/sport/live/football/53287562"
},
{
"text": "Football",
"link": "https://bbc.com/sport/football"
},
{
"text": "Premier League: Aston Villa 1-0 Arsenal - Trezeguet with huge goal",
"link": "https://bbc.com/sport/live/football/53287562"
},
{
"text": "Sterling-inspired Man City hammer Watford",
"link": "https://bbc.com/sport/football/53399056"
},
{
"text": "Football",
"link": "https://bbc.com/sport/football"
},
{
"text": "Sterling-inspired Man City hammer Watford",
"link": "https://bbc.com/sport/football/53399056"
},
{
"text": "Did De Gea peak in 2017? The numbers behind his rise and fall",
"link": "https://bbc.com/sport/football/53473330"
},
{
"text": "Football",
"link": "https://bbc.com/sport/football"
},
{
"text": "Did De Gea peak in 2017? The numbers behind his rise and fall",
"link": "https://bbc.com/sport/football/53473330"
},
{
"text": "Edit",
"link": "https://bbc.com/#"
},
{
"text": "Tue 13°C",
"link": "https://bbc.com/weather/2643743?day=0"
},
{
"text": "Wed 25°C 14°C",
"link": "https://bbc.com/weather/2643743?day=1"
},
{
"text": "Thu 22°C 15°C",
"link": "https://bbc.com/weather/2643743?day=2"
},
{
"text": "Fri 25°C 16°C",
"link": "https://bbc.com/weather/2643743?day=3"
},
{
"text": "Visit Reel",
"link": "https://bbc.com/reel"
},
{
"text": "The surprising benefits of sleeping with bees",
"link": "https://bbc.com/reel/video/p08lb5l4/the-therapeutic-sound-of-slovenian-bees"
},
{
"text": "How the British military hired a magician to outwit the Nazis",
"link": "https://bbc.com/reel/video/p08l3m3q/did-this-british-magician-defeat-the-nazis-with-magic-"
},
{
"text": "The physics that suggests we have no free will",
"link": "https://bbc.com/reel/video/p086tg3k/the-physics-that-suggests-we-have-no-free-will"
},
{
"text": "The world’s most self-reliant ‘nation’?",
"link": "https://www.bbc.com/travel/story/20200721-nakhchivan-the-worlds-most-sustainable-nation"
},
{
"text": "Travel",
"link": "http://www.bbc.com/travel"
},
{
"text": "The world’s most self-reliant ‘nation’?",
"link": "https://www.bbc.com/travel/story/20200721-nakhchivan-the-worlds-most-sustainable-nation"
},
{
"text": "The greatest summer novels ever written",
"link": "https://www.bbc.com/culture/article/20200717-what-makes-a-great-summer-novel"
},
{
"text": "Culture",
"link": "http://www.bbc.com/culture"
},
{
"text": "The greatest summer novels ever written",
"link": "https://www.bbc.com/culture/article/20200717-what-makes-a-great-summer-novel"
},
{
"text": "Why planning can reduce pandemic stress",
"link": "https://www.bbc.com/worklife/article/20200720-how-planning-helps-us-cope-with-uncertainty"
},
{
"text": "Worklife",
"link": "http://www.bbc.com/worklife"
},
{
"text": "Why planning can reduce pandemic stress",
"link": "https://www.bbc.com/worklife/article/20200720-how-planning-helps-us-cope-with-uncertainty"
},
{
"text": "The undetected Covid-19 immunity",
"link": "https://www.bbc.com/future/article/20200716-the-people-with-hidden-protection-from-covid-19"
},
{
"text": "Future",
"link": "http://www.bbc.com/future"
},
{
"text": "The undetected Covid-19 immunity",
"link": "https://www.bbc.com/future/article/20200716-the-people-with-hidden-protection-from-covid-19"
},
{
"text": "Where women are kidnapped to be brides",
"link": "https://www.bbc.com/news/world-asia-53418099"
},
{
"text": "Asia",
"link": "https://bbc.com/news/world/asia"
},
{
"text": "Where women are kidnapped to be brides",
"link": "https://www.bbc.com/news/world-asia-53418099"
},
{
"text": "Greta to donate one-million-euro prize",
"link": "https://www.bbc.com/news/science-environment-53477604"
},
{
"text": "Science & Environment",
"link": "https://bbc.com/news/science_and_environment"
},
{
"text": "Greta to donate one-million-euro prize",
"link": "https://www.bbc.com/news/science-environment-53477604"
},
{
"text": "Lockdown Brits splurge on tea and biscuits",
"link": "https://www.bbc.com/news/business-53484355"
},
{
"text": "Business",
"link": "https://bbc.com/news/business"
},
{
"text": "Lockdown Brits splurge on tea and biscuits",
"link": "https://www.bbc.com/news/business-53484355"
},
{
"text": "1 Ryanair to close base after pilots reject pay cut",
"link": "https://bbc.com/news/business-53485673"
},
{
"text": "2 EU reaches 'pivotal' coronavirus recovery deal",
"link": "https://bbc.com/news/world-europe-53481542"
},
{
"text": "3 Apple makes 2030 carbon-neutral pledge",
"link": "https://bbc.com/news/technology-53485560"
},
{
"text": "4 UK quarterly borrowing hits record high",
"link": "https://bbc.com/news/business-53482895"
},
{
"text": "Future Planet",
"link": "https://bbc.com/future/future-planet"
},
{
"text": "The most powerful renewable energy",
"link": "https://bbc.com/future/article/20200713-the-most-powerful-renewable-energy"
},
{
"text": "The spies hunting fish pirates",
"link": "https://bbc.com/future/article/20200708-the-albatrosses-who-catch-pirates-on-the-high-seas"
},
{
"text": "The strongest law on climate yet",
"link": "https://bbc.com/future/article/20200706-the-law-that-could-make-climate-change-illegal"
},
{
"text": "Technology of Business",
"link": "https://www.bbc.com/news/business-11428889"
},
{
"text": "Business",
"link": "https://bbc.com/news/business"
},
{
"text": "The stress of leading a start-up through the pandemic",
"link": "https://www.bbc.com/news/business-53366436"
},
{
"text": "The stress of leading a start-up through the pandemic",
"link": "https://www.bbc.com/news/business-53366436"
},
{
"text": "Follow the food",
"link": "https://www.bbc.com/future/bespoke/follow-the-food/index.html"
},
{
"text": "Future",
"link": "http://www.bbc.com/future"
},
{
"text": "The people creating an oasis with seawater",
"link": "https://www.bbc.com/future/bespoke/follow-the-food/the-people-creating-an-oasis-with-seawater.html"
},
{
"text": "The people creating an oasis with seawater",
"link": "https://www.bbc.com/future/bespoke/follow-the-food/the-people-creating-an-oasis-with-seawater.html"
},
{
"text": "'Fighting to prove we're British'",
"link": "https://bbc.com/news/uk-53477811"
},
{
"text": "UK",
"link": "https://bbc.com/news/uk"
},
{
"text": "'Fighting to prove we're British'",
"link": "https://bbc.com/news/uk-53477811"
},
{
"text": "Recommended",
"link": "https://bbc.com/#"
},
{
"text": "Most Watched",
"link": "https://bbc.com/#"
},
{
"text": "Latest",
"link": "https://bbc.com/#"
},
{
"text": "'Fighting to prove we're British'",
"link": "https://bbc.com/news/uk-53477811"
},
{
"text": "UK",
"link": "https://bbc.com/news/uk"
},
{
"text": "'Fighting to prove we're British'",
"link": "https://bbc.com/news/uk-53477811"
},
{
"text": "Hope probe: UAE launches Mars mission",
"link": "https://bbc.com/news/science-environment-53467616"
},
{
"text": "Science & Environment",
"link": "https://bbc.com/news/science_and_environment"
},
{
"text": "Hope probe: UAE launches Mars mission",
"link": "https://bbc.com/news/science-environment-53467616"
},
{
"text": "ICYMI: Luxury grapes and a 'criminal'...",
"link": "https://bbc.com/news/world-53450251"
},
{
"text": "World",
"link": "https://bbc.com/news/world"
},
{
"text": "ICYMI: Luxury grapes and a 'criminal' kangaroo",
"link": "https://bbc.com/news/world-53450251"
},
{
"text": "'We're doing a dangerous and amazing job'",
"link": "https://bbc.com/news/world-africa-53478150"
},
{
"text": "Africa",
"link": "https://bbc.com/news/world/africa"
},
{
"text": "'We're doing a dangerous and amazing job'",
"link": "https://bbc.com/news/world-africa-53478150"
},
{
"text": "Spectacular views from new tourist train",
"link": "https://bbc.com/news/uk-53484392"
},
{
"text": "UK",
"link": "https://bbc.com/news/uk"
},
{
"text": "Spectacular views from new tourist train",
"link": "https://bbc.com/news/uk-53484392"
},
{
"text": "Ready for the total solar eclipse?",
"link": "https://bbc.com/news/world-us-canada-40362067"
},
{
"text": "US & Canada",
"link": "https://bbc.com/news/world/us_and_canada"
},
{
"text": "Ready for the total solar eclipse?",
"link": "https://bbc.com/news/world-us-canada-40362067"
},
{
"text": "Great Mosque of al-Nuri destroyed",
"link": "https://bbc.com/news/world-middle-east-40369547"
},
{
"text": "Middle East",
"link": "https://bbc.com/news/world/middle_east"
},
{
"text": "Great Mosque of al-Nuri destroyed",
"link": "https://bbc.com/news/world-middle-east-40369547"
},
{
"text": "Video 'shows Mosul mosque's destruction'",
"link": "https://bbc.com/news/world-middle-east-40366136"
},
{
"text": "Middle East",
"link": "https://bbc.com/news/world/middle_east"
},
{
"text": "Video 'shows Mosul mosque's destruction'",
"link": "https://bbc.com/news/world-middle-east-40366136"
},
{
"text": "The rise of the female fitness model",
"link": "https://bbc.com/news/uk-england-40058866"
},
{
"text": "England",
"link": "https://bbc.com/news/england"
},
{
"text": "The rise of the female fitness model",
"link": "https://bbc.com/news/uk-england-40058866"
},
{
"text": "Car burglar caught with his pants down",
"link": "https://bbc.com/news/world-us-canada-40360953"
},
{
"text": "US & Canada",
"link": "https://bbc.com/news/world/us_and_canada"
},
{
"text": "Car burglar caught with his pants down",
"link": "https://bbc.com/news/world-us-canada-40360953"
},
{
"text": "One-minute World News",
"link": "https://bbc.com/news/10462520"
},
{
"text": "Video",
"link": "https://bbc.com/news/video_and_audio"
},
{
"text": "One-minute World News",
"link": "https://bbc.com/news/10462520"
},
{
"text": "'There is no such concentration camp in...",
"link": "https://bbc.com/news/world-asia-53463242"
},
{
"text": "Asia",
"link": "https://bbc.com/news/world/asia"
},
{
"text": "'There is no such concentration camp in Xinjiang'",
"link": "https://bbc.com/news/world-asia-53463242"
},
{
"text": "Ten-year-old punk publisher's surprise...",
"link": "https://bbc.com/news/entertainment-arts-53480366"
},
{
"text": "Entertainment & Arts",
"link": "https://bbc.com/news/entertainment_and_arts"
},
{
"text": "Ten-year-old punk publisher's surprise success",
"link": "https://bbc.com/news/entertainment-arts-53480366"
},
{
"text": "Job lessons from 2008 for the class of 2020",
"link": "https://bbc.com/news/business-53397576"
},
{
"text": "Business",
"link": "https://bbc.com/news/business"
},
{
"text": "Job lessons from 2008 for the class of 2020",
"link": "https://bbc.com/news/business-53397576"
},
{
"text": "BBC World Service",
"link": "http://www.bbc.co.uk/worldserviceradio/"
},
{
"text": "On Air: Newshour",
"link": "http://www.bbc.co.uk/programmes/p002vsnk"
},
{
"text": "Dua Lipa courts controversy with nationalist tweet",
"link": "https://www.bbc.com/news/world-europe-53483451"
},
{
"text": "Europe",
"link": "https://bbc.com/news/world/europe"
},
{
"text": "Dua Lipa courts controversy with nationalist tweet",
"link": "https://www.bbc.com/news/world-europe-53483451"
},
{
"text": "LinkedIn to cut 960 jobs as recruitment falls",
"link": "https://bbc.com/news/business-53484764"
},
{
"text": "Business",
"link": "https://bbc.com/news/business"
},
{
"text": "LinkedIn to cut 960 jobs as recruitment falls",
"link": "https://bbc.com/news/business-53484764"
},
{
"text": "Elon Musk offered Amber Heard '24/7 security'",
"link": "https://bbc.com/news/uk-53489105"
},
{
"text": "UK",
"link": "https://bbc.com/news/uk"
},
{
"text": "Elon Musk offered Amber Heard '24/7 security'",
"link": "https://bbc.com/news/uk-53489105"
},
{
"text": "US report accuses China of 'digital authoritarianism'",
"link": "https://bbc.com/news/technology-53490042"
},
{
"text": "Technology",
"link": "https://bbc.com/news/technology"
},
{
"text": "US report accuses China of 'digital authoritarianism'",
"link": "https://bbc.com/news/technology-53490042"
},
{
"text": "River Thames 'severely polluted with plastic'",
"link": "https://bbc.com/news/science-environment-53479635"
},
{
"text": "Science & Environment",
"link": "https://bbc.com/news/science_and_environment"
},
{
"text": "River Thames 'severely polluted with plastic'",
"link": "https://bbc.com/news/science-environment-53479635"
},
{
"text": "Oxford coronavirus vaccine triggers immune response",
"link": "https://bbc.com/news/uk-53469839"
},
{
"text": "UK",
"link": "https://bbc.com/news/uk"
},
{
"text": "Oxford coronavirus vaccine triggers immune response",
"link": "https://bbc.com/news/uk-53469839"
},
{
"text": "Coronavirus not over by Christmas, experts warn",
"link": "https://bbc.com/news/uk-53488142"
},
{
"text": "Health",
"link": "https://bbc.com/news/health"
},
{
"text": "Coronavirus not over by Christmas, experts warn",
"link": "https://bbc.com/news/uk-53488142"
},
{
"text": "Churchill through the eyes of India By Yogita Limaye",
"link": "https://www.bbc.com/news/world-asia-india-53405121"
},
{
"text": "England's test and trace scheme 'breaks data law' By Rory Cellan-Jones",
"link": "https://www.bbc.com/news/technology-53466471"
},
{
"text": "Encouraging results in coronavirus vaccine trials By Fergus Walsh",
"link": "https://www.bbc.com/news/health-53426367"
},
{
"text": "What has happened to the report into Priti Patel? By Laura Kuenssberg",
"link": "https://www.bbc.com/news/uk-politics-53422019"
},
{
"text": "Covid fears putting off Chinese students from UK By Sean Coughlan",
"link": "https://www.bbc.com/news/education-53416337"
},
{
"text": "UK ministers shift the message on face coverings By Laura Kuenssberg",
"link": "https://www.bbc.com/news/uk-politics-53396422"
},
{
"text": "'Hope' and ambition drive UAE's Mars mission By Jonathan Amos",
"link": "https://www.bbc.com/news/science-environment-53394737"
},
{
"text": "Wounds of Dutch history expose deep racial divide By Anna Holligan",
"link": "https://www.bbc.co.uk/news/world-europe-53261944"
},
{
"text": "The Boss",
"link": "https://www.bbc.com/news/business"
},
{
"text": "'I was standing at the wrong end of a machine gun'",
"link": "https://www.bbc.com/news/business-53346196"
},
{
"text": "Business",
"link": "https://bbc.com/news/business"
},
{
"text": "'I was standing at the wrong end of a machine gun'",
"link": "https://www.bbc.com/news/business-53346196"
},
{
"text": "Designed",
"link": "https://www.bbc.com/culture/designed"
},
{
"text": "The craft taking the world by storm",
"link": "https://www.bbc.com/culture/article/20200714-why-glamorous-glassware-is-booming"
},
{
"text": "Culture",
"link": "http://www.bbc.com/culture"
},
{
"text": "The craft taking the world by storm",
"link": "https://www.bbc.com/culture/article/20200714-why-glamorous-glassware-is-booming"
},
{
"text": "World in pictures",
"link": "https://bbc.com/news/in_pictures"
},
{
"text": "Portrait of life under Covid for visually impaired",
"link": "https://bbc.com/news/in-pictures-53403780"
},
{
"text": "In Pictures",
"link": "https://bbc.com/news/in_pictures"
},
{
"text": "Portrait of life under Covid for visually impaired",
"link": "https://bbc.com/news/in-pictures-53403780"
},
{
"text": "Closest ever pictures taken of the Sun",
"link": "https://bbc.com/news/science-environment-53429054"
},
{
"text": "Science & Environment",
"link": "https://bbc.com/news/science_and_environment"
},
{
"text": "Closest ever pictures taken of the Sun",
"link": "https://bbc.com/news/science-environment-53429054"
},
{
"text": "Africa's top shots: Handprints and sea spray",
"link": "https://bbc.com/news/world-africa-53432147"
},
{
"text": "Africa",
"link": "https://bbc.com/news/world/africa"
},
{
"text": "Africa's top shots: Handprints and sea spray",
"link": "https://bbc.com/news/world-africa-53432147"
},
{
"text": "How coronavirus swept through Brazil",
"link": "https://bbc.com/news/world-latin-america-53429430"
},
{
"text": "Latin America & Caribbean",
"link": "https://bbc.com/news/world/latin_america"
},
{
"text": "How coronavirus swept through Brazil",
"link": "https://bbc.com/news/world-latin-america-53429430"
},
{
"text": "Waving goodbye to a coastal landmark",
"link": "https://bbc.com/news/uk-england-suffolk-53263525"
},
{
"text": "Suffolk",
"link": "https://bbc.com/news/england/suffolk"
},
{
"text": "Waving goodbye to a coastal landmark",
"link": "https://bbc.com/news/uk-england-suffolk-53263525"
},
{
"text": "BBC in other languages",
"link": "https://bbc.com/worldserviceradio"
},
{
"text": "Persian",
"link": "https://bbc.com/persian/"
},
{
"text": "چین و ایران؛ موضوع مذاکرات 'صریح' پومپئو در لندن",
"link": "https://bbc.com/persian/iran-53491989"
},
{
"text": "Spanish",
"link": "https://bbc.com/mundo/"
},
{
"text": "\"El mundo está tratando los síntomas de la pandemia, pero no las causas\"",
"link": "https://bbc.com/mundo/noticias-53435056"
},
{
"text": "Urdu",
"link": "https://bbc.com/urdu/"
},
{
"text": "اسلام آباد سے اغوا کیے گئے صحافی مطیع اللہ جان گھر واپس پہنچ گئے",
"link": "https://bbc.com/urdu/pakistan-53482486"
},
{
"text": "Vietnamese",
"link": "https://bbc.com/vietnamese/"
},
{
"text": "Dự án Cần Giờ: Đừng để có lỗi với nhân dân",
"link": "https://bbc.com/vietnamese/forum-53491052"
},
{
"text": "More Languages",
"link": "https://bbc.com/ws/languages"
},
{
"text": "Arabic عربي",
"link": "https://bbc.com/arabic/"
},
{
"text": "Azeri AZƏRBAYCAN",
"link": "https://bbc.com/azeri/"
},
{
"text": "Bangla বাংলা",
"link": "https://bbc.com/bengali/"
},
{
"text": "Burmese မြန်မာ",
"link": "https://bbc.com/burmese/"
},
{
"text": "Chinese 中文网",
"link": "https://bbc.com/zhongwen/simp/"
},
{
"text": "French AFRIQUE",
"link": "https://bbc.com/afrique/"
},
{
"text": "Hausa HAUSA",
"link": "https://bbc.com/hausa/"
},
{
"text": "Hindi हिन्दी",
"link": "https://bbc.com/hindi/"
},
{
"text": "Indonesian INDONESIA",
"link": "https://bbc.com/indonesia/"
},
{
"text": "Japanese 日本語",
"link": "https://bbc.com/japanese/"
},
{
"text": "Kinyarwanda GAHUZA",
"link": "https://bbc.com/gahuza/"
},
{
"text": "Kirundi KIRUNDI",
"link": "https://bbc.com/gahuza/"
},
{
"text": "Kyrgyz Кыргыз",
"link": "https://bbc.com/kyrgyz/"
},
{
"text": "Marathi मराठी",
"link": "https://bbc.com/marathi/"
},
{
"text": "Nepali नेपाली",
"link": "https://bbc.com/nepali/"
},
{
"text": "Pashto پښتو",
"link": "https://bbc.com/pashto/"
},
{
"text": "Persian فارسی",
"link": "https://bbc.com/persian/"
},
{
"text": "Portuguese BRASIL",
"link": "https://bbc.com/portuguese/"
},
{
"text": "Russian НА РУССКОМ",
"link": "https://bbc.com/russian/"
},
{
"text": "Sinhala සිංහල",
"link": "https://bbc.com/sinhala/"
},
{
"text": "Somali SOMALI",
"link": "https://bbc.com/somali/"
},
{
"text": "Spanish MUNDO",
"link": "https://bbc.com/mundo/"
},
{
"text": "Swahili SWAHILI",
"link": "https://bbc.com/swahili/"
},
{
"text": "Tamil தமிழ்",
"link": "https://bbc.com/tamil/"
},
{
"text": "Turkish TÜRKÇE",
"link": "https://bbc.com/turkce/"
},
{
"text": "Ukrainian УКРАЇНСЬКA",
"link": "https://bbc.com/ukrainian/"
},
{
"text": "Urdu اردو",
"link": "https://bbc.com/urdu/"
},
{
"text": "Uzbek O'ZBEK",
"link": "https://bbc.com/uzbek/"
},
{
"text": "Vietnamese TIẾNG VIỆT",
"link": "https://bbc.com/vietnamese/"
},
{
"text": "Home",
"link": "https://www.bbc.co.uk"
},
{
"text": "News",
"link": "https://www.bbc.co.uk/news"
},
{
"text": "Sport",
"link": "https://www.bbc.co.uk/sport"
},
{
"text": "Weather",
"link": "https://www.bbc.co.uk/weather"
},
{
"text": "iPlayer",
"link": "https://www.bbc.co.uk/iplayer"
},
{
"text": "Sounds",
"link": "https://www.bbc.co.uk/sounds"
},
{
"text": "CBBC",
"link": "https://www.bbc.co.uk/cbbc"
},
{
"text": "CBeebies",
"link": "https://www.bbc.co.uk/cbeebies"
},
{
"text": "Food",
"link": "https://www.bbc.co.uk/food"
},
{
"text": "Bitesize",
"link": "https://www.bbc.co.uk/bitesize"
},
{
"text": "Arts",
"link": "https://www.bbc.co.uk/arts"
},
{
"text": "Taster",
"link": "https://www.bbc.co.uk/taster"
},
{
"text": "Local",
"link": "https://www.bbc.co.uk/news/localnews"
},
{
"text": "Three",
"link": "https://www.bbc.co.uk/bbcthree"
},
{
"text": "Terms of Use",
"link": "https://www.bbc.co.uk/usingthebbc/terms/"
},
{
"text": "About the BBC",
"link": "https://www.bbc.co.uk/aboutthebbc"
},
{
"text": "Privacy Policy",
"link": "https://www.bbc.co.uk/usingthebbc/privacy/"
},
{
"text": "Cookies",
"link": "https://www.bbc.co.uk/usingthebbc/cookies/"
},
{
"text": "Accessibility Help",
"link": "https://www.bbc.co.uk/accessibility/"
},
{
"text": "Parental Guidance",
"link": "https://www.bbc.co.uk/guidance"
},
{
"text": "Contact the BBC",
"link": "https://www.bbc.co.uk/contact"
},
{
"text": "Get Personalised Newsletters",
"link": "https://www.bbc.co.uk/bbcnewsletter"
},
{
"text": "Read about our approach to external linking.",
"link": "https://www.bbc.co.uk/help/web/links/"
}
],
"media": {
"img": [
{
"alt": "",
"link": "https://a1.api.bbc.co.uk/hit.xiti?&col=1&from=p&ptag=js&s=598253&p=home.page&x2=[responsive]&x3=[bbc_website]&x4=[en]&x7=[Index-home]&x8=[reverb-1.4.0-nojs]&x11=[HOMEPAGE_GNL]&x12=[GNL_HOMEPAGE]"
},
{
"alt": "President Putin",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/6BD6/production/_113560672_2_putin_getty_images.jpg"
},
{
"alt": "Trump supporter wearing a mask",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/131B9/production/_113556287_trump_getty.jpg"
},
{
"alt": "If Only writer Ece Yorenc",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/1992/production/_113564560_ece_get.jpg"
},
{
"alt": "Painting of Beethoven",
"link": "https://ichef.bbc.co.uk/wwhp/144/ibroadcast/images/live/p0/8l/bh/p08lbhx6.jpg"
},
{
"alt": "Close up of man's face in space helmet",
"link": "https://ichef.bbc.co.uk/wwhp/144/ibroadcast/images/live/p0/8l/fm/p08lfmqt.jpg"
},
{
"alt": "Security forces stand over detained gunman",
"link": "https://bbc.com/"
},
{
"alt": "Michael Cohen, the former lawyer for US President Donald Trump, leaves his apartment to begin serving a three-year sentence",
"link": "https://bbc.com/"
},
{
"alt": "A brown bear paws at a women's calf",
"link": "https://bbc.com/"
},
{
"alt": "Villa score",
"link": "https://bbc.com/"
},
{
"alt": "Raheem Sterling (right) celebrates with team-mates",
"link": "https://bbc.com/"
},
{
"alt": "David de Gea",
"link": "https://bbc.com/"
},
{
"alt": "Partly Cloudy",
"link": "https://static.bbci.co.uk/weather/0.5.284/images/icons/individual_56_icons/en_on_light_bg/2.gif"
},
{
"alt": "Sunny Intervals",
"link": "https://static.bbci.co.uk/weather/0.5.284/images/icons/individual_56_icons/en_on_light_bg/3.gif"
},
{
"alt": "Sunny Intervals",
"link": "https://static.bbci.co.uk/weather/0.5.284/images/icons/individual_56_icons/en_on_light_bg/3.gif"
},
{
"alt": "Sunny Intervals",
"link": "https://static.bbci.co.uk/weather/0.5.284/images/icons/individual_56_icons/en_on_light_bg/3.gif"
},
{
"alt": "The surprising benefits of sleeping with bees",
"link": "https://bbc.com/"
},
{
"alt": "How the British military hired a magician to outwit the Nazis",
"link": "https://bbc.com/"
},
{
"alt": "The physics that suggests we have no free will",
"link": "https://bbc.com/"
},
{
"alt": "This locavore lover's paradise be the planet’s next great food destination",
"link": "https://bbc.com/"
},
{
"alt": "From The Go-Between to Call Me By Your Name, stories that raise the temperature",
"link": "https://bbc.com/"
},
{
"alt": "The process of planning can help you handle uncertainty",
"link": "https://bbc.com/"
},
{
"alt": "Some people had a secret store of immunity before the virus even existed",
"link": "https://bbc.com/"
},
{
"alt": "Women with baskets ready for rituals at the main Pasola Festival",
"link": "https://bbc.com/"
},
{
"alt": "Greta Thunberg",
"link": "https://bbc.com/"
},
{
"alt": "Woman dips biscuit in tea",
"link": "https://bbc.com/"
},
{
"alt": "The most powerful renewable energy",
"link": "https://bbc.com/"
},
{
"alt": "The spies hunting fish pirates",
"link": "https://bbc.com/"
},
{
"alt": "The strongest law on climate yet",
"link": "https://bbc.com/"
},
{
"alt": "Priya Lakhani, founder Century Tech",
"link": "https://bbc.com/"
},
{
"alt": "seawater greenhouse",
"link": "https://bbc.com/"
},
{
"alt": "Three people fighting to be recognised as British",
"link": "https://bbc.com/"
},
{
"alt": "Three people fighting to be recognised as British",
"link": "https://bbc.com/"
},
{
"alt": "Hope probe launch",
"link": "https://bbc.com/"
},
{
"alt": "Kangaroo being taken to a police car",
"link": "https://bbc.com/"
},
{
"alt": "Claudia Mangwegape",
"link": "https://bbc.com/"
},
{
"alt": "Train on viaduct",
"link": "https://bbc.com/"
},
{
"alt": "Solar eclipse to cross North America",
"link": "https://bbc.com/"
},
{
"alt": "Blast in Iraq destroys Great Mosque of al-Nuri according to Iraqi forces",
"link": "https://bbc.com/"
},
{
"alt": "File image of Mosul's Great Mosque of al-Nuri and video purportedly showing it being blown up on 21 June 2017",
"link": "https://bbc.com/"
},
{
"alt": "Adrienn Varga",
"link": "https://bbc.com/"
},
{
"alt": "Car thief",
"link": "https://bbc.com/"
},
{
"alt": "BBC News",
"link": "https://bbc.com/"
},
{
"alt": "China's ambassador to the UK, Liu Xiaoming",
"link": "https://bbc.com/"
},
{
"alt": "Arlo Lippiatt",
"link": "https://bbc.com/"
},
{
"alt": "Young people",
"link": "https://bbc.com/"
},
{
"alt": "Dua Lipa at the 2019 Grammy Awards",
"link": "https://bbc.com/"
},
{
"alt": "LinkedIn",
"link": "https://bbc.com/"
},
{
"alt": "A composite image of Johnny Depp and Amber Heard",
"link": "https://bbc.com/"
},
{
"alt": "Man with phone",
"link": "https://bbc.com/"
},
{
"alt": "Plastic, Thames Estuary",
"link": "https://bbc.com/"
},
{
"alt": "Woman being vaccinated",
"link": "https://bbc.com/"
},
{
"alt": "Coronavirus testing",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "",
"link": "https://bbc.com/"
},
{
"alt": "Sir Jack Petchey",
"link": "https://bbc.com/"
},
{
"alt": "the art of glassmaking",
"link": "https://bbc.com/"
},
{
"alt": "A blind woman stands with her walking cane whilst wearing a face mask and gloves",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/5B46/production/_113366332_3_lookdown_04.jpg"
},
{
"alt": "Ultraviolet image of the Sun",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/113AC/production/_113427507_solar_orbiter_eui-fullsun01.jpg"
},
{
"alt": "A red handprint on a cave wall.",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/F96C/production/_113425836_tv062458845.jpg"
},
{
"alt": "Aerial view showing graves in the Nossa Senhora Aparecida cemetery in Manaus, Brazil",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/D3C7/production/_113451245_gettyimages-1221662798.jpg"
},
{
"alt": "Orfordness Lighthouse",
"link": "https://ichef.bbc.co.uk/wwhp/144/cpsprodpb/CFDA/production/_113401235_on2.jpg"
},
{
"alt": "",
"link": "https://bbc.com/"
}
],
"video": [],
"audio": []
},
"text": {
"h1": [
"BBC Homepage"
],
"h2": [
"Accessibility links",
"News",
"Sport",
"London Weather",
"Editor’s Picks",
"Latest Business News",
"Technology of Business",
"ADVERTISEMENT",
"Follow the food",
"Featured video",
"BBC World Service",
"More around the BBC",
"From Our Correspondents",
"The Boss",
"Designed",
"World in pictures",
"BBC in other languages",
"More Languages",
"Explore the BBC"
],
"h3": [
"UK 'badly underestimated' Russian threat",
"Why are Americans so angry about masks?",
"Netflix cancels Turkish show in gay character row",
"The near-death experience of a genius",
"Why some people never get bored",
"Ukraine police free hostages in bus stand-off",
"Ex-Trump lawyer sues government over 'gag order'",
"Woman takes selfie as wild bear sniffs her hair",
"Premier League: Aston Villa 1-0 Arsenal - Trezeguet with huge goal",
"Sterling-inspired Man City hammer Watford",
"Did De Gea peak in 2017? The numbers behind his rise and fall",
"Tue",
"Wed",
"Thu",
"Fri",
"The surprising benefits of sleeping with bees",
"How the British military hired a magician to outwit the Nazis",
"The physics that suggests we have no free will",
"The world’s most self-reliant ‘nation’?",
"The greatest summer novels ever written",
"Why planning can reduce pandemic stress",
"The undetected Covid-19 immunity",
"Where women are kidnapped to be brides",
"Greta to donate one-million-euro prize",
"Lockdown Brits splurge on tea and biscuits",
"Ryanair to close base after pilots reject pay cut",
"EU reaches 'pivotal' coronavirus recovery deal",
"Apple makes 2030 carbon-neutral pledge",
"UK quarterly borrowing hits record high",
"The most powerful renewable energy",
"The spies hunting fish pirates",
"The strongest law on climate yet",
"The stress of leading a start-up through the pandemic",
"The people creating an oasis with seawater",
"'Fighting to prove we're British'",
"'Fighting to prove we're British'",
"Hope probe: UAE launches Mars mission",
"ICYMI: Luxury grapes and a 'criminal'...",
"'We're doing a dangerous and amazing job'",
"Spectacular views from new tourist train",
"Ready for the total solar eclipse?",
"Great Mosque of al-Nuri destroyed",
"Video 'shows Mosul mosque's destruction'",
"The rise of the female fitness model",
"Car burglar caught with his pants down",
"One-minute World News",
"'There is no such concentration camp in...",
"Ten-year-old punk publisher's surprise...",
"Job lessons from 2008 for the class of 2020",
"Dua Lipa courts controversy with nationalist tweet",
"LinkedIn to cut 960 jobs as recruitment falls",
"Elon Musk offered Amber Heard '24/7 security'",
"US report accuses China of 'digital authoritarianism'",
"River Thames 'severely polluted with plastic'",
"Oxford coronavirus vaccine triggers immune response",
"Coronavirus not over by Christmas, experts warn",
"Churchill through the eyes of India By Yogita Limaye",
"England's test and trace scheme 'breaks data law' By Rory Cellan-Jones",
"Encouraging results in coronavirus vaccine trials By Fergus Walsh",
"What has happened to the report into Priti Patel? By Laura Kuenssberg",
"Covid fears putting off Chinese students from UK By Sean Coughlan",
"UK ministers shift the message on face coverings By Laura Kuenssberg",
"'Hope' and ambition drive UAE's Mars mission By Jonathan Amos",
"Wounds of Dutch history expose deep racial divide By Anna Holligan",
"'I was standing at the wrong end of a machine gun'",
"The craft taking the world by storm",
"Portrait of life under Covid for visually impaired",
"Closest ever pictures taken of the Sun",
"Africa's top shots: Handprints and sea spray",
"How coronavirus swept through Brazil",
"Waving goodbye to a coastal landmark",
"Persian",
"Spanish",
"Urdu",
"Vietnamese"
],
"h4": [],
"h5": [],
"h6": [],
"p": [
"Russian influence in the UK is now \"the new normal\", according to a parliamentary committee.",
"A siege on a bus in the city of Lutsk has ended with the arrest of the hostage-taker, officials say.",
"Michael Cohen claims he was sent back to jail in retaliation for writing a book about the president.",
"This bear definitely was not social distancing as he pawed at an incredibly calm park visitor.",
"Text and BBC Radio 5 Live coverage of Aston Villa v Arsenal in a huge game at the bottom of the Premier League.",
"Managerless Watford's relegation fears increase as they are hammered by a Raheem Sterling-inspired Manchester City.",
"Manchester United keeper David de Gea has come under scrutiny again after his errors contributed to his side's FA Cup semi-final defeat by Chelsea.",
"13°C",
"25°C",
"14°C",
"22°C",
"15°C",
"25°C",
"16°C",
"The most amazing videos from the BBC",
"This locavore lover's paradise be the planet’s next great food destination",
"From The Go-Between to Call Me By Your Name, stories that raise the temperature",
"The process of planning can help you handle uncertainty",
"Some people had a secret store of immunity before the virus even existed",
"On an island east of Bali, women are still being abducted for marriage",
"Campaigner Greta Thunberg says she will donate the money to projects that tackle the climate crisis",
"People working from home have spent heavily on tea, coffee, biscuits and new books",
"Solutions for a sustainable world",
"Three people struggling to secure their lives in the UK after learning they're not legally British.",
"Some accuse the singer of stoking Albanian nationalism, while others say she is defending people's...",
"چین و ایران؛ موضوع مذاکرات 'صریح' پومپئو در لندن",
"\"El mundo está tratando los síntomas de la pandemia, pero no las causas\"",
"اسلام آباد سے اغوا کیے گئے صحافی مطیع اللہ جان گھر واپس پہنچ گئے",
"Dự án Cần Giờ: Đừng để có lỗi với nhân dân"
],
"b": [],
"i": [],
"u": [],
"strike": [],
"sup": [],
"sub": [],
"small": [
"Copyright © 2020 BBC. The BBC is not responsible for the content of external sites. Read about our approach to external linking."
],
"tt": [],
"pre": [],
"blockquote": [],
"strong": [],
"em": [
"Copyright © 2020 BBC."
],
"font": []
},
"table": {
"th": [],
"td": []
},
"list": [
"Skip to content",
"Accessibility Help",
"Home",
"News",
"Sport",
"Weather",
"iPlayer",
"Sounds",
"CBBC",
"CBeebies",
"Food",
"Bitesize",
"Arts",
"Taster",
"Local",
"Three",
"Menu",
"UK 'badly underestimated' Russian threat \n \n \n \n Russian influence in the UK is now \"the new normal\", according to a parliamentary committee. \n \n UK Politics\n \n \n \n\n \n UK 'badly underestimated' Russian threat",
"Why are Americans so angry about masks? \n \n \n \n US\n \n \n \n\n \n Why are Americans so angry about masks?",
"Netflix cancels Turkish show in gay character row \n \n \n \n Entertainment & Arts\n \n \n \n\n \n Netflix cancels Turkish show in gay character row",
"The near-death experience of a genius \n \n \n \n Culture\n \n \n \n\n \n The near-death experience of a genius",
"Why some people never get bored \n \n \n \n Future\n \n \n \n\n \n Why some people never get bored",
"Ukraine police free hostages in bus stand-off \n \n \n \n A siege on a bus in the city of Lutsk has ended with the arrest of the hostage-taker, officials say. \n \n Europe\n \n \n \n\n \n Ukraine police free hostages in bus stand-off",
"Ex-Trump lawyer sues government over 'gag order' \n \n \n \n Michael Cohen claims he was sent back to jail in retaliation for writing a book about the president. \n \n US & Canada\n \n \n \n\n \n Ex-Trump lawyer sues government over 'gag order'",
"Woman takes selfie as wild bear sniffs her hair \n \n \n \n This bear definitely was not social distancing as he pawed at an incredibly calm park visitor. \n \n Latin America & Caribbean\n \n \n \n\n \n Woman takes selfie as wild bear sniffs her hair",
"Premier League: Aston Villa 1-0 Arsenal - Trezeguet with huge goal \n \n \n \n Text and BBC Radio 5 Live coverage of Aston Villa v Arsenal in a huge game at the bottom of the Premier League. \n \n Football\n \n \n \n\n \n Premier League: Aston Villa 1-0 Arsenal - Trezeguet with huge goal",
"Sterling-inspired Man City hammer Watford \n \n \n \n Managerless Watford's relegation fears increase as they are hammered by a Raheem Sterling-inspired Manchester City. \n \n Football\n \n \n \n\n \n Sterling-inspired Man City hammer Watford",
"Did De Gea peak in 2017? The numbers behind his rise and fall \n \n \n \n Manchester United keeper David de Gea has come under scrutiny again after his errors contributed to his side's FA Cup semi-final defeat by Chelsea. \n \n Football\n \n \n \n\n \n Did De Gea peak in 2017? The numbers behind his rise and fall",
"Tue 13°C",
"Wed 25°C 14°C",
"Thu 22°C 15°C",
"Fri 25°C 16°C",
"The surprising benefits of sleeping with bees",
"How the British military hired a magician to outwit the Nazis",
"The physics that suggests we have no free will",
"The world’s most self-reliant ‘nation’? \n \n \n \n This locavore lover's paradise be the planet’s next great food destination \n \n Travel\n \n \n \n\n \n The world’s most self-reliant ‘nation’?",
"The greatest summer novels ever written \n \n \n \n From The Go-Between to Call Me By Your Name, stories that raise the temperature \n \n Culture\n \n \n \n\n \n The greatest summer novels ever written",
"Why planning can reduce pandemic stress \n \n \n \n The process of planning can help you handle uncertainty \n \n Worklife\n \n \n \n\n \n Why planning can reduce pandemic stress",
"The undetected Covid-19 immunity \n \n \n \n Some people had a secret store of immunity before the virus even existed \n \n Future\n \n \n \n\n \n The undetected Covid-19 immunity",
"Where women are kidnapped to be brides \n \n \n \n On an island east of Bali, women are still being abducted for marriage \n \n Asia\n \n \n \n\n \n Where women are kidnapped to be brides",
"Greta to donate one-million-euro prize \n \n \n \n Campaigner Greta Thunberg says she will donate the money to projects that tackle the climate crisis \n \n Science & Environment\n \n \n \n\n \n Greta to donate one-million-euro prize",
"Lockdown Brits splurge on tea and biscuits \n \n \n \n People working from home have spent heavily on tea, coffee, biscuits and new books \n \n Business\n \n \n \n\n \n Lockdown Brits splurge on tea and biscuits",
"1 Ryanair to close base after pilots reject pay cut",
"2 EU reaches 'pivotal' coronavirus recovery deal",
"3 Apple makes 2030 carbon-neutral pledge",
"4 UK quarterly borrowing hits record high",
"The most powerful renewable energy",
"The spies hunting fish pirates",
"The strongest law on climate yet",
"Technology of Business \n \n \n \n \n\n \n \n\n \n \n Business\n \n \n \n The stress of leading a start-up through the pandemic \n \n \n \n\n \n The stress of leading a start-up through the pandemic \n\n \n\n \n /*<![CDATA[*/(function() {if (window.bbcdotcom && bbcdotcom.slotAsync) {bbcdotcom.slotAsync(\"module_feature-1\", [1,2,3,4]);}})();/*]]>*/",
"ADVERTISEMENT /*<![CDATA[*/(function() {if (window.bbcdotcom && bbcdotcom.slotAsync) {bbcdotcom.slotAsync(\"infeed\", [1,2,3,4]);}})();/*]]>*/",
"Follow the food \n \n \n \n \n\n \n \n\n \n \n Future\n \n \n \n The people creating an oasis with seawater \n \n \n \n\n \n The people creating an oasis with seawater \n\n \n\n \n /*<![CDATA[*/(function() {if (window.bbcdotcom && bbcdotcom.slotAsync) {bbcdotcom.slotAsync(\"module_feature-2\", [1,2,3,4]);}})();/*]]>*/",
"Recommended",
"Most Watched",
"Latest",
"'Fighting to prove we're British' \n \n \n \n UK\n \n \n \n\n \n 'Fighting to prove we're British'",
"Hope probe: UAE launches Mars mission \n \n \n \n Science & Environment\n \n \n \n\n \n Hope probe: UAE launches Mars mission",
"ICYMI: Luxury grapes and a 'criminal'... \n \n \n \n World\n \n \n \n\n \n ICYMI: Luxury grapes and a 'criminal' kangaroo",
"'We're doing a dangerous and amazing job' \n \n \n \n Africa\n \n \n \n\n \n 'We're doing a dangerous and amazing job'",
"Spectacular views from new tourist train \n \n \n \n UK\n \n \n \n\n \n Spectacular views from new tourist train",
"Ready for the total solar eclipse? \n \n \n \n US & Canada\n \n \n \n\n \n Ready for the total solar eclipse?",
"Great Mosque of al-Nuri destroyed \n \n \n \n Middle East\n \n \n \n\n \n Great Mosque of al-Nuri destroyed",
"Video 'shows Mosul mosque's destruction' \n \n \n \n Middle East\n \n \n \n\n \n Video 'shows Mosul mosque's destruction'",
"The rise of the female fitness model \n \n \n \n England\n \n \n \n\n \n The rise of the female fitness model",
"Car burglar caught with his pants down \n \n \n \n US & Canada\n \n \n \n\n \n Car burglar caught with his pants down",
"One-minute World News \n \n \n \n Video\n \n \n \n\n \n One-minute World News",
"'There is no such concentration camp in... \n \n \n \n Asia\n \n \n \n\n \n 'There is no such concentration camp in Xinjiang'",
"Ten-year-old punk publisher's surprise... \n \n \n \n Entertainment & Arts\n \n \n \n\n \n Ten-year-old punk publisher's surprise success",
"Job lessons from 2008 for the class of 2020 \n \n \n \n Business\n \n \n \n\n \n Job lessons from 2008 for the class of 2020",
"Dua Lipa courts controversy with nationalist tweet \n \n \n \n Some accuse the singer of stoking Albanian nationalism, while others say she is defending people's... \n \n Europe\n \n \n \n\n \n Dua Lipa courts controversy with nationalist tweet",
"LinkedIn to cut 960 jobs as recruitment falls \n \n \n \n Business\n \n \n \n\n \n LinkedIn to cut 960 jobs as recruitment falls",
"Elon Musk offered Amber Heard '24/7 security' \n \n \n \n UK\n \n \n \n\n \n Elon Musk offered Amber Heard '24/7 security'",
"US report accuses China of 'digital authoritarianism' \n \n \n \n Technology\n \n \n \n\n \n US report accuses China of 'digital authoritarianism'",
"River Thames 'severely polluted with plastic' \n \n \n \n Science & Environment\n \n \n \n\n \n River Thames 'severely polluted with plastic'",
"Oxford coronavirus vaccine triggers immune response \n \n \n \n UK\n \n \n \n\n \n Oxford coronavirus vaccine triggers immune response",
"Coronavirus not over by Christmas, experts warn \n \n \n \n Health\n \n \n \n\n \n Coronavirus not over by Christmas, experts warn",
"Churchill through the eyes of India By Yogita Limaye",
"England's test and trace scheme 'breaks data law' By Rory Cellan-Jones",
"Encouraging results in coronavirus vaccine trials By Fergus Walsh",
"What has happened to the report into Priti Patel? By Laura Kuenssberg",
"Covid fears putting off Chinese students from UK By Sean Coughlan",
"UK ministers shift the message on face coverings By Laura Kuenssberg",
"'Hope' and ambition drive UAE's Mars mission By Jonathan Amos",
"Wounds of Dutch history expose deep racial divide By Anna Holligan",
"The Boss \n \n \n \n \n\n \n \n\n \n \n 'I was standing at the wrong end of a machine gun' \n \n \n \n Business\n \n \n \n\n \n 'I was standing at the wrong end of a machine gun'",
"/*<![CDATA[*/(function() {if (window.bbcdotcom && bbcdotcom.slotAsync) {bbcdotcom.slotAsync(\"mpu_bottom\", [1,2,3,4]);}})();/*]]>*/",
"Designed \n \n \n \n \n\n \n \n\n \n \n The craft taking the world by storm \n \n \n \n Culture\n \n \n \n\n \n The craft taking the world by storm",
"Portrait of life under Covid for visually impaired \n \n \n \n In Pictures\n \n \n \n\n \n Portrait of life under Covid for visually impaired",
"Closest ever pictures taken of the Sun \n \n \n \n Science & Environment\n \n \n \n\n \n Closest ever pictures taken of the Sun",
"Africa's top shots: Handprints and sea spray \n \n \n \n Africa\n \n \n \n\n \n Africa's top shots: Handprints and sea spray",
"How coronavirus swept through Brazil \n \n \n \n Latin America & Caribbean\n \n \n \n\n \n How coronavirus swept through Brazil",
"Waving goodbye to a coastal landmark \n \n \n \n Suffolk\n \n \n \n\n \n Waving goodbye to a coastal landmark",
"Persian چین و ایران؛ موضوع مذاکرات 'صریح' پومپئو در لندن",
"Spanish \"El mundo está tratando los síntomas de la pandemia, pero no las causas\"",
"Urdu اسلام آباد سے اغوا کیے گئے صحافی مطیع اللہ جان گھر واپس پہنچ گئے",
"Vietnamese Dự án Cần Giờ: Đừng để có lỗi với nhân dân",
"Arabic عربي",
"Azeri AZƏRBAYCAN",
"Bangla বাংলা",
"Burmese မြန်မာ",
"Chinese 中文网",
"French AFRIQUE",
"Hausa HAUSA",
"Hindi हिन्दी",
"Indonesian INDONESIA",
"Japanese 日本語",
"Kinyarwanda GAHUZA",
"Kirundi KIRUNDI",
"Kyrgyz Кыргыз",
"Marathi मराठी",
"Nepali नेपाली",
"Pashto پښتو",
"Persian فارسی",
"Portuguese BRASIL",
"Russian НА РУССКОМ",
"Sinhala සිංහල",
"Somali SOMALI",
"Spanish MUNDO",
"Swahili SWAHILI",
"Tamil தமிழ்",
"Turkish TÜRKÇE",
"Ukrainian УКРАЇНСЬКA",
"Urdu اردو",
"Uzbek O'ZBEK",
"Vietnamese TIẾNG VIỆT",
"Home",
"News",
"Sport",
"Weather",
"iPlayer",
"Sounds",
"CBBC",
"CBeebies",
"Food",
"Bitesize",
"Arts",
"Taster",
"Local",
"Three",
"Terms of Use",
"About the BBC",
"Privacy Policy",
"Cookies",
"Accessibility Help",
"Parental Guidance",
"Contact the BBC",
"Get Personalised Newsletters"
]
}
}If you migrate to google storage with the key you got from the Fetch endpoint, you can get the html, image of that web page and dynamically scrape it by adding the url of html to the part with the cache keyword
That’s it, clap if you want to know more 😉
Author: Emin Muhammadi (https://muemin.me). You can contact here for ideas and suggestions.
