Web scraping is now legal
Here’s what that means for Data Scientists
In late 2019, the US Court of Appeals denied LinkedIn’s request to prevent HiQ, an analytics company, from scraping its data.
The decision was a historic moment in the data privacy and data regulation era. It showed that any data that is publicly available and not copyrighted is fair game for web crawlers.
But commercial use of scraped data is still limited
The decision does not, however, grant HiQ or other web crawlers the freedom to use data obtained by scraping for unlimited commercial purposes.
For example, a web crawler would be allowed to search Youtube for video titles, but it could not re-post the Youtube videos on its own site, since the videos are copyrighted.
In general, the copyright for data, including data for media files like video or music, is still enforceable regardless of how the data was obtained.
Some forms of web scraping are also still illegal
The decision also does not grant web crawlers the freedom to obtain data from sites that require authentication.
For example, a web crawler that logged-in to Facebook and downloaded user data would…
