The Startup
Published in

The Startup

How to get a random line from a file? — One Pass Solution Using Python

When a random line is to be selected from a small file, we can do it pretty quickly, even with a naive algorithm. As the small file can be loaded into RAM, the selection of a random line is straightforward. But for the files of the magnitude of terabytes and petabytes, such naive approach doesn’t work. RAM can’t load the whole file at once so we need to read it from the storage devices.

Photo by Safar Safarov on Unsplash

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Suraj Regmi

Suraj Regmi

125 Followers

CS Graduate Student at UAH, Former Data Scientist at World Bank — the views and the content here represent my own and not of my employers.