How to get a random line from a file? — One Pass Solution Using Python
When a random line is to be selected from a small file, we can do it pretty quickly, even with a naive algorithm. As the small file can be loaded into RAM, the selection of a random line is straightforward. But for the files of the magnitude of terabytes and petabytes, such naive approach doesn’t work. RAM can’t load the whole file at once so we need to read it from the storage devices.