How to Read Large File In Java

Reading file in-memory vs streaming through it

Suraj Mishra

·

Follow

Published in

Javarevisited

·

3 min read

·

May 17, 2023

--

4

Share

Introduction

• In this small article, we will learn how to read the file in memory vs streaming through it. We will discuss how we can read a large file.

Preparing Input Files

• We don’t need to search and download large files, instead, we can generate the large file using the below code.
• Our file contains numbers from 1 to 100 million and the size of the file is approx 900 MB and sufficient for our use case.

public static void prepFile() throws IOException {
     BufferedWriter bufferedWriter = Files.newBufferedWriter(Path.of(TARGET_FILE), StandardOpenOption.CREATE);
     IntStream.rangeClosed(1,100000000)
                .forEach(a-> write(bufferedWriter, a));
     bufferedWriter.flush();
}

public static void write(BufferedWriter bufferedWriter, int a) {
        try {
            bufferedWriter.write(a + "\n");
        } catch (IOException e) {
            System.out.println("Exception: " + e);
            throw new RuntimeException(e);
        }
}

Read Small File

• If the file is relatively smaller we can read all of them in memory using readAllLines() method.
• Files API is an…