Memory mapped files and MappedByteBuffers in Java

In this short article, we’ll be looking at the memory-mapped file MappedByteBufferin the java.nio package. This utility can be quite useful for efficient file reads.

Memory-mapped files

A memory-mapped file is a segment of virtual memory that has been assigned a direct byte-for-byte correlation with some portion of a file.

• Reading and writing in the memory-mapped file is generally done by the operating system to write content into a disk.
• A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.
• Memory-Mapped Files in Java is a rather new Java concept for many programmers and developers, though it’s been there since JDK 1.4 along with package ‘ java.nio ’.
• Java IO has been considerably fast after the introduction of NIO, and memory-mapped file offers the fastest IO operation possible in Java,
  that’s why high-performance Java applications should use memory-mapped files for persisting data.
• Memory Mapped is already quite popular in the high-frequency trading space, where the electronic trading system needs to be super fast and one-way latency to exchange has to be on the sub-microsecond level.

There are two types of memory-mapped files:

• Persisted memory-mapped files: Persisted files are associated with a disk source file. The data is saved to the source file on the disk when the last process has finished working with the file. These memory-mapped files are suitable for working with extremely large source files.
• Non-persisted memory-mapped files: Non-persisted files are memory-mapped files that are not associated with a file on a disk. When the last process has finished working with the file, the data is lost, and the file is reclaimed by garbage collection. These files are suitable for creating shared memory for inter-process communications (IPC).

Benefits of Memory Mapped Files

The principal benefits of memory mapping are efficiency, faster file access, the ability to share memory between applications, automatic memory management, and more points described below

• Accessing files via memory map is faster than using I/O functions such as fread and fwrite. Data are read and written using the virtual memory capabilities built into the operating system rather than having to allocate, copy into, and then deallocate data buffers owned by the process.
• As the user process deal with the mapped memory space, page faults will be generated automatically to bring in the file data from the disk.
• If the user modifies the mapped memory space, the affected page is automatically marked as dirty and will be subsequently flushed to the disk to update the file.
• The virtual memory subsystem of the operating system will perform intelligent caching of the pages, automatically managing memory according to system load.
• The data is always page-aligned, and no buffer copying is ever needed.
  Very large files can be mapped without consuming large amounts of memory to copy the data.

Example of memory-mapped file

The following illustration shows how multiple processes can have multiple and overlapping views of the same memory-mapped file at the same time.
It shows multiple and overlapped views of a memory-mapped file:

MappedByteBuffer will also not be visible to other programs that have mapped the same file; instead, they will cause private copies of the modified portions of the buffer to be created.

import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
 
public class MemoryMappedFile 
{
  static int length = 0x8FFFFFF;  
 
  public static void main(String[] args) throws Exception 
  {
    try(RandomAccessFile file = new RandomAccessFile("test.dat", "rw")) 
    {
      MappedByteBuffer out = file.getChannel()
                    .map(FileChannel.MapMode.READ_WRITE, 0, length);
       
      for (int i = 0; i < length; i++) 
      {
        out.put((byte) 'x');
      }
       
      System.out.println("Finished writing");
    }
  }
}

The file created with the above program is 128 MB long, which is probably larger than the space your OS will allow. The file appears accessible all at once because only portions of it are brought into memory, and other parts are swapped out. This way, a very large file (up to 2 GB) can easily be modified.

Advantages and disadvantages of Memory Mapped files

There are several advantages and disadvantages of memory-mapped files:

• Possibly, the main advantage of Memory Mapped IO is performance, which is important to build high-performance systems like for electronic trading.
• Memory Mapped Files are way faster than standard file access via normal IO.
• Another significant advantage of memory-mapped IO is that it allows you to load a potentially larger file that is not otherwise accessible.
• The disadvantage of memory-mapped IO is that it potentially complicates your cache controller, as device accesses behave differently from normal memory accesses.

Conclusion

In this article, we learned these points:

• Memory Mapped Files are way faster than standard file access via normal IO.
• Multiple processes can have multiple and overlapping views of the same memory-mapped file.
• Types of memory-mapped files.