Read last N bytes from a file in Deno

Purpose

The purpose of this article is to learn how to read the last N bytes from a file in Deno. Some applications may need to read the contents of a file from the end rather than reading the file from the start. For example — tailing utility.

The file could be of any size: small or large.

If the file is small, here is how to go about it:

• Read the entire file into memory (Uint8Array)
• Slice the array array.length-N

The above approach for getting the last N bytes from a small file is good. However, this approach doesn't work for large files (size in GBs).

If the file is large, here is how to go about it:

• Open the file
• Seek to N bytes from the end Seek.end-N
• Read into a block of size N
• Close the file

This approach would work for files of any size. The time complexity would be almost the same regardless of the size of the file.

Seek and read

There are two important functions in the reading procedure:

• Seek: Seek is useful in moving the file read pointer (aka cursor) by a certain number of bytes from the desired location.
• Read: Read a certain number of bytes into a buffer or storage

Seek

Deno.seek is useful in seeking or moving the file read pointer to the desired location.

Deno.seek(rid: number, offset: number, whence: SeekMode): Promise<number>

There are three locations to seek from (aka SeekMode):

• Deno.SeekMode.Start: Seek from the start
• Deno.SeekMode.Current: Seek from the current location
• Deno.SeekMode.End: Seek from the end

For the problem we’re solving in this article, we’d seek from the end of the file.

In addition to the position to seek from, the seek function also needs the bytes to move the read pointer from the seek position(offset). For example — Move the cursor by 10 bytes from the start of the file. Or, move the cursor by 100 bytes from the current read position. Or, move the cursor by 1000 bytes from the end of the file.

The seek function returns the new position of the cursor.

await Deno.seek(file.rid, 10, Deno.SeekMode.Start);
await Deno.seek(file.rid, 100, Deno.SeekMode.Current);
await Deno.seek(file.rid, -1000, Deno.SeekMode.End);

It’s important to note that seeking from the end would require the offset to be negative end-offset . If a positive offset is given with seekmode as end, the cursor would stay at the end of the file, and the next read operation would return null.

Read

Deno.read is useful is reading a chunk from the opened file. The amount of data being read would be less than or equal to the size of the buffer receiving the data. Therefore, storage of chunk size needs to be created before reading.

const buf = new Uint8Array(1000); //create storage
await Deno.seek(file.rid, -1000, Deno.SeekMode.End); //seek from the end
const bytesRead=await Deno.read(file.rid, buf); //read

It’s worth noting that, bytesRead contains the actual number of bytes that got read from the file. Also, bytesRead would be null if the end of the file has been reached and there were no more bytes to read.

Complete Example

Now that we’ve gone through the two useful functions, let’s go over a complete example:

const fileName=`readings${Deno.args[0] || '1G'}.txt`, 
      nBytesToRead=parseInt(Deno.args[1]) || 1000;
const file = await Deno.open(fileName, {read: true});
const buf = new Uint8Array(nBytesToRead);
const cursorPos=await Deno.seek(file.rid, 0-nBytesToRead, Deno.SeekMode.End);
const numberOfBytesRead = await Deno.read(file.rid, buf);
file.close();
console.log(`Read ${numberOfBytesRead} from the end of ${fileName}. Read cursor position was ${cursorPos}.`);

Here is the explanation:

• Read the file size and bytes to read from command-line args or substitute default values
• Open the file
• Create storage of required size (bytes to read)
• Seek from the end of file by bytes to read
• Read from the file
• Close the file

Now, let’s run this example for three file sizes: 1G, 2G, and 5G. In all three cases, we’ll try to read 1000 bytes from the end. Here is the output of all three runs:

time deno run --allow-all deno_last_n_bytes.ts 1G 1000
Read 1000 from the end of readings1G.txt. Read cursor position was 999999000.
real 0m0.046s
user 0m0.028s
sys 0m0.014s
--
time deno run --allow-all deno_last_n_bytes.ts 2G 1000
Read 1000 from the end of readings2G.txt. Read cursor position was 1999999000.
real 0m0.051s
user 0m0.026s
sys 0m0.014s
--
time deno run --allow-all deno_last_n_bytes.ts 5G 1000
Read 1000 from the end of readings5G.txt. Read cursor position was 4999999000.
real 0m0.047s
user 0m0.028s
sys 0m0.014s

As expected, the time taken is the same regardless of the size of the file.