How can a computer distinguish whether a file is an image or text?

How does a computer know which file is an image and which is a text file?

At first glance, this might seem simple — each file has an extension, so you can just differentiate based on that.

Images will have extensions like .png, .jpeg, etc.
Text files will have extensions like .txt.

But have you ever accidentally downloaded an image file with a .txt extension and found you couldn’t open it? And as a programmer, I have a few more questions:

• What if the file doesn’t have an extension?
• What if a user changes the extension of an image file to .txt before uploading it to a server?

There are many issues to explore here, right? Let’s dive into it!

In simple terms, the file extension doesn’t determine the type of the file.

Indeed, a file may not have an extension, and even if it does, the extension might not accurately reflect the file type.

The importance of the file extension also depends on the operating system.

• (Optional) Generally, operating systems like Linux don’t care about the file extension. Instead, they look for executable files and run them accordingly. The system checks the file’s header and content to find suitable applications to execute the file. For example, files that start with certain headers like…#!/bin/bash . This is called a shebang. It is a special way to tell the operating system which program should be used to open the file.
• (Mandatory) Some operating systems, like Windows, prioritize user convenience and therefore give more importance to file extensions.
  — File Association (Registry): Based on the file extension, Windows links it to specific applications and uses that application to open the file.
  — Default Programs: Windows also allows you to customize which application should open a file based on its extension.

What is a magic number?

First, we need to visualize how files are stored — they are saved in binary format as follows:

0100 0111 0100 1001 0100 0110 ....

Typically, the beginning of a file contains special byte sequences, known as magic numbers, which help specialized applications identify the type of file.

Let’s look at an example of a PNG file below. Once the file type is determined, it will be followed by additional metadata. The length and content of this metadata depend on the type of file.

💡 Metadata is data that describes detailed information about the data, such as the data used to describe that a file is an image and its specific characteristics.

Examples of Magic Numbers for Various Files:

- PNG: 89 50 4E 47 0D 0A 1A 0A
- JPEG: FF D8 FF
- PDF: 25 50 44 46
- ZIP: 50 4B 03 04

What is a MIME Type?

This is a definition that most programmers will encounter frequently.

MIME type (short for Multipurpose Internet Mail Extensions) is a standard string used to identify the format of a file. It consists of two parts: the type and the subtype, separated by a slash (/).

Examples:

- image/png: PNG image format
- image/jpeg: JPEG image format
- text/plain: Plain text
- application/pdf: PDF document
- video/mp4: MP4 video format

How It Works:

When you open a file on a computer, the operating system checks the MIME type of the file (usually determined based on the file extension or magic number) to find the appropriate application to open the file. Similarly, when a web browser downloads a file, it looks at the MIME type to determine how to handle the file (whether to display it directly in the browser, download it, or open it with another application).

Finding MIME Types:

You can find a comprehensive list of MIME types recognized by IANA (Internet Assigned Numbers Authority) on their website or on other sources such as MDN Web Docs:

• https://www.iana.org/assignments/media-types/media-types.xhtml
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types

Bonus — Validate File

Validating file input is a complex process, and there is no single method that is fully effective! We need to combine multiple methods.

• Based on what we’ve learned, when checking the format of a file, we should not rely solely on the file extension, as it can be easily bypassed.
• Checking the MIME type is more reliable but still has vulnerabilities. A file upload request typically includes a header containing the Content-Type field to specify the MIME type of the file. Attackers can modify this field, so be cautious.

💡 I tried checking the multer library and found that it uses a library called busboy, which relies entirely on the `Content-Type` field for MIME type checking.

• Check the file header, specifically the magic number or file signature, and then compare it against the list of reliable types provided in the links below. If it doesn’t match any in the list, throw an error.
  — https://en.wikipedia.org/wiki/List_of_file_signatures
  — https://en.wikipedia.org/wiki/Magic_number_(programming)
  — https://gist.github.com/leommoore/f9e57ba2aa4bf197ebc5
• Finally, check the file content using antivirus or anti-malware software to scan the entire file and eliminate any harmful files.

Conclusion

In summary, determining a file type is not solely dependent on the file extension but involves a combination of factors such as the operating system, magic number, and MIME type. Understanding these elements will help you handle files more accurately and securely.

I hope this article has provided you with valuable insights on this topic. If you have any questions or suggestions, feel free to leave a comment below!