Become a Techie
Published in

Become a Techie

File structure read failures

What is a file structure?

Each file type is made up of a set of structures which follow a strict specification. If a file does not conform to a specification meeting the minimum rules outlined, this could result in file structure read failures. There are many circumstances which I have come across when some file types do not have a clearly defined, up to date specification, which can be found online. Therefore, it can be very easy to miss type definitions, structures and rules. This can have serious negative implications upon the file being read. I will provide a real-life example which I encountered with an MP4 file structure below.

Case study: ‘EditListBox’ structure problem while reading MP4 files:

I was working on a program that reads MP4 files. The program uses a MP4 definition class I have created and compares the file structure against the file being read at run-time. I was investigating an issue where half of the MP4 files I would try to read in code would fail. The failure was not clear to me because after extensively reading from various sources of MP4 specifications, it seemed as though I had defined all the necessary structures required for a file of type MP4. Moving forward, I managed to crack down where the failure was occurring and what the reason for the error was — bits were being shifted right, which was messing up the entire read operation on the file. A right shift of bits means that the bits are being knocked forward by zeros. The reason was because two types — ‘media_rate_fraction’ and ‘media_rate_integer’, were defined as global types to the ‘EditListBox’ structure. To solve the issue, I had to move the definitions of the above two types to the children structures of ‘EditListBox’. Lesson learnt — always ensure that any file structures you define include all types it supports… In the correct structure class!

Key reasons why file structure reads may fail:

  • Incorrect structures defined — Structures must be defined to specification. If you try to create structures with your own names/types/definitions, without complying with specification, you will find that your file will fail to read.
  • Structures are defined correctly, but not being called in the correct order — You may have created a definition document by specification, but it is very important in the order that the structures you define get read. You need to make sure that all structures get read in order by specification which define a file, from top to bottom.
  • Incorrect defined structure types — It is very important to define the types in your structures according to specification. This is a very important point to make because the difference between defining an unassigned int32 and defining an unassigned int32 in big endian can make a massive difference to if the file gets read in a valid way or not. Therefore, be careful to make sure that you follow the specifications thoroughly.
  • Ensure you have valid structural rules when reading files — Make sure that you have defined rules according to specification. This is important to ensure that structures get read only when conditions are met. If you fail to include necessary rules, your structures will be read and assigned in wrong places in the document which will result in the file being rendered invalid.

Check out Elecard Video Format Analyzer — a tool that is free and I found very useful for displaying video format structures (including MP4) as a tree.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Become a Techie

Become a Techie

Become a Techie blog providing you exclusive step by step tutorials! No matter your level of technical expertise, from devs to the average user.