Understanding 3D medical image orientation for programmers
If you’re new to working with medical images, then you’re in for a minefield. You’re going to have to put intuition aside because in alot of this, there isn’t any, or at least there was to start off with. Now, we have just convention built upon convention.
The typical start point on medical orientation is acknowledging the two widely used ‘perspectives’ commonly known as the radiological and neurological view. The former originated from looking at the chest, where the patient would be viewed infront, facing opposite them and the latter at brains, where the patient would be viewed behind from above.
In 3D medical images, we tend to commonly talk about the axial slice (head-feet), we could say a chest radiologist would look at a slice from below and a radiologist from above.
However, as programmers we like to talk about 3D volumes in terms of matrix indices i, j, k. So knowing which volume axis our code relates to which anatomical direction will let us get our job done.
Orientation Conventions
Typically how a medical image is stored, is not the orientation in which it is viewed. Image orientation metadata tells us the relation of how the image is stored to the real world, more concretely, the relation of store volume axes IJK to the real world. i.e. left/right, back/front and up/down.
╔════════╦════════════╗
║ Common ║ Anatomical ║
╠════════╬════════════╣
║ Left ║ Left ║
║ Right ║ Right ║
║ Up ║ Superior ║
║ Down ║ Inferior ║
║ Front ║ Anterior ║
║ Back ║ Posterior ║
╚════════╩════════════╝This brings us back to our clinical radiological and neurological views which we are going to refer to as LPS (Left, Posterior and Superior) and RAS (Right, Anterior and Superior) respectively from now on. Different data formats and applications tend to use one of these.
A one-to-one mapping of translating IJKtoLPS will mean that the 1st axis points to the left, 2nd to the posterior and 3rd to the superior. This can be denoted in a 3x3 affine matrix as below.
1 0 0
IJKtoLPS = 0 1 0
0 0 1Each column represents a basis vector that encodes each image axis from left to right respectively. Since the relation in this case is one-to-one, the affine matrix was the identity. The left image in the figure below demonstrates this.
Suppose that the image is infact saved in the RAS format such as in the right figure above, then the IJKtoLPSmatrix would show that the 1st and 2nd axes are now flipped.
-1 0 0
IJKtoLPS = 0 -1 0
0 0 1In fact you can have have all sorts of combinations, here are a few examples that you can work through.
╔═══════════╦══════╗
║ IJKtoLPS ║ Code ║
╠═══════════╬══════╣
║ 0 -1 0 ║ ║
║ 0 0 1 ║ SRP ║
║ 1 0 0 ║ ║
╠═══════════╬══════╣
║ 0 0 -1 ║ ║
║ -1 0 0 ║ AIR ║
║ 0 -1 0 ║ ║
╠═══════════╬══════╣
║ 0 1 0 ║ ║
║ 0 0 -1 ║ SLA ║
║ 1 0 0 ║ ║
╚═══════════╩══════╝However, we still have our other widely used reference frame RAS. The principle is the same but now the identity matrix points to the the Right and Anterior.
1 0 0
IJKtoLPS = 0 1 0
0 0 1So now the codes in the examples above from the RAS perspective have become SLA, PIL and SRP respectively.
Switching between the two reference planes is just a matter of matrix taking the dot product.
-1 0 0
IJKtoLPS = 0 -1 0 ∙ IJKtoRAS
0 0 1 -1 0 0
IJKtoRAS = 0 -1 0 ∙ IJKtoLPS
0 0 1
Image origin and spacing
No explanation on image orientation would be complete without talking about its origin. For the affine matrix of a 3D image is infact 4x4, with the 4th column accounting for where the top left voxel in our 3D volume originates in our real world. For an image in RAS orientation, where the top left voxel was calibrated to originate from [-9, 2, -5] mm by the scanner, the 4x4 3D affine matrix would be
-1 0 0 -9
IJKtoLPS = 0 -1 0 2
0 0 1 -5
0 0 0 1Furthermore, we’ve assumed that the size of each voxel has been 1 x 1 x 1 mm, in reality this is not always the case. For this case we’ll say it is infact 0.25 x 0.50 x 1.00 mm. The Affine matrix would take on the final form of
-0.25 0.00 0.00 -9.00
IJKtoLPS = 0.00 -0.50 0.00 2.00
0.00 0.00 1.00 -5.00
0.00 0.00 0.00 1.00Note the 1 on the end of the 4th column which is required by the definition of an affine matrix.
Programming
Useful packages in python for working with 3D medical images come as dicom_numpy for DICOM (.dcm) and nibabel for NIFTI (.nii). Both of these packages can load the 3D volumes in their respecive formats and return to you the affine matrix. However, beware that dicom images work in LPS and nifti and therefore nibabel in RAS. So if you were to want to convert from DICOM to NIFTI in this way we would need to make the conversion on the affine matrix. Tools like dicom2nifti carry this out automatically anyway and several conversion packages exist for every programming language.
I place a special note on nibabel as it has a number of tools for working with orientations, for example aff2axcodes() will return the 3 letter axes code. Infact nibabel has a handy tool for forcing nibabel image objects into its native RAS format nib.as_closest_canonical(img).
Medical Imaging Formats and programs convention
As alluded to already, the reference axis code depends on the application/format you’re working with. Below is a compiled list of some common applications. Some formats like NRRD specify the reference that the affine should be interpreted in within the file header. MATLAB loads the medical imaging data as it comes so dicomread in LPS and niftiread as RAS, so you’d have to apply your own transformation if you wanted to use MATLAB to convert formats.
╔═════════════╦═════════════╗
║ Tool ║ Orientation ║
╠═════════════╬═════════════╣
║ DICOM ║ LPS ║
║ NIFTI ║ RAS ║
║ NRRD ║ flexible ║
║ dicom_numpy ║ LPS ║
║ nibabel ║ RAS ║
║ MATLAB ║ raw ║
║ Slicer ║ RAS ║
║ ITKsnap ║ LPI ║
╚═════════════╩═════════════╝This article exists thanks to some helpful resourses out there and great for further reading:
- https://nipy.org/nibabel/image_orientation.html
- https://nipy.org/nibabel/coordinate_systems.html
- https://www.slicer.org/wiki/Coordinate_systems
- https://itk.org/Wiki/Proposals:Orientation
CT Images were available under CC0:
