IOU (Intersection over Union)

What is IOU and where is it used?

• IOU(Intersection over Union) is a term used to describe the extent of overlap of two boxes. The greater the region of overlap, the greater the IOU.
• IOU is mainly used in applications related to object detection, where we train a model to output a box that fits perfectly around an object. For example in the image below, we have a green box, and a blue box. The green box represents the correct box, and the blue box represents the prediction from our model. The aim of this model would be to keep improving its prediction, until the blue box and the green box perfectly overlap, i.e the IOU between the two boxes becomes equal to 1.

Figure 13: Depiction of the task of object detection

• IOU is also used in non max suppression, which is used to eliminate multiple boxes that surround the same object, based on which box has a higher confidence. I will go over non max suppression in more detail in my next post.

Calculating IOU:

Let us assume that box 1 is represented by [x1, y1, x2, y2], and box 2 is represented by [x3, y3, x4, y4]. (We will use this convention later to calculate the areas.)

Figure 1: Intersection of two boxes

Figure 2: Diagrammatic Representation of the formula to calculate IOU

1. Calculating the area of the intersection of the boxes

Figure 3: Boxes can intersect in many different ways, or even completely overlap each other

• Let us represent the coordinates of the intersected rectangle as
  [x_inter1, y_inter1, x_inter2, y_inter2 ], to denote the coordinates of the top left, and bottom right of the intersection.

Figure 4: Naming the coordinates that need to be calculated

• Lets start by defining the reference coordinate system. We use the computer graphics convention of +X axis moving to the right, and +Y axis moving downward.

Figure 5: Reference Coordinate Axis

• To calculate the top left corner of the intersection, we compare the top left corners of each of the boxes. We can see from the examples above, that x_inter1 can be found by seeing which box has its top left corner more to the right. Similarly y_inter1 can be found by seeing which box has its top left corner lower than the other. Mathematically they can be calculated as:

Figure 6: Calculating the top left corner of the intersection

• To calculate the bottom right corner of the intersection, we compare the bottom right corners of each of the boxes. x_inter2 can be found by seeing which box has its bottom right corner more to the left. Similarly y_inter2 can be found by seeing which box has its bottom right corner higher than the other. Mathematically they can be calculated as:

Figure 7: Calculating the botom right corner of the intersection

• For example, Figure 4 has cases where the boxes overlap perfectly, or have the same value for a particular coordinate. In such cases, the min or max operation of comparing a value with itself, is simply the value itself.

Figure 8: Example when comparing boxes with the same value for a coordinate

• Now that we have the coordinates of the intersection, the area of the intersection is simply the area of the rectangle formed. (In practice we would take the modulus of the width and height, to make sure that even if the orders of the boxes are changed, the width and height would still be positive; i.e if x_inter1>x_inter2 , we would still get a positive value of width)

Figure 8: Calculating the area of the intersection

2. Calculating the area of the union of the boxes

• The union of the boxes is the total area covered by both the boxes, as shown in Figure 2.
• To find the total area, we first calculate the area of the individual boxes.

Figure 9: Calculating the areas of the two boxes

• If we look at the total area covered by the two boxes, we see that the portion of the intersection is covered in both the boxes, i.e the area of the intersection is included in both area_box1 and area_box2.

Figure 10: Depicting the intersection area being covered in both boxes.

• Since we want to account for the common area of intersection only once, we can subtract the area of intersection we calculated, from the total area of the two boxes.

Figure 11: Area of the union of the boxes

3. Calculating the IOU

Figure 12: Formula to calculate IOU

What is the range of values that the IOU can have?

• The IOU of two boxes can have any values between 0 and 1.
• In case there are 2 boxes that do not intersect, the area of their intersection would be 0, and therefore the IOU would also be 0.
• In case there are 2 boxes that completely overlap, the area of the intersection would be equal to the area of their union, and therefore the IOU would be 1.

Coding a function for IOU in python:

• For the same image displayed above, lets write a function in python to calculate the IOU of the green and the blue box.

Figure 14: Function in python to implement IOU

• The function IOU takes in 2 boxes, box1 and box2 as input. The data in each box is a list containing[x1, y1, x2, y2], which is the top left, and bottom right coordinates.
• We find the area of the intersection, followed by the area of the union, as described earlier.
• The abs function is an inbuilt function in python to calculate the modulus. This ensures we never end up with a negative width or height.
• The IOU returned is a value of type float, that lies between 0 and 1.
• I have written a code that allows us to visualize the effect of varying the bounding boxes on the IOU. The link to the complete code is here