Open in app

Sign in

Write

Sign in

REAL TIME DISTANCE CALCULATION USING ARUCO MARKERS

Deepanshusachdeva
Analytics Vidhya
Published in
7 min readMay 27, 2021

This week I came across a very hot topic — Augmented Reality, but before understanding that we need to have an intuition behind ArUco markers.

ArUco Markers .. What are they?

ArUco markers are similar to QR codes, the differentiating factor being that QR codes stores much more information than a ArUco marker and is therefore difficult to employ for what we are are about to do.
It is a binary matrix composed of a black border around it , it is a synthetic marker used to instantly locate it in an image or video.
This is what a ArUco Marker looks like-

ArUco Marker

Now, as it is binary and contains some sort of information , we need to have some variations, so these markers have two unique properties- ID and Type

Now the id of the marker depends on what type it is, Lets understand this is some detail:
In the above marker, If you will see, there is a 1 box padding on all sides, also in the marker you can see that in one row(after removing the padding) you can fit 6 white boxes, so this is a 6X6 marker, now the type of the marker you cannot guess (and you should not!). So the type is predefined and is kind of a hiccup here, there are total of 25 types(or dictionaries) of markers in cv2, and each dictionary contains the same number of bits or blocks and a fixed number of markers(50,100,250 or 1000).
The above attached image of the marker is — id 23 of DICT_6X6_250.
So, the type here is DICT_6X6_250 and id is 23.(Note that here id cannot exceed 250 as there are only 250 markers available in the dictionary).

Uses in Industry:

For Pose Estimation, automation in robots, 2D projection, Augmented Reality and what not! And of course today we are going to learn about how to use them for distance calculation.

Before Using them Lets see how they are generated-
In Python you can generate them with the help of cv2.aruco.drawMarker() function, Lets see it in action!

The required packages are:

import cv2
import numpy as np
import argparse
import sys
import os

Now, you need the id and type of the marker , so lets take them as arguments-

ap.add_argument("-i", "--id", type=int, required=True,
                help="id of the aruco marker to be generated")
ap.add_argument("-t", "--type", type=str,
                default="DICT_ARUCO_ORIGINAL", help="type of ARUCO marker")

Now, for the purpose of generating these markers, we are making a dictionary of ArUco dictionaries.

ARUCO_DICT = {
    "DICT_4X4_50": cv2.aruco.DICT_4X4_50,
    "DICT_4X4_100": cv2.aruco.DICT_4X4_100,
    "DICT_4X4_250": cv2.aruco.DICT_4X4_250,
    "DICT_4X4_1000": cv2.aruco.DICT_4X4_1000,
    "DICT_5X5_50": cv2.aruco.DICT_5X5_50,
    "DICT_5X5_100": cv2.aruco.DICT_5X5_100,
    "DICT_5X5_250": cv2.aruco.DICT_5X5_250,
    "DICT_5X5_1000": cv2.aruco.DICT_5X5_1000,
    "DICT_6X6_50": cv2.aruco.DICT_6X6_50,
    "DICT_6X6_100": cv2.aruco.DICT_6X6_100,
    "DICT_6X6_250": cv2.aruco.DICT_6X6_250,
    "DICT_6X6_1000": cv2.aruco.DICT_6X6_1000,
    "DICT_7X7_50": cv2.aruco.DICT_7X7_50,
    "DICT_7X7_100": cv2.aruco.DICT_7X7_100,
    "DICT_7X7_250": cv2.aruco.DICT_7X7_250,
    "DICT_7X7_1000": cv2.aruco.DICT_7X7_1000,
    "DICT_ARUCO_ORIGINAL": cv2.aruco.DICT_ARUCO_ORIGINAL,
    "DICT_APRILTAG_16h5": cv2.aruco.DICT_APRILTAG_16h5,
    "DICT_APRILTAG_25h9": cv2.aruco.DICT_APRILTAG_25h9,
    "DICT_APRILTAG_36h10": cv2.aruco.DICT_APRILTAG_36h10,
    "DICT_APRILTAG_36h11": cv2.aruco.DICT_APRILTAG_36h11
}

Now using the type given, we will get the Dictionary of marker present in cv2,

arucoDict = cv2.aruco.Dictionary_get(ARUCO_DICT[args['type']])tag = np.zeros((300, 300, 1), dtype="uint8")
cv2.aruco.drawMarker(arucoDict, args["id"], 300, tag, 1)

Now, using ArUco dictionary we use cv2.drawMarker(), the arguments in this functions are: arucoDict- the dictionary of the marker to be generated, args[‘id’] — the id of the marker in the dictionary , 300 — the size of the marker, tag — the canvas for tag , padding- the padding around the marker

Now using cv2.imshow(), we will visualize the marker

cv2.imwrite('output_marker.jpg',tag)
cv2.imshow("marker", tag)
cv2.waitKey(0)
cv2.destroyAllWindows()

Run, using the command

python .\aruco_marker_generation.py --id 50 --type DICT_6X6_250

Our marker is generated and saved to disk.

Now, we will try to detect a marker in an image ,the arguments that we need are:

ap.add_argument("-i", "--image", required=True, help="path to the image")
ap.add_argument("-t", "--type", required=True,
                help="tag of the marker to b detected")

Here, as you can see, we actually need the type of the marker to be detected, so this is a issue while working with unknown markers.
However we will address this issue in the next article, for now lets detect a marker in the given image,

image = cv2.imread(args['image'])ARUCO_DICT = {
    "DICT_4X4_50": cv2.aruco.DICT_4X4_50,
    "DICT_4X4_100": cv2.aruco.DICT_4X4_100,
    "DICT_4X4_250": cv2.aruco.DICT_4X4_250,
    "DICT_4X4_1000": cv2.aruco.DICT_4X4_1000,
    "DICT_5X5_50": cv2.aruco.DICT_5X5_50,
    "DICT_5X5_100": cv2.aruco.DICT_5X5_100,
    "DICT_5X5_250": cv2.aruco.DICT_5X5_250,
    "DICT_5X5_1000": cv2.aruco.DICT_5X5_1000,
    "DICT_6X6_50": cv2.aruco.DICT_6X6_50,
    "DICT_6X6_100": cv2.aruco.DICT_6X6_100,
    "DICT_6X6_250": cv2.aruco.DICT_6X6_250,
    "DICT_6X6_1000": cv2.aruco.DICT_6X6_1000,
    "DICT_7X7_50": cv2.aruco.DICT_7X7_50,
    "DICT_7X7_100": cv2.aruco.DICT_7X7_100,
    "DICT_7X7_250": cv2.aruco.DICT_7X7_250,
    "DICT_7X7_1000": cv2.aruco.DICT_7X7_1000,
    "DICT_ARUCO_ORIGINAL": cv2.aruco.DICT_ARUCO_ORIGINAL,
    "DICT_APRILTAG_16h5": cv2.aruco.DICT_APRILTAG_16h5,
    "DICT_APRILTAG_25h9": cv2.aruco.DICT_APRILTAG_25h9,
    "DICT_APRILTAG_36h10": cv2.aruco.DICT_APRILTAG_36h10,
    "DICT_APRILTAG_36h11": cv2.aruco.DICT_APRILTAG_36h11
}

Now , to detect our marker we will use cv2.aruco.detectMarkers() ,we need, the type of the marker- i.e. the aruco marker dictionary, and the image that we need to pass the image that contains the markers, Now there is one more parameter to it, which are the parameters of the Detector that we are using, we will initialize it using cv2.aruco.DetectorParameters_create().

arucoDict = cv2.aruco.Dictionary_get(ARUCO_DICT[args['type']])
arucoParams = cv2.aruco.DetectorParameters_create()
(corners, ids, rejected) = cv2.aruco.detectMarkers(
    image, arucoDict, parameters=arucoParams)

This, cv2.aruco.detectMarkers() returns a three tuple where corners are all the 4 points of each marker detected, ids is the detected id of the type of marker that we are detecting, and rejected are the points which were classified as non markers.

Now, we will visualize these markers one by one, using the snippet below.

if len(corners) > 0:
    ids = ids.flatten()
    for (markerCorner, markerId) in zip(corners, ids):         corners_abcd = markerCorner.reshape((4, 2))
        (topLeft, topRight, bottomRight, bottomLeft) = corners_abcd         topRightPoint = (int(topRight[0]), int(topRight[1]))
         topLeftPoint = (int(topLeft[0]), int(topLeft[1]))
         bottomRightPoint = (int(bottomRight[0]),      int(bottomRight[1]))
         bottomLeftPoint = (int(bottomLeft[0]), int(bottomLeft[1]))        cv2.line(image, topLeftPoint, topRightPoint, (0, 255, 0), 2)
        cv2.line(image, topRightPoint, bottomRightPoint, (0, 255, 0), 2)
        cv2.line(image, bottomRightPoint, bottomLeftPoint, (0, 255, 0), 2)
        cv2.line(image, bottomLeftPoint, topLeftPoint, (0, 255, 0), 2)        cX = int((topLeft[0] + bottomRight[0])//2)
        cY = int((topLeft[1] + bottomRight[1])//2)
        cv2.circle(image, (cX, cY), 4, (255, 0, 0), -1)        cv2.putText(image, str(
            int(markerId)), (int(topLeft[0]-10),   int(topLeft[1]-10)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255))        # print(arucoDict)
        cv2.imshow("[INFO] marker detected", image)
        cv2.waitKey(0)else:
    # print("[INFO] No marker Detected")
    passcv2.destroyAllWindows()

Every corner has four points where each point consist of two coordinates(x,y),
and the points are in clockwise order starting from top-left.

Using this we can detect the markers of the type mentioned in the argument.
Now, we will use these techniques to actually calculate real time distance between two markers, Now are going to do that with a little bit of maths , but the point to note here is that — you need to take a print out of the marker and know its actual size in cms or inches or meter.
Now I took a printout of two markers of type DICT_ARUCO_ORIGINAL and each of them had a size of 7.5 cm on each side.
Now using the code below , we can actually visualize the distance between two markers.

import cv2
import imutils
import sys
import os
import numpy as np
import argparse
import time
from imutils.video import VideoStream

we will use live video stream to calculate the distance.

ap = argparse.ArgumentParser()

ap.add_argument(“-t”, “ — type”, required=True,
 help=”tag of the marker to b detected”)
args = vars(ap.parse_args())

We just need the type of the marker to be detected.

vs = VideoStream(src=0).start()
time.sleep(2.0)ARUCO_DICT = {
    "DICT_4X4_50": cv2.aruco.DICT_4X4_50,
    "DICT_4X4_100": cv2.aruco.DICT_4X4_100,
    "DICT_4X4_250": cv2.aruco.DICT_4X4_250,
    "DICT_4X4_1000": cv2.aruco.DICT_4X4_1000,
    "DICT_5X5_50": cv2.aruco.DICT_5X5_50,
    "DICT_5X5_100": cv2.aruco.DICT_5X5_100,
    "DICT_5X5_250": cv2.aruco.DICT_5X5_250,
    "DICT_5X5_1000": cv2.aruco.DICT_5X5_1000,
    "DICT_6X6_50": cv2.aruco.DICT_6X6_50,
    "DICT_6X6_100": cv2.aruco.DICT_6X6_100,
    "DICT_6X6_250": cv2.aruco.DICT_6X6_250,
    "DICT_6X6_1000": cv2.aruco.DICT_6X6_1000,
    "DICT_7X7_50": cv2.aruco.DICT_7X7_50,
    "DICT_7X7_100": cv2.aruco.DICT_7X7_100,
    "DICT_7X7_250": cv2.aruco.DICT_7X7_250,
    "DICT_7X7_1000": cv2.aruco.DICT_7X7_1000,
    "DICT_ARUCO_ORIGINAL": cv2.aruco.DICT_ARUCO_ORIGINAL,
    "DICT_APRILTAG_16h5": cv2.aruco.DICT_APRILTAG_16h5,
    "DICT_APRILTAG_25h9": cv2.aruco.DICT_APRILTAG_25h9,
    "DICT_APRILTAG_36h10": cv2.aruco.DICT_APRILTAG_36h10,
    "DICT_APRILTAG_36h11": cv2.aruco.DICT_APRILTAG_36h11
}arucoDict = cv2.aruco.Dictionary_get(ARUCO_DICT[args['type']])
arucoParams = cv2.aruco.DetectorParameters_create()

We will now use the same code as before to detect the marker , but this time in real time video stream.

CACHED_PTS = None
CACHED_IDS = None
Line_Pts = None
measure = None
while True:
    Dist = []
    image = vs.read()
    image = imutils.resize(image, width=800)
    (corners, ids, rejected) = cv2.aruco.detectMarkers(
        image, arucoDict, parameters=arucoParams)     if len(corners) <= 0:
        if CACHED_PTS is not None:
            corners = CACHED_PTS     if len(corners) > 0:
        CACHED_PTS = corners        if ids is not None:
           ids = ids.flatten()
           CACHED_IDS = ids
        else:
           if CACHED_IDS is not None:
              ids = CACHED_IDS
        if len(corners) < 2:
           if len(CACHED_PTS) >= 2:
              corners = CACHED_PTS
        for (markerCorner, markerId) in zip(corners, ids):
            print("[INFO] Marker detected")
            corners_abcd = markerCorner.reshape((4, 2))
            (topLeft, topRight, bottomRight, bottomLeft) = corners_abcd            topRightPoint = (int(topRight[0]), int(topRight[1]))
            topLeftPoint = (int(topLeft[0]), int(topLeft[1]))
            bottomRightPoint = (int(bottomRight[0]), int(bottomRight[1]))
            bottomLeftPoint = (int(bottomLeft[0]), int(bottomLeft[1]))            cv2.line(image, topLeftPoint, topRightPoint, (0, 255, 0), 2)
            cv2.line(image, topRightPoint, bottomRightPoint, (0, 255, 0), 2)
            cv2.line(image, bottomRightPoint, bottomLeftPoint, (0, 255, 0), 2)
            cv2.line(image, bottomLeftPoint, topLeftPoint, (0, 255, 0), 2)            cX = int((topLeft[0] + bottomRight[0])//2)
            cY = int((topLeft[1] + bottomRight[1])//2)

Now until now the code is more or less the same , but now we will make some changes , we will take the ratio of pixels to cm , as now we know the topLeft and topRight corner we can calculate the Euclidean distance between them and compare them to the actual distance(in this case 7.5 cm) and as after this we will know the ratio , by using two points on two different markers we can actually calculate is using Euclidean formula.

In the code I am taking the center and top-Left point as reference , and the ratio is calculated with the assumption that they in a line , you can change that ratio using Euclidean distance as a reference.

measure = abs(3.5/(topLeft[0]-cX))
            cv2.circle(image, (cX, cY), 4, (255, 0, 0), -1)
            cv2.putText(image, str(
                int(markerId)), (int(topLeft[0]-10), int(topLeft[1]-10)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255))
            Dist.append((cX, cY))
            # print(arucoDict)
           if len(Dist) == 0:
             if Line_Pts is not None:
                 Dist = Line_Pts
           if len(Dist) == 2:
             Line_Pts = Dist
           if len(Dist) == 2:
             cv2.line(image, Dist[0], Dist[1], (255, 0, 255), 2)
             ed = ((Dist[0][0] - Dist[1][0])**2 +
              ((Dist[0][1] - Dist[1][1])**2))**(0.5)
             cv2.putText(image, str(int(measure*(ed))) + "cm", (int(300), int(
            300)), cv2.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255))
          cv2.imshow("[INFO] marker detected", image)
          key = cv2.waitKey(1) & 0xFF
          if key == ord('q'):
             break
cv2.destroyAllWindows()
vs.stop()

And voila! you can actually calculate the real time distance between two markers.

Next, we will see a short article on Augmented reality.

Reference:

PyimageSearch
OpenCV

Augmented Reality
Opencv
Python
Computer Vision
Machine Learning

--

--

Analytics Vidhya
Analytics Vidhya

Published in Analytics Vidhya

71K Followers
Last published Oct 15, 2024

Analytics Vidhya is a community of Generative AI and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Deepanshusachdeva
Deepanshusachdeva

Written by Deepanshusachdeva

41 Followers
6 Following

Making Machine Learning Easy!

Responses (3)

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams