Know Inside Position — Visual Cues

Published in

Managing Drone Flights Inside

12 min readMar 11, 2019

Smart Visual Internal Positioning Ecosystem (SVIPE).

Pronounced as SWIPE

UAVs (also referred to as drones) have wide ranging use inside large buildings for example large warehouses. They are relatively low-cost and can freely roam the facilities without interfering with other entities. They also can reach any height within the building. They can carry a variety of attachments, for example cameras, which can simplify operations.

A primary challenge inside the buildings is to know their exact position in the 3D space inside. GPS cannot be reliably used and typically due to large amount of metal and other radio noise — traditional methods of positioning such vehicles is not easy.

The solution described here provides following capabilities:

Accurately know the position of the vehicle inside the building’s 3D space.
Accurately orient the vehicle
Rely on visual cues

Solution Overview

Our solution will rely on following attachments to the UAV.

1. A digital view finder mounted on top of the vehicle

2. A measurement instrument to measure distance to an obstacle.

3. A gyroscope so that above items are always pointed up

The basic concept is that we would have standardized homing placards mounted on the ceiling of the building. These placards will have following properties:

They will be of a material that our “distance measuring instrument” (referenced above as #2) can reliably detect
They will have a clear visual cue that is unique to each placard which can be easily detected by the digital view finder at any angle.
The visual cue will also assist in orienting the UAV.

The UAV will detect these visual cues and fly below one of the closest placards. It will then use the visual cues to orient itself to attain the desired vector. While in flight; it will have the capability to detect the placard directly above. The above capabilities will allow the drone to reliably operate within the building.

Terminology

Following terminology will be used within this document:

Understanding the 3D Space

The 3D space inside a building can be viewed as follows:

Another way to look at the 3D space and an object inside can be as follows. The cube can be any object, for example a UAV or the placards.

So when we are talking about accurately knowing the UAV position — we are talking about its X,Y, and Z position in the 3D space.

X and Y typically refers to one of the dimensions on the flat surface
Z typically refers to the height

Note that X, Y, and Z are arbitrary when applied to a physical object. When these are referenced below, it is assumed that the arbitrary nature of the coordinate system is understood. For example, it is completely valid to imagine a building as if its horizontal surface is imagines to be Y and Z.

The whole building can be imagined as a cube. We can then choose any unit, e.g. millimeters and start from one corner of the building and map out every position along the floor with the millimeters from the corner. These will serve as X and Y coordinate. Then from the same corner we can measure distance to the ceiling in the same unit — and that will serve as Z coordinate.

Longitude and Latitude Data (NMEA Compatibility)

Note that if we knew the longitude, latitude, and height of the origin, we can determine the position in terms of these units as well. So this solution can easily publish the NMEA compatible data.

Homing Placard

Following is an example of such a homing placard. The exact visual cues can be flexible — the only requirements are that:

a. They can be easily detected by our view finder

b. Each visual cue is distinct within the domain that we are operating.

c. It needs to have a cue to assist in orientation.

d. It needs to be able to read the placard identifier from any angle.

For identifying, one option may be a barcode — but it may also be a text that can be recognized — for example:

112233445566778899 is a sample visual cue. We may choose to repeat the identifier to improve accuracy as described in the sample above.
The line in the middle is to assist in orientation. This is aligned with the “X Axis” in the space defined above.
The triangle in the exact middle is to assist in orienting
The two circles on the end are to easily detect the ends of the line.
The exact dimension of the placard is known in the database.
The exact distance between the center (triangle) and the ends (circles) is also known
The exact position of the placard in the 3D space will be known

View Finder Orientation Tool

The view finder will have the following view inside it that will be super-imposed on to the homing placard to allow the UAV to orient itself:

Figure 5

This allows us to do the following:

If the line here gets super-imposed onto the line of the placard, then it is aligned on the X axis
If the triangle intersects with the triangle of the placard, then it is aligned on the Y axis
If the direction of the triangle matches the direction of the triangle on the placard, then its orientation is known.
If the triangles intersect then if the lines are not super-imposed, we can detect the angle and as a result orient it in a different direction relative to the placard orientation.

Distance Finder

If the UAV is under a homing placard, then the distance finder’s reported distance will accurately place the UAV in the 3D space. We will then know X, Y, and Z position.

Homing Placard Database

Each “homing placard” will be stored in a database along with its position and dimensions. The UAV will have access to this information. Following is a sample representation of such a table:

The X,Y,Z are for the center of the triangle in the placard
X1,Y1,X2,Y2 define the rectangle of the homing placard
The line is assumed to be at the center of the placard, so we know the exact position of that as well based on above data.
Based on above, we also know the exact position of the ends of the line and as such we know the exact distance from the center of the line to the ends.

Above database represents minimum information for positioning and orientation.

Homing Process

This describes the process of homing the UAV such that its position becomes known accurately. Note that accuracy here is subject to acceptable margin of error.

The view finder will look for any homing placards within its visual range.
It will choose one of them for homing
It will achieve a safe height (along Z axis). It can determine distance to the ceiling. It is understood that height achieved before homing is complete may not be completely accurate.
It will move along X and Y axis until its view finder’s center intersects with the center of the homing placard (i.e. two triangles intersect)
It will then rotate to make sure that triangles are pointed in the same direction.
It will then fine tune to ensure that the lines are super-imposed

Following is an illustration of the result when the homing process completes. Red line is the view finder and black is the placard. The red line is super-imposed on the line at the center of the placard:

Note that when homing is achieved we accurately know X and Y coordinates. At this time the distance reported by the “Distance Finder” is the distance to the homing plate. And since homing plate’s Z position is known — we also know the Z position of the drone.

The view finder is looking at the space in two dimensions. At this known distance from the plate, it is important for the UAV to measure distance as well so that while flying it could verify the path. To do that:

It will analyze the image and calculate the distance at this scale from the center to each of the ends of the line on the placard.
Since we know the actual distance as well, using the ratio of actual distance to the distance between two points in the view finder; we can accurately determine the distance to the center of any placard by simply analyzing the data from the view finder (assuming the height is known) (concept #A1)

As a result:

We know the X, Y, and Z of the UAV within our acceptable margin of error.
We also know the vector that the UAV would travel within our defined margin of error.
At any time during flight, while Z is known (i.e. height is known) based strictly on the 2D image viewed by the view finder we will be able to measure the actual distance in XY plane of the drone from any of the placards in its visual range.
Refer to concept #A1 above. At the time of homing we analyzed the image as captured by the view finder. If the drone flies at same height (assuming drone capability to maintain the height within acceptable margin of error) — the distance from any placard’s triangle to the triangle of the view finder will be enough to determine the distance to that placard within acceptable margin of error.

Setting Flight Vector

Once the UAV is homed, we can rotate it to achieve any target flight vector from that plate. For instance, following illustrates a 90-degree turn facing left after homing has been completed. The red image is the view finder and black is that of the placard:

In Flight Waypoint Verification

While in flight when the center of the view finder (triangle) goes over any placard, at that time it can verify its position again. As it passes under a placard, it will also accurately determine the distance from the center of the view finder to the center of the placard. This will be important to guide the UAV in-flight. Due to random interference during flight, it will not be possible to assume that it will always be flying directly under the homing placard such that the two triangles always intersect; but as it is flying if the distance between the two triangles remains in an acceptable margin of error we can let the flight continue. When we violate the margin of error, we can ask the done to perform homing again.

Illustration

Here is an illustration of the discussion.

Building Layout

Let’s say building is as follows:

Width (X axis) is 1000 feet (304800 mm)
Length (Y axis) is 3000 feet (914400 mm).
Height Z axis is 100 feet (30480 mm)

The placards will be placed 5 feet (1524 mm) below from the ceiling. This implies that Z axis of the placards will 95 feet (100–5) or 28956 mm
For this illustration let’s say the placards will be placed to cover the whole width of the building (X axis)
And the placards will be placed at specific intervals along the length of the building (Y axis) at every 20 feet (6096 mm)
So, if we were to look at the ceiling we will see 50 rows of placards spaced every 20 feet.

Please note that this is strictly for illustration. The solution does not require the placards to be placed evenly. Only requirement is that the position of placards is known.

Homing Placrd Layout

Our homing placard dimensions are:

Width (X axis) is 4 feet (1219.2 mm)
Length (Y axis) is 2 feet (609.6 mm)
Line at the center is 4 feet long and at the middle of Y axis, i.e. at offset of 1 foot along Y axis
Distance from center of the line to end of the line is X axis width / 2 or 2 feet (609.6 mm)

Sample Database

We are going to show data for 3 homing placards for illustration. The coordinate unit will be mm (millimeters). Let’s assume that our coordinate system (X Y Z system) increases values on X axis as we move to the right. And increase values along Y axis as we move towards the front of the building from the point of origin (i.e. our coordinate system here matches the one illustrated in Figure 3). Note that X and Y position here is of the center of placard (where we have the triangle)

Placard #1

Identifier of PL1.
Y axis position (position along length of building) is at 40 feet (12192 mm)
X axis position (position along width of building) is at 200 feet (60960 mm)

Placard #2

Identifier of PL2.
Y axis position (position along length of building) is at 60 feet (18288 mm)
X axis position (position along width of building) is at 300 feet (91440 mm)

Placard #3

Identifier of PL3.
Y axis position (position along length of building) is at 80 feet (24384 mm)
X axis position (position along width of building) is at 200 feet (60960 mm)

Since placards are evenly sized of known dimensions X1, Y1, X2, Y2 can be computed as follows:

X1 = (X — placard_width/2)
Y1 = (Y — placard_length/2)
X2 = (X + placard_width/2)
Y2 = (Y + placard_width/2)
Since line is at the center, Y position of the line at the center of the placard is that of the placard itself. For the length of the line, it will be from X1 to X2 with same Y

Illustration of estimating distance to any placard’s center while moving

For this illustration we are assuming that view finder has a square view, so the image is shortened by same dimensions along X and Y axis. If that is not the case concept of aspect ratio will come into play (https://en.wikipedia.org/wiki/Aspect_ratio_(image)) .

Let’s say we want to fly drone at 5 feet below the placards which is 90 feet. So, the Z axis of the drone during level flight is 27432.

Once the UAV is homed, we know that it is directly under the homing plate and the two triangles are on top of each other. We also know the distance of the UAV from the homing plate along Z axis accurately based on the distance measuring tool onboard. At this time let’s say that the view finder determines the homing plate line at the center to be 3 inches long, i.e. 76.2 mm.

Since we know the actual length of the line to be 1219.2 mm we know that along the X axis the image in view finder reduces as 1219.2 / 76.2, i.e. factor of 16. So as long as we assume level flight, the distance of any placard center (triangle at center) to the current position of the drone in X-Y plane can be determined by utilizing distance formula (https://www.chilimath.com/lessons/intermediate-algebra/distance-formula/) along the X Y plane:

And then adapt it using the ratio described above. If the view finder is not perfect square, the result can be further adapter by employing concepts of aspect ratio.

Summary

There is an established need to utilize these UAVs within the confines of large buildings such as warehouses. The above technology would allow us to position the UAV accurately and we will be able to verify its flight path within our acceptable margin of error as well.

Once a UAV’s position is accurately determined and its orientation can be accurately set and in flight we can verify its path easily — navigating inside a building will become quite simple.

The actual flight path can then be planned in terms of the homing placards that the UAV needs to fly under during flight. A turn can be described as well.

The concept is general and UAV software can be pre-programmed with these concepts so that they can be “inside ready”. Installing the homing placards on the ceiling will be the only requirement — but going forward, it can be incorporated into the new building construction to put such placards at the time of construction to reduce the cost. We can assume millimeters as a standard unit and as such putting the placards and providing the exact data at the time of construction will reduce the cost even further.