Reliability. Choosing an indoor positioning technology
This post is part of the “Choosing an indoor positioning technology” series. The series aims to help anybody evaluating and buying Indoor Positioning Systems (IPS) to ask the right questions and make better informed decisions. This post focuses on Reliability. If you are only after the core message and evaluation criteria, you can skip ahead and simply focus on the bolded text.
Series Background
Choosing an Indoor Positioning System for use cases such as indoor wayfinding is often a daunting task. Complex technologies, foreign terminology and a large pool of vendors with varying value propositions can easily overwhelm anyone. Most companies will pitch you their solution or compare different technologies and their respective advantages and disadvantages. However, the truth is that the fit of an indoor positioning technology depends upon the use case you want to solve. Therefore, instead of pitching our solution and contrasting its performance to others, we thought we will give you a tool to evaluate different technologies for your intended use case.
More specifically, we will present to you a range of suboptimal questions we see many customers ask and explain why they are suboptimal. Then we will present to you a set of questions we believe are more appropriate given prior explanations. These questions you should ask yourself and/or the vendor you are evaluating, in order to make a better decision on the solution you need.
The wrong questions to ask about reliability
When evaluating indoor positioning solutions, the focus usually rests on comparing the accuracy of different technologies. This of course has its merit, since accuracy and the lack of it indoors, is the sole reason for the existence of indoor positioning technologies. Evaluating the accuracy of different technologies is important and we have previously written about the right questions customers should ask here. However, figuring out the right level of accuracy is only the first part. The second part is to understand the reliability of that accuracy.
The wrong questions to ask about reliability is not to ask any questions. Most customers tend to solely focus on accuracy itself, as mentioned above. They are often unaware that most technologies’ accuracy will fluctuate greatly. Vendors usually provide the best-case accuracy for their solution which often contains two big assumptions: (1) static environment and (2) homogenous indoor layout.
With static environment we mean that there is no changes in indoor layout and no movement in furniture and even people. With homogeneous indoor layout we mean that the rooms, corridors, open spaces, ceiling and walls are the same throughout the entire indoor space. Unfortunately, the indoor environment is constantly evolving and very messy. People are running around, furniture is being moved, walls are torn down and new walls are being put up. Rooms might have different ceiling heights compared to hallways or open areas and the walls might not be equally thick everywhere.
As you can see, both assumptions of (1) a static environment and (2) a homogenous indoor layout are highly unrealistic. They lead to great variability in the stated accuracy and hence, it is important to evaluate the reliability of accuracy for different indoor positioning technologies.
The right questions to ask about reliability
Reliability of the accuracy will vary from technology to technology and even from vendor to vendor. Hence, simply asking vendors about the reliability and drivers of their technology’s accuracy is a great start. However, if you want to do a more systematic assessment you should ask yourself and/or the vendor the following questions:
- How reliable do I need the room/zone accuracy to be for the use case and/or a good user experience?
There are two types of reliability: deterministic (100% certain) or probabilistic (<100% certain). Ask yourself how important room accuracy is for your use case and the user experience. Is it ok if the technology sometimes falsely states a person/object is in a room or outside of the room? High reliability is especially important when the stakes are high (e.g. use cases for prisons, psychiatric wards, visually impaired) or when users expect it (e.g. indoor navigation). - How reliable do I need the 2D/3D accuracy to be for the use case and/or a good user experience?
Here technologies fall on a reliability spectrum ranging from low to high. Analyzing different technologies and placing them on the spectrum will not only allow you to focus on the right technologies for you, but also make you understand the price differences between them. More reliability usually comes with a higher price tag. - Does this technology require any calibration/fingerprinting?1
As a rule of thumb, the accuracy of technologies that require calibration/fingerprinting is often less reliable. After the initial calibration the accuracy deteriorates almost immediately due to the constant movement of people and furniture indoors. In the best case, reliability can be somewhat increased through continuous re-calibration. However, this process is time intensive and not scalable. - Will this technology suffer from interference of other signals in my indoor environment?
Since indoor positioning systems are based on signal processing, the accuracy’s reliability can deteriorate if other similar signals are introduced. For example, you can imagine that light-based positioning technologies might suffer from interference from room or sun light. Most technologies will suffer from a certain degree of interference, but some do more than others. Therefore, understanding your environment and the signals present is crucial to assess the degree of interference. - What are other drivers for the accuracy of the technology?
While this is a broad question, it ensures that you cover your bases. Understanding what other drivers of accuracy are will allow you to infer potential risk factors for the reliability even if we have not covered them in one of the four questions above.
In summary, reliability of accuracy is as important if not more important than the accuracy of the technology. Being based in Norway who has one of the highest rates of electric cars, allow us the following analogy. You do not want to buy an electric car with a stated 500km in range to then realize that the 500km estimate is only reliable if the car is driven under 10km/h.
1 Calibration/Fingerprinting: Reliability and consistency of indoor positioning solutions is often dependent on calibration. Calibration or fingerprinting is the concept of mapping out venues for signal strength (e.g. WiFi strength at any given point in the building). Calibration is important for some technologies since the position of a person will be assessed based on the signal strength between the signal receiving device (e.g. person’s phone) and signal emitters (e.g. WiFi access point). Positioning technologies that require calibration are less reliable, because the constant change of furniture and human presence in indoor environments alters the signal strengths. For example, after calibration as soon as you turn around and your back is between the signal emitting and the signal receiving device, you have altered the signal strength since now your body is in the way.
Originally published at https://forkbeardtech.com on April 6, 2021.
