How well does my robot work?

RoQME: A project aimed at measuring quality of service in robotics

Photo by Alex Knight on Unsplash

In the coming years, service robotics is called to play an important role in our lives. Robots will not be limited to simple domestic tasks, but they will end up being our companions, co-workers, assistants in shopping centers, or caregivers for our elders, to name just a few examples. Much of the research in service robotics focuses on functional aspects, that is, on the development of the robot tasks. Note that domestic, outdoor or crowded public spaces, like any other real-world environment, are inherently open-ended and, therefore, show a huge number of potential situations and contingencies. This poses a great challenge to the functionality of a robot and, as a result, the non-functional aspects are usually forgotten.

But, what do we mean with non-functional aspects? Broadly, non-functional aspects (a.k.a., non-functional or extra-functional properties) define “how” a system performs rather than “what” it does. Examples of non-functional properties include safety, reliability, efficiency or usability, among others. These properties can be observed during the robot operation to provide information about the quality of its service. Although engineers can use this feedback to evaluate and improve their designs, ideally, a robot should be able to adapt its own behavior automatically in order to optimize certain non-functional properties, such as user satisfaction or energy consumption.

RoQME (Robot Quality-of-Service MEtrics) is a new project coordinated by the University of Extremadura and participated by Biometric Vox and the University of Málaga. It is a 1-year project, as of March 2018, funded by the EU Horizon 2020 as part of the RoMoSys project. RoQME aims at helping roboticists to deal with non-functional properties by providing them with a toolchain to create global Quality of Service (QoS) metrics. The automatic estimation of these metrics at runtime, based on the contextual information observed, can then be used for different purposes, such as the adaptation of the robot behavior, among others.

A robotic salesman as an illustrative example

Imagine that we want to build a social robot to promote products and services in a shopping center. The main goal of this robot will be to move around, interact with people, and show them offers and discounts on its screen. In the following, the development process of this hypothetical robot will help us to illustrate the intended goals and main ideas behind the RoQME project.

1. The robot body

The design and construction of the robot body will require a lot of work, but it should be possible to have something like a mobile platform on wheels, equipped with a computer connected to a LIDAR sensor for location, navigation and obstacle avoidance; a microphone, a touch screen and speakers for human-robot interaction; and a camera for people recognition. Despite its potential, at this point, the robot will be just a big hole in our budget. It will be useless until some software provides some interesting functionality.

Figure 1. A possible design of the salesman robot. Image obtained from CLARC: a Robotic Architecture for Comprehensive Geriatric Assessment

2. Programming the functionality

Developing robotic software from scratch usually involves a huge effort. Note that it requires implementing code for a variety of problems, such as motion planning, simultaneous location and mapping (SLAM), tasks coordination, computer vision, or speech recognition and synthesis, among others. For this reason, the first step towards a cost-effective software development is “reuse”. But, what is reuse? Does it mean building an amalgam of copy-pasted pieces of undocumented code, which suddenly, after patching again and again with no principle, ends up running, in an unpredictable way but functional, at least for 10 minutes under certain fixed conditions? Obviously not. Software reuse implies taking advantage of code that has been specifically conceived for reuse. As building blocks, this code encapsulates functionality, hiding unnecessary implementation details under a well-defined interface. Therefore, as far as possible, the role of the engineer in charge of developing the software for a robot should be composing the final system out of existing components, with the consequent increase in productivity. This is the idea behind Component-Based Software Engineering (CBSE) and the driving force of the RobMoSys project (titled Composable Models and Software for Robotics Systems).

Going back to our example, we can use the SmartMDSD Toolchain to develop the software of our salesman robot. This is an Integrated Development Environment (IDE) for robotics that is compliant to the RobMoSys approach. In short, it offers a graphical modeling editor that allows us to design our robotic system by selecting and connecting existing components or new ones, specifically created by us, for our robot. Once we complete our graphical model, we can automatically generate the software to be executed in the robot by simply pressing a button. Although this process involves many details that we have not explained, in essence, this kind of practice seems much better (more maintainable, cost-effective, extensible, less prone to errors, etc.) than programming from scratch or carrying out a “copy-and-paste” software reuse (or rather, misuse).

Figure 2. Screenshot of the SmartMDSD Toochain

3. Dealing with non-functional properties

At this point, our salesman robot is perfectly functional, and now it has a soul (made of software), it would be appropriate to give it a name: let’s call it Robbie. Thanks to its ability to avoid obstacles and plan routes, Robbie can move through the hallways of the different shops in the mall. Besides, as its computer vision algorithms allow it to recognize people, Robbie takes the initiative and introduces itself to those passing its way using its synthetic voice. Once the attention of a customer is captured, Robbie displays commercial information on its touch screen taking into account his/her preferences and questions.

All these features refer to the functional part of Robbie, but what about the non- functional aspects? How well does Robbie do as a salesman? How can we measure its performance?

To answer these questions, first, we have to establish the non-functional properties under which Robbie is going to be evaluated. A service can be good or bad depending on the criteria considered. For that reason, it is not the same to look at the service paying attention to energy consumption or under the prism of customer satisfaction. In our example, we are going to consider “effectiveness”, which, like most non-functional properties, is an abstract concept with a vague interpretation. However, in this case, we will define effectiveness as the degree to which Robbie is successful in producing the following desired results:

• Acceptance among customers.
• Robbie manages to attract the interest of the customers.
• Robbie engages customers in the service, achieving a high level of interaction.
• High ratio of customers served to customers detected.
• Robbie covers most of the shopping center.
• Robbie does not bump into anything or anybody.

Now, we need to quantify how “effective” Robbie is. For that, we use a QoS metric, which expresses, as a real number in the range [0,1], the degree of fulfilment of the non- functional property. In other words, our QoS metric will be a score of how well the system performs according to the previous definition of “effectiveness”. If the metric is 1, the system is optimal in terms of this property, while 0 indicates the opposite. Besides, this value can change as the behavior of the robot evolves during its operation. Thus, the system will show an improvement if, in a period of time, this metric varies from 0.43 to 0.52, or a deterioration if the variation trend is the opposite. Lastly, as the challenge is to evaluate subjective properties, QoS metrics should not be taken as accurate and rigorous measures, but as rough indicators or estimates. With all that, RoQME ambitions to provide a comprehensive set of tools enabling (1) the specification of QoS metrics, defined in terms of the contextual information available; and (2) the continuous evaluation of these metrics at runtime.

4. Modeling QoS metrics at design time and using them at runtime

One of the objectives of RoQME is to provide a domain-specific language (DSL) to specify QoS metrics. In essence, the key modelling concepts that users will be able to manage are (in bold): context variables (e.g., battery level) and, from them, relevant context patterns (e.g., “the battery level drops by more than 1% per minute”). The detection of a context pattern will be considered an observation in a belief network, which is used to define the dynamics of QoS metrics (e.g., energy consumption).

Following our example, Table1 shows the context variables that could be specified with RoQME. In addition, Table 2 includes some context patterns defined in terms of these variables. The detection of a pattern will be a positive (or negative) observation that reinforces (or reduces) the belief that Robbie is optimal in terms of effectiveness. Basically, the corresponding QoS metric is the quantification of this belief after applying probabilistic reasoning. That is, a value of 0.67 can be understood as the Robbie’s probability of showing maximum effectiveness.

Once the QoS metrics have been specified, we will have to deploy them on the RoQME runtime platform, installed in the robot. This platform will consist of (1) a number of monitors, in charge of updating the context variables from raw contextual data; (2) an event processor, to search for the event patterns described in the RoQME models and, when found, to produce observations; and, finally (3) a probabilistic reasoner that will compute a numeric estimation for each metric. At runtime, this information will be accessible to other components so that it can be used for different purposes, such as adapting the robot behaviour or benchmarking. Finally, Figure 3 shows a simple example of how effectiveness would vary during the operation of Robbie.

Table 1. Context variables for Robbie

Table 2. Context patterns for Robbie according to the definition of effectiveness previously introduced

Figure 3. Example of the effectiveness evaluation at runtime

Why to use RoQME?

The RoQME framework, delivered as an Eclipse plug-in, will provide both modelling and code generation tools, enabling the specification of QoS metrics and their continuous evaluation at runtime. Although RoQME is currently a project in progress, we believe that its results will not only be valuable for the robotics community but also applicable to other domains, such as soft biometrics. As a conclusion, in the following, we summarize the main advantages that the use of RoQME will bring.

• RoQME promotes a systematic, model-driven, and tool-supported method to develop QoS metrics. This approach contrasts with the ad-hoc implementation of the QoS concerns, e.g., using C++ (or any other programming language), which usually leads to increased complexity and poor reuse.
• Unlike general-purpose programming languages, the RoQME DSL addresses a very particular problem in a more compact and efficient way thanks to its specialized features. In this vein, among other elements, we plan to include advanced operators to express trends and correlations in context data. Moreover, the RoQME framework will allow designers to validate the syntactic and semantic correctness of their QoS metric specifications in order to significantly reduce runtime errors.
• In RoQME, a QoS metric specification can be transformed into a probabilistic network, which is a well-known mathematical abstraction that has been successfully applied to many domains, such as medical diagnostics. As a result, we can benefit from existing tools and techniques that are used for the analysis and simulation of probabilistic networks.
• The RoQME framework has been designed to be loosely coupled with the robotic architecture, so that, although a continuous monitoring of the context may have an impact on the robot performance, the runtime platform of RoQME could easily be activated or deactivated without affecting the basic functionality of the robot.
• RoQME promotes the application of QoS metrics to develop new robotic features. For instance, robot self-adaptation at runtime based on QoS metrics.
• RoQME is an Integrated Technical Project (ITP) of the EU H2020 RobMoSys project, which is a major effort to realize a step-change towards a European ecosystem for software development in robotics.
• All the RoQME software artefacts and documentation will be distributed using open-source and open-access policies.

RoQME has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 732410, in the form of financial support to third parties of the RobMoSys project