rydesafely — out of stealth and empowering a safer autonomous future

We’ve raised Pre-seed:

With the backing of Entrepreneur First, Rydesafely has finally come out of the stealth mode today upon the raising of our Pre-Seed Round. For those who don’t know what we do, we tackle the massive automotive problem of validating the adequacy, safety, and effectiveness of machine learning (ML) based assistive and autonomous software.

Rydesafely fixes the gaps in the current validation techniques for ML based software in the automotive industry. Inability to validate effectively or in a sustainable way is the prime reason why we haven’t been able to incorporate any meaningful degree of autonomy to this industry.

One main reason why the autonomous future is in a trough of disillusionment is the inability of Automotive manufacturers (OEMs) and their suppliers (Tier-1s) to effectively gauge the safe deployment of such features. As a result, safely deploying them in any scope (aka Operational Design Domain or ODD) — let alone scaling them to other ODDs — has become a formidable task. While the size of this problem might deter some, we see this as an opportunity to make a positive impact as the world currently faces annually about 1.35 million deaths and about $1.7 trillion loss.

A bit more about the problem we are solving:

ML has opened the doors to fix many of humanity’s problems that were thought unsolvable even a decade ago. While this has given way to new opportunities, it has posed unprecedented challenges in validating their effectiveness and safety. This isn’t a problem for a company like Netflix and their TV recommendations… but can be the difference between life and death when applied to a car.

So what are the main problems with ML systems:

→ their notorious nature of failing silently

→ the inability of the traditional software testing methodologies to be sufficiently interrogate & check the logic learnt comprehensively

The bottom-line is ML systems can learn spurious logic that depends on the many moving pieces like training datasets, hyper parameters, model architecture etc.

An simple example of spurious logic learned by machine learning ADAS is if a system is only trained in broad daylight and then correlates certain pedestrians exclusively with bright surroundings

For example, a pedestrian detection system that is trained on a broad daylight might incorrectly correlate pedestrians exclusively with a bright background. The fact that current methods to solve this problem can be misleading — like the widely used evaluation metrics -does not help either. As a result, such incorrect logic would fail embarrassingly at night or on a rainy day, while still yielding good results on commonly used evaluation metrics on daylight based dataset. This was just a simple example. What makes fixing this problem a tough nut to crack is the complexity and unusual nature of the real-world. The unspoken and obvious semantics of a real scene are actually the toughest to gauge.

Beyond the initial detection of ML system spurious logic, there is a wider problem. There is also an inability to create software tests that can assess the system quality reliably. This leaves unanswered questions. It is not clear how to define tests for systems for coverage, adequacy etc, especially in safety-critical situations where even a small mistake could turn fatal. Traditional software engineering methodologies are not suited for testing machine learning systems. Reason being they were made to test where a logic is known beforehand, not something that is supposed to be figured out on it’s own. Traditional automotive metrics like coverage have no meaning when it comes to machine learning.

So what do we do about it:

Our platform assesses the ML model and identifies the key failure modes. It does so by interrogating the logic that resides in the weights of the model to gauge the efficacy of key concepts it has learnt. In simple terms, it can identify when and where automotive client’s machine learning models will fail for a specific ODD.

In addition, we then can combine such failure modes and some domain/ODD specific prior data to create software tests artefacts (inputs and test oracles). They are generated to expose the faults in the system that can occur due to factors such as inadequate training data, edge-cases, behaviour regressions etc. Further, we use novel test adequacy metrics which can comprehensively cover for all important situations for a given ODD. Our tests are a game-changer in the sense that they create realistic situations without violating the semantics of the real-world. These are applied at the levels of model-level testing and integration-level testing.

By the ability to detect chinks in the armoury of the ML software, our platform reduces the number of times OEMs, and/or Tier-1s have to get to expensive on-road testing and be smart about tedious data-gathering step. Effectively, this expedites their software-Development-Life-Cycle of their ML based software.

Consider joining us:

rydesafely has set out on this journey to help OEMs, Tier-1s make roads safer by offering them a one-stop platform to effectively validate their ML based software. This is indeed a hard-problem, but we don’t want to have it too easy, do we? If you want to be involved in the pursuit of solving a hard problem, whilst changing the way the world operates for the better, then consider joining us here

Consider licensing our platform:

If you’re an OEM, Tier-1 or tech player in need of faster and more robust detection of weaknesses in your neural network models, then reach out here via email, LinkedIn, call or check out our website at www.rydesafely.com. We can help