[Team Experiments] ComputedFuelLevel : Working to improve a noisy signal
As you might know, at Xee Engineering we are using the CAN Bus to get signals from cars.
Every signal value we provide has been read (and only read).
The way the signals are broadcasted on the CAN Bus by the car is completely independent of our work. We are just here, reading what’s going on.
A “good” example of that is the way we are reading the Fuel Level signal.
To read the fuel level of the car, we are looking for the signal computed from the tank’s sensor raw data.
This works fine in most of the cases, but with the time, we started to see multiple side cases.
Finally, this signal was sent in 3 different ways (depending on the car maker and model)
- In a continuous way.
Well, not really continuous but, calculation made by the sensors is smoothed with a resolution of more or less 1L.
- In a step way.
Like continuous but with a 2L~5L resolution.
- In an raw way.
Data not smoothed at all, the raw data comes from the sensor directly.
The thing is, working with this signal is very difficult because you don’t know what you got till you ask for it.
We really wanted to provide a “great”, or at least good, signal for all the cars. But how?
We tried a lot of things, working on the signal itself, working on the way we send it to the platform but, in the end, here is what we’ve done:
Like one of our teammate said:
As a sound analogy, the raw signal we got was very noisy, almost only by a white noise.
First approach was to add a moving average filter.
Then, because the Fuel Level has to be a low frequency, almost a continuous signal, so we’ve added a quite violent low pass filter.
If we sum up, here is what is done:
- Remove all the white noise we can / Filter the “high frequencies” changes
- Reduce the noise amplitude
- Remove absurd values
The pros the cons
With this filter, the results we got were quite good.
Here are the results of the signals above filtered (red is the Fuel Level, blue the Computed Fuel Level):
So here are the pros:
- The continuous signal was great and still is
- The step signal has not changed a lot as you can see
- The last one, with a lot of “jumps” in values is very smoothed and looks like the step one
And the cons:
- It has a few delay to compute (due to some filters)
We are very happy to provide a smoothed signal for all the cars (for which the Fuel Level is available as raw data).
But we also want to provide the raw data as we read it from the CAN Bus.
So what we decided is to provide a new signal in our API!
This signal is named ComputedFuelLevel and comes with the FuelLevel, we do not erase the “raw” one to provide the “computed” one.
You can request this new signal like you were doing for the other ones. And we created a new section in the signals list, which is Computed.
You can see the full list here https://github.com/xee-lab/xee-api-docs/blob/master/api/api/v3/cars/signals_list.md
Here is a small list of tests we made to ensure the algorithm of filtering! Enjoy ;)