Smoothening noisy GNSS data
Most of us are fairly regular in ordering our favorite food through Zomato or Swiggy. You must have checked the tracking feature present on such apps while waiting for your food to get delivered. The route between the food outlet (source) and your home (destination) visible on the app is smooth and clear, and it feels great to watch the rider travel along those paths. A similar feature is present on our Simha app, wherein a customer can track the entire activity (traveled path) for the day. Sometimes due to the presence of noisy points captured by the device, the travel path isn’t smooth, which affects the user experience. Have a look at the example below.
To address this, we first endeavored to identify the root cause of the issue and find possible solutions. After carefully observing the patterns in the data, it was clear that some environmental conditions are affecting the data captured during trips. It was quite difficult to know when such conditions might occur; the noise was completely random in nature. Therefore, we shifted focus to ways of mitigating the noise in the path. The data collected from the device is time-series data, and a better way of dealing with such data is filtering.
Several filters were experimented with to find out the best one that suits our data, and finally, we found it. The Savitzky-Golay filter came to our rescue. Let’s understand this filter. The Wikipedia description for Savitzky–Golay filter is a digital filter that can be applied to a set of digital data points to smooth the data and increase the precision of the data without distorting the signal tendency. Before diving into it, let’s look at the example below.
The signal above is the sine wave which got distorted due to the presence of some noise. The SG filter, in simple terms, considers the window of points (w) and fits these points on a polynomial of degree (n) to generate smooth data. The parameter (w) and the polynomial degree (n) are the hyperparameters and can be tuned as per the data to get the desired output.
Let’s have a look at the python-based implementation of the above example.
# Import Necessary Packages
import numpy as np
import plotly.graph_objects as go
from scipy.signal import savgol_filter# Define Noisy Sinusoidal signal
time = np.arange(0, 10, 0.1);
f = np.sin(time)*10
f1 = [np.random.randn(1)[0]+i for i in f]
# Plot the noisy sinusoidal signal
fig = go.Figure()
fig.add_trace(go.Scatter(x=time, y=f1,
mode='lines',
name='Distorted Sinusoidal Signal',
))
fig.update_layout(title="Distorted Sinusoidal Signal")
fig.update_xaxes(title_text='Time')
fig.update_yaxes(title_text='Amplitude')
fig.show()# Apply the Savitzky- Golay Filter on the noisy data
f2 = savgol_filter(f1, window_length = 17, polyorder = 2)# Compare the smoothen output with the original input
fig = go.Figure()
fig.add_trace(go.Scatter(x=time, y=f1,
mode='lines',
name='Distorted Sinusoidal Signal',
))
fig.add_trace(go.Scatter(x=time, y=f2,
mode='lines',
name='Smoothed signal'))
fig.update_xaxes(title_text='Time')
fig.update_yaxes(title_text='Amplitude')
fig.update_layout(title="Signal")
fig.show()
After applying this filter to the data with a proper set of parameters, the noise from the signal gets completely removed. The plot in red now closely resembles the sine function. We were able to get rid of the noise that was present in the signal. This approach can be extended for the traveled path data to make it smooth.
Let’s see how the travel paths were present before and after the use of this filter. First, we identify the optimal value of window size and degree of the polynomial based on our data. And then use these values in a filter to smoothen our data. This process, obviously, involved several iterations. The results obtained after the usage of filters were quite promising and acceptable. Have a look !!!
After applying this filter, the noise from the traveled path gets satisfactorily removed. We also checked whether this filter affects the smooth travel path, which is free from noise. And as expected, it didn’t. Clearly, with the usage of this filter, we can definitely observe a boost in the user experience of Simha customers!!!!
