Syto — pure Scala library for filter design/implementation and digital signal processing

5 min readSep 10, 2019

One year ago I worked on a project that extensively used digital signal processing methods, especially data filtering. The prototype was implemented with Matlab that has a quite mature filtering toolbox but production technology stack consisted of Spark/Scala. At first glance the task seemed to be trivial — translate approved and validated signal processing logic from Matlab to Scala.

How did I get the inspiration to create Syto?

Having done a lot of investigations I discovered a lack of appropriate tools in Scala domain. But the solution was found — invoke an external Python workflow based on Scipy within from Spark job. It made me implement a bridge between Spark Scala logic and Python library, using pipe technology, which I mentioned in one of the previous posts. Scipy is a great Python library that contains digital signal processing functionality as well as many other scientific features and the project’s needs were covered successfully with this integration. But it gave me some food for thoughts — why there was no Scala library that provided digital filter design/implementation? Wouldn’t it be nice to have JVM-based functionality to be used by Scala developers seamlessly? This is how I decided to dive into digital signal processing theory and create Syto.

Syto (ˈsiːto) a Ukrainian word, meaning sieve — a mesh strainer, which can be considered as a trivial mechanical filter. I decided to borrow this word from my native language to express the main idea behind a toolbox being under development. This is pure Scala open-source library for digital signal processing (DSP in further), primarily oriented on filter design and implementation.

I understood that I would not be able to implement all DSP functionality existing in Scipy by the first release. So, the idea was to focus on digital filters that are an essential part of DSP and IIR filter family was chosen as a candidate for the debut.

When do you have to use Syto?

The answer is simple: you have to filter some sequence using Scala. Usually, scientists prefer Python, Matlab or R to tackle their needs. But sometimes it happens the technology stack is based on Scala. The very first use case can Spark which is originally written in Scala. However, I tend to leave the integration of Spark and Syto out of the scope of this post because I’m going to cover this subject with many details in the future. In the meantime, I’ll try to draw your attention to simpler but still practical example such as an electrocardiogram (ECG) filtering. ECG acts at very low frequency i.e. 0.5 Hz to 100 Hz. The original signal is affected by various kinds of noises i.e. power line interference, electrode contact noise. IIR filters, especially elliptic ones, are considered to be effective tools to remove noise at low frequencies.

How to use Syto

From the very beginning, I was convinced that Syto has to ship an intuitive approach to filtering. In this context “intuitive” means “following a common sense and based on general, the well-know procedure”. Let me briefly touch it. Filtering consists of two sub-parts: filter design and filtering itself. Let come through the filter design workflow depicted below.

Choose an approximation
Find normalized analog filter with defined approximation
Set up the sampling rate if a digital filter is desired
Make appropriate frequency transformation
Get polynomials coefficients of a transfer function

At this point, we have all the parameters to start processing the data. The only step left is to choose filtering method — lfilt or filtfilt.

Expressed in Scala code the usage of Syto looks much simpler

Likely, the users will want to get more about the library’s usage. I encourage them to inspect the unit tests provided alongside the production source code. These tests were written not only to validate the correctness of the calculations which sometimes are quite comprehensive. They serve as expressive documentation as well.

Compare Scipy and Syto

Scipy is a great collection of mathematical algorithms. There is no sense to compare the whole toolbox with young Syto. My first priority was to provide filter design/implementation API. Therefore, the table below matches filter design functionality from scipy.signal only and Syto.

However, my plan is to keep bringing new algorithm to Syto to be mature DSP library. A full comparison between scipy.signal and Syto you can find in the project’s Github repository

Plan

Syto has just started its way and there are a lot of things/feature to implement. It’s worth mentioning that the existing road map for the library not abstract but consist of a list of reachable tasks:

Publish jar artifact to one of public Maven repository
Implement functionality for FIR filter family
Integrate Syto with Java to make it JVM library, not for Scala only
Implement multi-dimensional filter toolset
Implement multi-rating filters

In addition to the tasks enlisted above, it’s worth mentioning that while developing filter design methods I’ve implemented several mathematical algorithms that are necessary for some kinds of approximations. For example, the elliptic filter doesn’t have an analytical solution to finding its poles and zeros. It made me dive into the numerical methods to implement such weird things like elliptic integrals, Jacobi elliptic functions, etc. This set of functionality is used internally but I believe one day after refactoring it should be also the part of Syto’s public API.

Summary

I hope this brief overview has shared some ideas behind Syto. Scala is a great language and Syto’s intention is to bring Scala’s power to the scientistic world. The library waits for its users as well as for their feedbacks and new expectations. Please find the library on its Github repo.

Sources

[1] Manish Chauhan, Rajesh Mehra: ECG Noise Reduction Filter Using Elliptic & Kaiser Techniques
[2] How to deal with ECG noise: https://vheart.io/blog/2017/how-to-deal-with-ecg-noise
[3] Mahesh S. Chavan, Ra.Agarwala, M.D.Uplane: Digital Elliptic Filter Application For Noise Reduction In ECG Signal