pyvlog: a Python package for smarter intersections
At HAL24K we use a wide range of Python packages for working with the various data formats we come across when building our solutions. However sometimes we encounter a data format that no package exists for, and so we have to build our own. As part of our open sourcing program we’ve decided to share some of these packages back with the wider community, beginning with our Python package for processing intersection data, pyvlog.
What is V-Log data?
V-Log is a messaging protocol for receiving data from connected devices at traffic intersections, whether traffic lights, loop sensors or emergency vehicle alerts. Developed by Vialis, it is the standard data format for intersections in the Netherlands and increasingly across Europe.
V-Log data comprises a sequence of hexadecimal strings, each corresponding to an update (or set of updates) for a particular type of traffic device. This makes it a very efficient format for streaming and storing intersection data, which it needs to be, given that updates are typically sent ten times a second.
However what V-Log gains in efficiency it loses in readability: it communicates only the changes at the intersection at each timestamp, so to discover the current status of a certain device you have to read back through the sequence of messages until the last time a change was communicated.
What does pyvlog do?
pyvlog converts a sequence of V-Log messages into a set of time-stamped statuses for all selected traffic devices at an intersection. This allows for the easy querying of traffic device data by time and device, which is invaluable when developing solutions based upon it.
Parser objects are provided for handling real-time streams of V-Log messages and calculating the current intersection status:
As well as converter functions for transforming historic V-Log messages into their corresponding intersection statuses:
Furthermore, the base parser class can be easily extended with different logging functions, allowing for the creation of custom parser classes for all manner of different applications.
pyvlog can be installed via pip:
pip install pyvlogWhat are we using it for?
pyvlog lies at the heart of HAL24K’s queue predictions, which are used to dynamically control traffic lights at intersections across Provincie Noord Holland. It allows us to process streams of V-Log messages in real-time, so that our deep learning models can query the intersection’s status and estimate the number of queueing vehicles in each lane, all in under a second.
What’s next?
HAL24K plans to open-source more internal tooling and data science libraries so that better solutions can be built that will benefit us all. As well as pyvlog, earlier this year we opened up OOProxy, one of the components we use for authenticating our machine learning APIs. We also support the community by hosting Jupyter Amsterdam meetups, which you can find out more about here.
About HAL24K
HAL24K is a Data Intelligence scale-up based in San Francisco, Amsterdam and Singapore, delivering operational and predictive intelligence to cities, countries and companies.
