Vis Hackathon 2019
The visualization engineering team at Uber is made up of dozens of technologists with very different specialties — mapping, web, data, low-level graphics, ML, you name it. Many of us fell in love with visualization by experimenting on our own and through the joy of making stuff. And with the team so large and so diverse we don’t always have a chance to work with each other!
Two excellent reasons to launch our second Vis Hackathon, in which 11 teams from across Uber Vis participated. We had two themes this time:
- Using the new speed datasets from movement.uber.com,
- Showcasing new features from our frameworks like deck.gl, loaders.gl, luma.gl, etc.
Without further ado, here’s what we did!
Urban Symphony
Jon Sadka
Close your eyes for a moment and imagine what it’s like sitting in a quickly accelerating car. Chances are that part of this imagination includes sound.
Sound is a natural byproduct of movement and speed, yet, by their nature, most visualizations about speed only express data visually. Urban Symphony is an experiment to see what happens when we used speed data to generate a melody for a given path of road segments.
Sparrow
Wesam Manassra, Ed Barwani, Adam Kidder, Ib Green
Best use of city data prize
Apache Arrow is a columnar in-memory data format that is interoperable across different languages, and makes it efficient to transport and scan large datasets without the need for serialization/deserialization on servers/clients.
We unlocked the powers of Arrow by building Sparrow, a tool to visualize a year’s worth of hourly speed data in the browser dynamically and efficiently (around 130 million speed readings). We built a lightweight Python server to read the dataset partitioned by day and stream it over HTTP. We can then visualize hourly speed data in the browser as soon as it arrives from the server. As the user scrubs to a certain date, data is loaded and flushed dynamically to maintain an upper bound on the total memory used by the browser.
This approach allows us to visualize very large data sets with low latency. As a future exploration, we’d like to stream this data directly to the GPU to enable even better performance.
Traffic Flow
Javid Hsueh
This project simplifies the road network and visualizes the traffic flow of the city. The junctions will move toward the major traffic direction at each hour.
The two main challenges of this project were the road network simplification and using transform feedback in a custom deck.gl layer to animate the flow on the edges.
ML Speeds Limit
David Schnurr, Ben Kramer
Basemaps like OSM often have inconsistent availability of speed limit data. Can we build a supervised machine learning model that utilizes Movement Speeds data to predict speed limits on roads where it’s not available?
Our team had limited ML experience, so a lot of time was spent learning about model selection (we settled on a decision tree regressor) and feature engineering.
The accuracy of the final model was actually relatively decent, and we think given extra time its accuracy could be improved even further with better feature engineering, hyper-parameter tuning, and testing deep learning approaches.
SpeedsUp
Bryant Luong, Lezhi Li
In this project, we tried to discover daily speed patterns of various streets in a city using machine learning on Movement Speeds data. On the right, we represent streets as time series of speed over the course of the day. We can then cluster these time series by similarity — the streets which are always fast, always slow, fast and slow at similar times, etc.
We used Apache Arrow to load a very large dataset in the browser and TensorFlow.js for efficient in-browser machine-learning.
Really Fast Rectangles
Brian Ford
Can aggregation and drawing functions based on the incremental lambda calculus be a practical way to speed up data visualization programs? To find out, we wrote one data visualization using d3.js and another one using incremental functions, and then benchmarked the two approaches.
While our intuition is correct and that lambda calculus can yield drastic performance improvement, we also found that writing aggregations in terms of changes is surprisingly difficult, and that not all types of aggregation benefit from this approach.
How to Animate Anything
Shan He
Grand prize
In this project, we wanted to experiment with deck.gl’s attributes transition in kepler.gl to visualize speed changes over time.
Animation is, at its core, the transition of visual attributes (position, color, radius, thickness) based on time. In kepler.gl, layer visual attributes can be encoded by selected field values. However, there is currently no way to animate these attributes based on time: e.g., you can draw road segments and color them by speed, but you can’t animate the colors based on speed changes over time.
The way we allow the animation of visual attributes (position, color, radius, etc) is based on an additional time table. The user can select a dataset to draw the shape (road segments), and another dataset that contains a time table (speed per segments per hour). They can then join these two data tables and enable animation for visual attributes that are based on value (speed, # of cars, # of passengers, etc).
Cinematic kepler.gl
Chris Gervang, Chun Jiang, Alison Lee
This project provides the ability to sequence animations and generate videos in kepler.gl using an easy magic keyframe UI. It integrates a WIP video library, hubble.gl, into kepler.gl delivering instant videos, right to your downloads! It demonstrates the new luma.gl Timeline and KeyFrames classes.
We were challenged with striking a balance between simplicity and functionality in the UI design, by writing new UI components into kepler.gl, and by figuring out how to integrate the necessary parts of luma.gl, deck.gl, kepler.gl, and hubble.gl together.
Citymorph
Xiaoji Chen, Jerome Cukier
Best use of frameworks prize
From the Movement Speeds dataset, we can deduce how much time it takes, on average, to traverse street segments at various times of the day. We can use that data to morph the maps of cities and from one given point, reposition all street intersections according to how much time it would take to go there.
One interesting aspect of this project is that we did the required graph traversal directly in the GPU.
(The interactive version of this project will be coming shortly to the deck.gl showcase.)
Hex Hedgehogs
Nick Rabinowitz, Ib Green
The Hex Flow project was intended to prototype visualization approaches for city-level trip flow patterns, using the H3 library and deck.gl. Using several weeks of anonymized routes from Jump Bike data, we aggregated flow patterns by hour of week for every H3 cell in the city, then built an interface that could display flow as an animated vector field.
The primary challenge was to aggregate tens of thousands of routes into a format that was both performant and visually comprehensible. Our final version included both offline and runtime aggregation, including multi-pass smoothing, to generate an H3-based vector field for each time slice and interpolate between them.
Validating the data processing and visualization would have been very difficult without knowledge of the city in question. We knew we were on the right track when we could see the clear bike commute patterns up and down Market Street at appropriate times!
Map Collage
Yang Wang, Jian Huang, Ravi Akkenapally
Map Collage is like photo collage, but (wait for it) for maps.
This consolidates multiple sub-maps into one canvas with dynamic boundaries generated by the underlying data. It frees you from constant zooming in and out when examining multiple places-of-interest with limited screen space (e.g. on mobile devices or in a UI widget).
The hardest part of the project was to interactively and dynamically generate the boundaries of each sub-map.
