Engineering a Hackathon with Looker
A breakdown of Hack:London hosted by Looker!
Hackathons Are Important
At Looker, we recognize the importance of hackathons. Before hosting our first ever external hackathon in 2018, we’d have internal hackathons (we would call them hackdays) every couple of months to let our engineers dream big and build their dream project. A number of important features in Looker came out of the minimum viable products built during our Looker hackdays.
Hackathons are important for our broader community too. A developer attending a Looker hackathon can use the time and resources provided to quickly build and iterate on projects that extend Looker. Attendees get to interact directly with both Looker employees and other Looker users who are experts in various aspects of Looker’s platform. Looker engineers, in turn, get to collaborate with customers and gain a better understanding of how customers use and want to use Looker.
We also like hosting hackathons because they’re fun, and we really enjoy seeing the impressive solutions attendees build in such a short time.
Our hope is that hackathon attendees get more enjoyment and benefit out of using Looker and that everyone learns more about the Looker API, the powerful integrations, and the numerous customizations that can be built on Looker.
Improving Hackathon Processes With Looker
As we learn from each Hackathon, we make iterative improvements. Since last May was only our second external Hackathon, we still have lots of room for process improvement. Our focus for the May event was to automate the registration and on-boarding experience for the Hackathon, all using Looker.
Looker Hackathons are ultimately enabled by the Looker Data Platform. The Looker Data Platform enables our customers to extend Looker and even build entirely new experiences on top of a unified and governed data model using our powerful APIs and SDKs.
Below are some enhancements to our Hackathon process that we built on top of Looker.
Clustered Looker Hackathon Instances
In the event attendees wanted to feel safe experimenting and playing with ideas beyond using their own instances, we set up multiple Looker instances (all configured identically) so projects could be developed against a “safe” Looker instance. To do this, we spun up multiple server instances based on what we created for last year’s Hackathon in San Francisco, updated with the latest version of Looker (6.12 at the time).
Pre-loaded Datasets
We made some fun and interesting datasets available to use on the playground instances:
- Game of Thrones, a dataset with facts about actor screen time, scene details, and character deaths
- Looker System Activity, an internal database and corresponding model that surfaces metadata about Looker instance usage
- Reddit Comments, a dataset containing 1.7 billion Reddit comments from 2005–2019
Automated Welcome Email
During preparations for our first Hackathon, we used the Looker admin UI on each Looker instance, which sent the attendee an email for each server containing a login link for that instance. This required a Looker admin to paste the list of emails for attendees into the “Add user” dialog and select a checkbox to send the email.
We wanted to automate this a bit more for our May registration process. The day before the London Hackathon, we created a Python script using the Looker API and the SDK Code generator that:
- Reads user emails from a CSV file
- Checks to see if the user is registered on each server
- If the user is missing, the script:
- Creates a new user
- Gives the user appropriate development roles
- Requests API keys for the user
- Records their login URLs to a file, to share with each user
If you’d like to do something similar, you can see the script at https://github.com/josephaxisa/hack_reg/blob/master/hack_reg.py.
Curating the Welcome Experience
Another big focus for this hackathon was ensuring developers had a great Looker experience right from the start. With this in mind, we built a custom homepage for all of our Hackathon instances in Looker. This homepage — apart from looking super slick — helped address three critical questions that all developers have at Hackathons:
- What datasets can I use?
- What have other folks already built?
- What pre-existing resources can I build on?
Our curated experience was able to answer these questions immediately so developers could get into the code faster.
Judging the Hackathon
To facilitate the judging of hacks and the presentation of awards, we created a Google form for the judges to fill out for each project. This form fed into a Google sheet, which then was analyzed by a Looker dashboard to show off the Best Hacks in real-time. Read about the Hackathon winners from the London Hack here.
Pre-Hacks: Built on Looker
In addition to the more custom-functionality that we built on top of Looker to power our hackathon, we promoted some cool, open-source developer tools that have been built on top of Looker to make developer experiences easier.
SDK Codegen
Looker provides some pre-packaged client language SDKs for specific languages, including R and Ruby. For language/platform SDKs not currently provided by Looker, the new SDK Codegen project provides an easy way to generate an SDK in a developer’s desired language. During the London Hackathon, a few different teams used this SDK codegen to generate an SDK for their preferred development language.
This open-source project also supports a command to serve up a local interactive UI to quickly search the API specification for any text and immediately find the functions containing the search value.
Looker’s Embed SDK
One of the hack projects in London leveraged the newly available Embed SDK, which makes it easier to embed Looker in your own web app. This SDK simplifies the creation of, and communication with, Looker visual elements embedded in another web site. The open-source repository also includes some demonstration code.
Looker Visualization Builder
Out of the box, Looker ships with a number of configurable visualizations to bring your data to life. While those visualizations cover a number of common use cases, they don’t cover everything. To that end, customers often build their own custom visualizations using the custom visualization API that can seamlessly integrate with your explores and dashboards.
To make it even easier for our Hackathoners, we built a Codepen-like playground to facilitate the creation of new custom visualizations. The playground provided sample data, hooks, and a more real-world environment for our customers’ visualizations to interact with during development.
London Hacks Looker
Armed with twelve hours, plenty of food and drink, and an additional appetite to build something novel on top of Looker, London was ready to hack!
Break it down
The hackathon wasn’t all fun and games — we also took some time to recharge. For our afternoon break, Scott led willing participants in a stretching and handstand session to reawaken their bodies. While many participated in the stretching portion, only Scott did a handstand at the end.
We now have a stretch goal for the next Hackathon — get at least one more person to safely attempt a handstand!
Quick Hack — Star Ratings
One of the sample datasets available on the Hackathon instances is Amazon Product Reviews. In the screenshot below, a quick hack by one of the London Hackathoners shows a Star Rating Vis column.
measure: average_star_rating_vis {
type: number
sql: ${average_star_rating} ;;
html:
<span style="text-align: left">
{% for i in (1..value) %}
⭐️
{% endfor %}
</span>
;;
}What’s next?
We’ve had a blast hosting and helping out at our first two Hackathons. The projects completed have both impressed and inspired us, and have encouraged us to host many more of these events in the future.
Our next scheduled Hackathon is on November 4th, 2019 in San Francisco, coinciding with JOIN 2019. We look forward to sharing more updates about this and future Looker hackathons and hope to see you in attendance at one in the near future!
Want to have a Hackathon near you? Tell us!
Curious to learn more about our Looker team? Check out our open positions!
