KillrVideo Python Pt. 1— the Backstory
Application development with the DataStax Python Driver
After a busy fall of travel doing advocacy at Strata NYC and several DataStax Developer Day events, I had the privilege of a month with no travel. I figured this was the perfect time to take on a development project I’d been thinking about for a while: creating an implementation of the KillrVideo reference application in Python — specifically, the microservices tier.
If you’re not familiar, KillrVideo is a reference application for learning Apache Cassandra and DataStax Enterprise created by Luke Tillman, Jon Haddad, and others. You can find the killrvideo-python repository on GitHub.
First of all, I thought I’d articulate some of my reasons for kicking off this project. Here they are quite simply:
- I’ve never done development on a deep level in Python — growth opportunity!
- Python is currently #3 on the TIOBE Index… and rising. The Python drivers are also among the top downloaded Cassandra / DSE Drivers. However, there aren’t nearly as many examples using these drivers as I would hope.
Python seems to have somewhat of a mixed reputation in the computing industry. On the one hand it’s growing greatly in popularity, especially in data science and data engineering fields, I’m assuming because of it’s flexibility and ease of use. For many of us in the rapidly growing group of people who call ourselves software developers, it’s the one of the first languages we encounter.
type() calls to navigate type mismatches as I’ve worked on this project. My other big learning is that you should prototype things at the command prompt as much as possible.
Application Development Approach
I’ll freely admit that there’s one factor that greatly simplifies this application development effort: the availability of the service tier implementations in other languages such as Java, Node.js and C#. For this effort I’ve been leaning heavily on the great work done previously on the Java implementation by DuyHai Doan and David Gilardi.
So my development process has been to implement one service at a time, leveraging the same algorithms and queries used by the Java implementation. As I started working and reading the Java code, I figured that it would be really useful to create a functional specifications for the services to make things easier for the next person to create an implementation in a new language. I’m using this to try to bait Adron Hall into doing a golang implementation ;).
By the way, I’m working on the services one at a time, in the order implied by the KillrVideo integration tests. Since most actions in the KillrVideo application are associated with a user, the first service to implement is the User Management Service, followed by the Video Catalog Service.
More to come
Here are some of the aspects that I plan to cover in this series, without committing to any particular order:
- Using the DataStax Python Driver for Cassandra, Search, and Graph queries, including pros and cons of the cqlengine mapper
- Using GRPC in Python
- Service discovery and Etcd
- Leveraging Python libraries to get things done quickly — email address validation, stop word removal, etc.
- The KillrVideo integration test approach
- Implementing KillrVideo events via Kafka
- KillrVideo Docker deployment and the move to Kubernetes that Aleksandr Volochnev is working on.
Thanks for reading. I’m excited that Amanda Moran has agreed to collaborate with me on this project, and would love to have contributions from anyone that’s interested in helping out!