Increasing Rust’s Reach: Porting a Python application to Rust

I participated in the Increasing Rust’s Reach program from August to November 2017. My project was #7 Finding Missing Pieces in the Crates Ecosystem with Andrew Gallant. I quickly learned that Andrew, aka @burntsushi, is well known in the Rust community for developing ripgrep, a command line search tool that is faster than the Silver Searcher and grep. Just a web search on ripgrep will show you how often it is mentioned by others. Visual Studio search is powered by ripgrep. I was in awe that I was selected to work with Andrew, AND he is a really nice and knowledgeable guy to work with.

The plan for my project was to take a small application in Python and port it to Rust. Andrew is a member of the Rust Libraries team that oversees the Rust standard library, rust-lang crates, conventions, and ecosystem support. The goal is to see how my project could reveal gaps and missing functionality in the Rust library and crates that a new user may encounter. The Library team could then consider how to make the library more usable.

At my previous job at Lawrence Berkeley National Laboratory (a U.S. Dept of Energy lab), I built systems to monitor energy use in buildings and urban systems for research to inform energy policy. I was primarily using Python and open source libraries for my projects. I became interested in learning a modern systems programming language, such as Go or Rust, to see how it could improve performance in our systems.

I decided on writing a Rust application for the Raspberry Pi. I also have the Sense HAT add-on board with LED matrix. I focused on writing out to the LED matrix, for example, this Python example to display a dynamic rainbow pattern.

It essentially comes down to translating the set_pixels() function and supporting function _get_fb_device().

It requires representing the pixel map with the correct type representation, reading a file, composing the path for another file, converting the pixel map to the binary representation, and writing to that file.

This seemingly basic and simple application took me down the journey to learn and discover:

  • Types
  • Options and Results
  • ? operator
  • Error handling
  • File handling
  • Path and PathBuf types
  • Parsing String and &str
  • Bit manipulation and 16-bit RGB565 representation
  • vec vs arrays, and the clone trait. Objects allocated on the heap vs stack.
  • You can import some external crates to (example) List of supported crates
  • “I don’t understand the compiler error. Let me try adding some muts and &s and see if it compiles.”
  • “Yay, it compiles! I have no idea why that worked! I’m not even sure if it’s actually still correct for the functionality I was working towards or that I just shut up the compiler”
  • Debugging Rust with breakpoints, not officially supported. However, you can use gdb/lldb. One option is use gdb/lldb with Visual Studio Code.
  • Many times, I was frustrated that I knew exactly how to express what I wanted to code in Python, but was clueless as to how to write it in Rust. Thus, learning a new language :)
  • Many times, it really just came down to Andrew telling me exactly what the code snippet should look like. And then of course it worked. :P

Through this project, I learned much of the Rust basics, but made slow progress towards accomplishing the goal I defined at the beginning of the Rust Reach program. It’s ok, as Rust is known to have a steep learning curve, and now I can write Rust code a bit more fluidly without getting stumped every few characters. I plan to continue to stumble along even after the program ends. I am almost to the point of writing out the file in the RPi, but still working out file write error.

In the following weeks, I plan to publish blog posts that discuss in detail the lessons I have learned during this project. For example, I plan to write a post on String vs &str and vec vs array, specifically in the context of my project. I also finally learned what turbofish is at RustBridge, so I may dedicate a post to explain that also. Note that I’m still learning Rust, so my explanation may be flawed and I welcome constructive criticism to improve my understanding of these topics.

Big thanks to: Andrew Gallant for volunteering as my mentor for this program; Carol (Nichols || Goulding) for coordinating the Increasing Rust’s Reach program and organizing the Rust Belt Rust conference; and the many other awesome members of the Rust Team and community for encouragement and advice during my learning process.

I will be giving a talk on this project at PyCascades (Jan 2018) in Vancouver, B.C., Canada. I’m hoping to have made some more progress on it by then!