Transpiling Python to Rust
We’ve been working on developing a consensus protocol for some time now. And because we were not sure which ideas would work best, how architecture will lay out and also because project was started on a hackathon, we decided to go with Python for prototyping. The plan was to port everything into Rust later, because blockchain would benefit from that sweet safety and reliability. After a few months our project has grown to 10k lines of code and still growing. So, it seemed to me, that porting will be a tedious task and I started to look for some way to convert, at least some parts of our code to Rust. Sure it won’t be idiomatic and safe, but better than nothing, right?
So, I started looking for some transpilers and didn’t find anything. This is why I decided to spend one or two evenings writing my own fuzzy syntax converter. Because Rust must have been influenced by Python: tuples, conditions, loops, ranges and general approach looks a bit similar too. Wikipedia doesn’t state Python as direct influence, but it should have been a common ancestor of some sort. Nevertheless I’ve tried to search for an easy path for transpiling and found rather hardcore tools like ROSE and DMS which are aiming for correctness not for some fuzzy code conversion. Writing a proper tool would take time and I’m not sure if it is possible at all, since Rust is super strict and Python, on the contrary, is quite relaxed.
So in my search I’ve stumbled upon py14 — Python project by Lukas Martinelli. It aims to show power of C++14 by transpiling Python to C++ full of template magic and auto declarations. It looked nice, so I took this project and quickly made it convert some procedural-only statements and expressions into proper Rust. Given its somewhat C-Like and somewhat Python-like syntax basic transpiling was a breeze. Adding more advanced features like classes and methods took me some time though. So after some grinding and experiments I’m ready to show you some examples.
What compiles
Consider simple Python code:
Using pyrs it transpiles to:
As you can see it uses vec! macro for creating array in place, substitutes print with println!for two arguments. And using std::iterator approach it transforms lambda to Rust closure, puts it into map and in the end list becomes collect::<Vec<_>>(). It is always a pleasure to use iterator interface, isn’t it?
Statements like this and also some simpler expressions are working properly: ifs, loops, assignments, calls, lambdas, constants, comparisons and even imports. But the problems arise when we move to classes and methods.
What does not compile
Python classes are tricky in different ways. They don’t have specific place for member definition. This is especially painful for language like Rust — the language without classes per se. And even worse problems happen to functions and methods. Because we don’t know what types they handle and Rust tries to “strike delicate balance” forcing you to be explicit about arguments and return types. So classes like this:
Transpile to this:
You can see that types were unable to be inferred so transpiler simply replaced them with typenames. The same thing happened to return type. And also struct fields were empirically taken from __init__ method. Though due to interpreted nature of Python new class fields may appear anywhere in code and it is not easily detectable, at least not yet. But on a good note, you can put type hints into your Python code and transpiler will bring them along to Rust. Oh, and look how Rusty these hint declarations look!
def cut(apple: Fruit, knife: Tool) -> Slice:There is also a bunch of totally unsupported stuff like try-except-raise and with keyword which are marked as “unsupported” in resulting code. Unstable stuff like yield, async-await. Some unclear things like del keyword, arguments unpacking, elipsis. And lot of not yet discovered Python intricacies.
Conclusion
Was it worth it? I’m not sure yet. Our project code is definitely having Rust syntax now, but it doesn’t compile and looks quite Pythonic. It doesn’t use references, treats everything as mutable and won’t pass through borrow checker for sure. But I feel that maybe transpiling from Python is an under explored area. Python’s internal module to manipulate Abstract Syntax Tree is nice and easy to use. With some effort it can be fit for some powerful analysis mechanisms to yield more correct and idiomatic code. If someone could make production-ready Python to JS transpiler than with power of rustc and clippy we can achieve something similar. But don’t take my word for it, grab pyrs from repository and try it for yourself! Go on and check its limits or port some Python code that you think be better off in Rust.
