So you think you know Python?
Well you probably do, it’s not that hard.
Rather than test your knowledge of the language, I’ll suggest seven different things you could do or learn next that will make you a better, more capable, Python developer. After all, it’s not the language itself that’s powerful, but how it’s used!
1. Master Unix and Learn (Some More) Bash
Hopefully you already love Bash more than Python and use it every day, but for many devs who started out with Python, it may never have been a major part of everyday operations. If Bash seems scary or clunky to you, I strongly recommend experimenting with some core commands and understanding loops, piping, and links.
The command line is the universal standard for getting programs to talk to each other, and if you can’t get your Python to talk to other programs, you’ll repeatedly hit barriers.
Bash has helped me get around awkward situations like not being able to use pip (in a serverless code execution environment), and not having admin rights on Windows (Git Bash was installed, thankfully).
You can always rely on Bash. With a strong grasp of it you can work around any problem. Not to mention you can run Bash commands with Python and vice versa.
2. Learn a tiny amount of HTML/JS/CSS
There are some things Python should not be used for, and Front End development is one of those things.
I’d also say the same about data visualisation. My favourite plotting libraries for Python (Plotly and Bokeh) render charts within a browser, using JavaScript. These achieve far better results than Matplotlib in terms of appearance, ease of use, and interactivity.
JavaScript is even eating into Java and C#’s market share. Electron allows you to build desktop applications with just HTML and JS (building on top of Chromium). It might surprise you to learn that MS Teams is an Electron app.
Don’t pigeonhole yourself as a purely back-end dev! Learn the basics of a framework like React and start showing off how great your work is with some stunning user interfaces. Plus, putting some thought into the user experience of your app will help streamline your back-end code.
Best of all, you get to call yourself a Full Stack developer.
3. Use Docker to industrialise your code
“Well it works on my machine” -Bad Python developer
Hopefully this one seems obvious. A Python developer running everything on their laptop, or on a VM with a lengthy setup script, is not a good developer.
Code that provides value should be able to run on its own, without being babysat, and should be able to be deployed and redeployed by anyone, otherwise it will end up being thrown away as soon as the person who wrote it isn’t around.
The first step to industrialising code is to move the code and all dependencies into a container. Docker makes this fairly straightforward with Dockerfiles, and allows anyone to run the container on any machine, provided they have Docker installed.
You can then make use of serverless tools like Google’s Cloud Run or AWS Lambda, which will run your container on request without the need to provision compute resources. This is the easiest way to turn your code into an API, and allow anyone to use without having to install anything. From here you can plug it into a web front-end and reach a much larger audience.
I’ve seen many git repos from universities and hobbyists alike that contain some amazing code but have absolutely no thought put into industrialisation. This is a great shame since it prevents most people from being able to play with (and therefore potentially contribute to) the code. If they had only provided a requirements.txt and Dockerfile, someone might have taken it and created something truly mind-blowing.
Disclaimer: I’m a huge fan of Cloud Functions/Lambdas which don’t require any Docker knowledge, but some tasks require containers.
4. Learn PostgreSQL
Everybody knows that Python is great for transforming data but that doesn’t mean it’s a great choice for an application backend or data pipeline. In fact, I would strongly discourage the use of Python (or pandas specifically) for any scenario in which a database could also be used. Database technologies like Postgres and MySQL have been designed to deal with data, and they do it really well.
Next time you think about joining two datasets with pandas, consider spinning up a PostgreSQL instance, importing your data into tables, and running a join on those tables (scripted by Python, using psycopg2). The setup may take longer, but when you find yourself needing to repeat the join again with updated data, you will see the benefits in terms of speed and reliability. You can then easily parameterise your SQL with Python formatting or Jinja2, massively reducing the amount of code needed to carry out all of your transformations.
SQL is dead easy to learn and Postgres is a great starting place. Once your data outgrows Postgres, you can apply the same knowledge to other, more powerful tools like Google BigQuery or Snowflake.
5. Don’t repeat yourself… ever again in your entire career
Have you ever copied code from one project to another? Perhaps you do this at the start of every project.
Save others from repeating your work! Make packages.
The first step is to collect together all the useful functions you use regularly. It might be best to group them up and make lots of separate packages to keep requirements simple.
Secondly make a setup.py so that you can pip install your package locally.
Thirdly, and most importantly, put your package into a git repo and share it with your team*. Python packages can be pip installed directly from git repos like so:
pip install "git+ssh://git@github.com/<username>/<repo>.git"Once you have a whole company using the same code across multiple projects, you start to see huge time savings both when starting new projects and when upgrading/fixing bugs in commonly used tools. I dread to think how long it would take us at DataSparQ to patch each of our projects separately if we weren’t using the same packages across all of them.
*Or, if you’re happy for it to be public, upload it to PyPI. That way you can save everyone from repeating your work!
6. Become a Git Expert
I’ve met developers who use Git every day and don’t actually understand it. As soon as something unexpected happens they don’t know what to do, and don’t know how to avoid problems in the first place.
Every team needs a Git expert. Someone who can confidently handle tricky merges, rebases, cherry picks, etc., and keep repos tidy.
If your releases are difficult and technical debt is piling up, the problem might be here. Make sure you understand branching strategies like Gitflow, and how to use Git + Agile + good documentation in the most complementary and efficient way.
7. Learn C to remove bottlenecks
I have come across problems that Python was too slow for. The solution was C, wrapped in Python (so you could say Python wasn’t too slow).
Ok you don’t need to learn C. I would suggest Go instead as it has a lot of support from cloud providers and is much easier to compile for different architectures, which was always the biggest pain with C for me.
Some of the most powerful packages like numpy, pillow, and TensorFlow are all just C modules wrapped in Python. That level of performance can also be achieved with very little effort these days. One could write a simple program in Go and call it from Python as part of a larger script, for example:
import subprocessdef run(cmd):
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, text=True)
return list(iter(p.stdout.readline, ""))
x = run("count_emojis 🚀 --file wikipedia.zip")
If anything, writing a hello_world program in C is worthwhile purely to appreciate how easy Python is.
Disclaimer 1: I only suggest this method for simplicity. It doesn’t benefit from shared memory but is much less painful to write, and less likely to make Python crash.
Disclaimer 2: Python is definitely powerful enough to count emojis.
Conclusion
There are 0 job roles that require Python skills and nothing else. Branching out and learning complementary skills will make a huge difference to how effective your code can be.
