The rise of remote development environments
Recently we have been implementing support for Gitpod, and Codespaces into our product, called Conveyor. We are adding this integration because they are the first implementation of remote IDEs that we would actually use. Conveyor assists in the delivery of data products from ideation to production. It allows customers to focus on solving their business problems instead of worrying about the infrastructure, support, and cross-cutting concerns (security, scaling,…), etc.
Why automate your development environment
We, software engineers, have been using containerized runtimes for more than 5 years in order to run our applications, and data processing pipelines,... We all enjoyed the benefits like:
- portability: create the image once and it run everywhere
- reproducible: the image definition (e.g. Dockerfile) describes all runtime and environment dependencies of your source code.
- lightweight: only ship what you need, no unnecessary drivers, libraries,…
It is strange that although we strongly believe in using containers for running our application, we rarely use them for creating our development environment.
Did any of you have a seamless development experience when joining a new project? Did setting up your local development environment take a couple of minutes or rather a couple of hours due to:
- You needed to install/manage the correct version of your programming language (e.g. java, python, go,…)
- You needed to install a bunch of additional tools that are required to test/build/run your project (maven, npm, SQL drivers, terraform,…)
- You needed to install the python virtual environment for your project or download the whole internet while pulling in maven dependencies.
Gitpod and Codespaces to the rescue
There have been multiple attempts to create remote development environments over the past years, but they never got a lot of traction because they were not user-friendly or fast enough. We think that Gitpod in particular but also Codespaces could be a game-changer as they provide a remote workstation that performs well on both aspects. This is the reason why we added support for them in Conveyor.
Getting started
You can start quickly with either service by using the correct URL:
- Gitpod: gitpod.io/#<repo_url>
- Codespaces: https://github.com/<repo_name>/codespaces
To use either one in Conveyor, click on the corresponding link to open your project code in a remote environment.
Codespaces only supports Github repositories whereas Gitpod can be used with Github, Bitbucket, and Gitlab.
For most real-world projects the default setup will not work out of the box as the default docker images used for remote workstations might not include all the tools that you need for your project. Both services use a similar mechanism to customize your remote workstation, in Gitpod you do this through:
- .gitpod.dockerfile: containing the base image that is used for your workspace. Default images exist for Go, Java, and Python,… and they come with some pre-installed tools like Homebrew, and Docker but can be completely customized afterward. A python project in Conveyor uses the following .gitpod.dockerfile:
- .gitpod.yml: contains tasks that you want to execute before starting your development environment. These tasks can be executed once when creating your workspace or every time you restart your workspace. The Dockerfile defines the image that is used for all workspaces created for the same repository. This means that compiling your code and downloading dependencies is best specified in the Gitpod.yaml as they can change with every commit. The python template of Conveyor has the following yaml file:
The Gitpod environment in your browser, which I am not a big fan of because of keybinding issues as well as the inability to use my custom IDE shortcuts. An easy solution is to connect the remote workstation to your local IDE, eliminating both drawbacks, and resulting in a user-friendly setup.
Additionally, I prefer Jetbrains over VScode, which is the default in Gitpod. Luckily, Jetbrains released Jetbrains Gateway (still in beta) with support for Gitpod. This way you can configure Gitpod to use Pycharm/Intellij/Goland when launching your workspace instead of VScode, which is exactly what I need.
Benefits
I am starting to use Gitpod more and more and I notice the following benefits:
- The project setup for every developer in the team is the same. No more struggles with supporting both macOS, Linux, & Windows environments.
- Fast network connectivity: pulling/pushing docker images goes a lot faster compared to running it from my home office.
- Fully customizable: as a developer, I can tweak the Gitpod dockerfile as well as the yaml file for every project.
- Quick startup: Setting up your workspace typically takes less than a minute.
- Gitpod is free for up to 50 hours a month and a personal subscription gets you 100 hours a month for only 8 euros. More info on their pricing page.
Drawbacks
Like any new technology there are also a couple of drawbacks, the ones I can think of at the moment:
- Learning curve: the first time creating the .gitpod.yaml file you will need to learn a couple of concepts (e.g. prebuilds, start tasks, browser settings, …) and how to use them.
- Developers can be quite picky about their development environment. This can be an issue when deciding on which tools to include in the gitpod workstation (e.g. bash vs zsh vs fish).
Conclusion
After experimenting with Gitpod for a couple of days, I am liking it more and more. I am going to start using it on projects that I work on sporadically (e.g. once a week/month) and that has a different setup from my day-to-day work. As mentioned, I use the Jetbrains gateway to connect PyCharm and Goland to my Gitpod workstation. If Gitpod continues to deliver on its promise, I might one day get rid of my beefy laptop and just use a remote IDE for all my development work.
Want to know more about Conveyor
Do you want to know more about Conveyor and make use of our Gitpod/Codespaces templates when creating your data projects, take a look at our website or our 📕 documentation.
If you want to try it out, use the following link. From there you can get started for free in your own AWS account with a few easy steps.
