Managing Python Project with Conda
This post describes an easy to use but effective workflow for building a python project from empty to be deploy-ready.
First, create a project folder and enter it.
Next, create an environment to manage the packages used by your project. Instead of using the mix of pip and virtualenv, we will use conda. conda combines package management and virtual environment in the same tool. I found it very intuitive to use.
# create env named my_proj
conda create -n my_proj
# activate env
source activate my_proj
# install pip
conda install pip
Now, do the real work, writing your code. You can use
pip install or
conda install to install your packages.
Remember to test your project before deployment.
We will use a docker container as our deployment option so you can easily run it almost anyway and rest assure the same behavior of your code.
The process is quite straightforward. Since we already have our environment nicely sandboxed, it is easy to export it. You can always use
conda env export > env.yaml to save the env configuration to a file. In the Dockerfile, simply recreate the
conda env create -f env.yaml and activate it. However, if you choose the pip route to install packages, just do
pip freeze > requirements.txt and
pip install -r requirements.txt to recover. Deploy using conda or pip takes almost the same effort (little). So pick whichever one you want.
A sample Dockerfile using conda is shown below:
# set up work directory.
# copy necessary files.
COPY ./*.* ./
# install packages.
RUN conda env create -f env.yaml
# run script.
CMD python my_script.py
Build the image using
docker build -t some_tag -f path/to/dockerfile [context directory] .
Run your docker container on the cloud platform you like, all major vendors have container service in place.