Escape Python Dependency Hell with Just 2 Commands

3 min readNov 19, 2024

Introduction

If you’ve ever worked on a Python project, chances are you’ve encountered dependency hell — that frustrating situation where package versions clash, breaking your code and wasting hours of debugging. But don’t worry! With the right tools, you can escape this chaos.

In this post, I’ll show you how to use pip-tools — specifically pip-compile and pip-sync — to take control of your Python packages.

pip freeze > requirements.txt:

Saves all installed packages and their versions from your current environment but includes everything, even indirect dependencies.

pip-compile:

Starts with your direct dependencies, resolves versions for all required packages, and creates a clean, conflict-free requirements.txt

Advantages of Using pip-tools

Automatically resolves and locks package versions
Keeps all environments in sync with pip-sync
Makes upgrades safer and easier to manage
Prevents version conflicts and unexpected bugs

Instead of this:

annotated-types==0.7.0
anyio==4.6.2.post1
build==1.2.2.post1
click==8.1.7
fastapi==0.115.5
idna==3.10
packaging==24.2
pip-tools==7.4.1
pydantic==2.9.2
pydantic_core==2.23.4
pyproject_hooks==1.2.0
setuptools==75.5.0
sniffio==1.3.1
SQLAlchemy==2.0.36
starlette==0.41.3
typing_extensions==4.12.2
wheel==0.45.0

Keep the list of packages like this:

FastAPI
SQLAlchemy

Let’s get dive into it

I kept using pip install for every package and then pip freeze > requirements.txt to save them, but it always led to a chaotic, unreliable file — sound familiar?

Setup

For the reference, let’s say I have a simple FastAPI app working with PostgreSQL database. So I need to install FastAPI and SQLAlchemy.

You’d probably wanted to create a virtual environment first to keep all the packages separated for a specific project but I’m not going to cover it in this post.

1. Create requirements.in file

Run this command to create an empty requirements.in file

$ cat requirements.in

Then open it in your favourite text editor (mine is vim) and add these 2 lines:

FastAPI
SQLAlchemy

I don’t need any specific version of FastAPI nor the SQLAlchemy package so I keep it simple.

2. Install pip-tools

Run this command to install pip-tools

$ pip install pip-tools

3. Create requirements.txt file

Now when you have installed pip-tools you can run pip-compile command which generates the the list of all the packages and it’s versions you need to install.

$ pip-compile requirements.in

When you run this command, it saves the output of the command into requirements.txt file whihc might look like this:

#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile requirements.in
#
annotated-types==0.7.0
    # via pydantic
anyio==4.6.2.post1
    # via starlette
fastapi==0.115.5
    # via -r requirements.in
idna==3.10
    # via anyio
pydantic==2.9.2
    # via fastapi
pydantic-core==2.23.4
    # via pydantic
sniffio==1.3.1
    # via anyio
sqlalchemy==2.0.36
    # via -r requirements.in
starlette==0.41.3
    # via fastapi
typing-extensions==4.12.2
    # via
    #   fastapi
    #   pydantic
    #   pydantic-core
    #   sqlalchemy

This requirements.txt, created with pip-compile, keeps your dependencies tidy, reliable, and easy to understand — no more messy surprises!

4. Install packages with pip-sync

The very last command of the whole setup is pip-sync which syncs the packages in your environment with the ones provided in the requirements.txt file.

$ pip-sync requirements.txt

Use pip-sync for managing dependencies in non-clean environments, but skip it in clean setups like new Docker builds in CI/CD pipelines, where pip install -r requirements.txt is enough.

To double check the correct versions you can run the command again and you should get the outbue like:

$ Everything up-to-date

You can also check it by running command pip-freeze and compare result with the requirements.txt file.

$ pip freeze
annotated-types==0.7.0
anyio==4.6.2.post1
build==1.2.2.post1
click==8.1.7
fastapi==0.115.5
idna==3.10
packaging==24.2
pip-tools==7.4.1
pydantic==2.9.2
pydantic_core==2.23.4
pyproject_hooks==1.2.0
setuptools==75.5.0
sniffio==1.3.1
SQLAlchemy==2.0.36
starlette==0.41.3
typing_extensions==4.12.2
wheel==0.45.0

Final words

Managing Python dependencies can feel overwhelming, but tools like pip-compile and pip-sync make it easier to maintain clean, reliable, and conflict-free environments.

Whether you’re working on a local setup or deploying in CI/CD pipelines, these tools are a must-have in your Python toolbox

🚀 Let’s keep learning and growing together — happy coding! 🚀

If you’ve found this guide helpful, I’d love to hear your thoughts! Feel free to connect with me for more tips and discussions:

Happy coding 🚀