Managing your requirements.txt with pip-tools in python
pip-tools is a package that allows us to separate direct dependencies from their sub-dependencies. pip-tools generates a dependency graph and uses this information to create a bespoke requirements.txt file for our project.
We use dependency management tools to create a list of modules that our application requires. In Python, the standard convention for tracking dependencies is to list them in a requirements.txt.
Pip-tools becomes handy when we have to include different requirements in different environment i.e production, staging. Applications have dependencies for running as well as dependencies for development. I like to have these in separate requirements.in and requirements-dev.in. This keeps the compiled dependencies separate from each other.
Generally, pip-tools = pip-compile + pip-sync. The pip-compile command lets you compile a requirements.txt file from your dependencies, specified in either setup.py or requirements.in.The pip-sync command will update your virtual environment to reflect exactly what's in requirements.txt. This will install/upgrade/uninstall everything necessary to match the requirements.txt contents.
Using pip-tools for managing requirements.txt file
- Installation
pip-tools must be installed in each of your project's virtual environments:
(venv)home: pip install pip-tools2. Create requirements.in file for production and dev-requirements.in file for development
touch requirements.in dev-requirements.in3. List down the dependencies needed for production in requirements.in and development in dev-requirements.in
# requirements.in
django
django-rest-frameworkWe list down only the dependencies required in development besides production in the dev-requirements.in . Here the -r requirements.txtin dev-requirements.in will list all the dependencies mentioned in the requirements.in file.
# dev-requirements.in
-r requirements.txt
django-debug-toolbar
pytest4. We compile the requirements.in file with the help of a command pip-compile
>> pip-compile## This command will generate a requirments.txt file with all the dependencies written on requirements.in file
## If we dont pass any arguments with the command the default value will be requirements.in file## For development
>> pip-compile dev-requirements.in## This command will generate a dev-requirements.txt file with all the dependencies written on dev-requirements.in file
5. Finally we install the package through pip-sync
>> pip-sync## This command will install all the package listed in the requirments.txt file.
## If we dont pass any arguments with the command the default value will be requirements.txt file>> pip-sync dev-requirements.txt#### This command will install all the package listed in the dev-requirements.txt file.
pip-sync will update your virtual environment to reflect exactly what's in there. This will install/upgrade/uninstall everything necessary to match the requirements.txt contents.
In these ways, you can manage your requirements.txt file effectively in python.
References:
