How to manage Python projects with Pipenv

Python’s package ecosystem lets you leverage the work of millions of other Python developers with a simple pip install command. And Python’s virtual environments let you isolate projects and their packages from one another.

But juggling environments and packages separately can be unwieldy. Doubly so if your projects have specific package requirements, and you want to focus on development instead of maintenance. What we need is a way to manage environments and packages together. 

Pipenv rolls the management of Python virtual environments and Python packages into a single tool. Pipenv ensures that each project uses the correct version of each package it needs, and that each of those packages has the correct dependencies as well.

Further, Pipenv generates a list of your project’s dependencies that can travel with it, allowing other users or developers to set up the same project in the same way. Other users will also need to install Pipenv to properly set up a Pipenv-managed project, but fortunately, installing and using Pipenv is a breeze. 

How Pipenv works

Typically when you create a Python project and use a virtual environment for its packages, you’re tasked with creating the virtual environment yourself (using the command py -m venv), installing dependencies into it, and tracking the dependencies manually.

Pipenv provides a way to do all of this semi-automatically. The virtual environment for your project is created and managed for you when you install packages via Pipenv’s command-line interface. Dependencies are tracked and locked, and you you can manage development and runtime dependencies separately. You can also migrate from existing old-school requirements.txt files, so you don’t need to tear your project apart and start it over from scratch to use Pipenv well.

Note that unlike other Python project management tools (such as Poetry), Pipenv does not manage the “scaffolding” of your project. That is, Pipenv does not create the internal structure of the project directory with mock tests, documentation stubs, etc., but focuses chiefly on package and environment management. This makes Pipenv a good choice if you just want a tool to focus on virtual environments and packages, and not an all-in-one solution.

IDG

Get started with Pipenv

Pipenv installs in the same manner as most any other Python package: pip install --user pipenv. The --user option is recommended to keep Pipenv from conflicting with other system-wide packages. You should also add the path to the user base binary directory to the system path, so that Pipenv commands get routed to the right place.

If you plan to make Pipenv a consistent part of your workflow, it’s also a good idea to keep your underlying Python installation as minimal as possible. That advice applies for most any Python installation that makes use of virtual environments.

Set up a new project with Pipenv

To begin a completely new project with Pipenv, just create a directory and populate it with the files you’d normally create for a project. If you tend to scaffold a project as you go, you can start with an empty directory.

Installing packages for a project isn’t appreciably different with Pipenv than with Pip; in fact, the syntax is much the same. Open a console in your project directory and type pipenv install <package_name> to install a package for the project. To specify that the package is for development, use the -d flag. You can use pip syntax to denote a specific version of a package (e.g., black==13.0b1).

Most projects need only the main and development (-d) sets of dependencies. If you want more, such as development vs. testing vs. bleeding edge, you can use the --categories flag to set categories for the dependencies you’re installing.

Package installs with Pipenv

When you install a package with Pipenv, two things happen.

First, Pipenv will check if a virtual environment has already been created for this project directory. If yes, Pipenv will install the package into the existing virtual environment. If no, Pipenv will create a virtual environment that uses the same edition of Python used to run Pipenv. Note that the virtual environment is not created in the project directory itself; it’s created in a directory managed by Pipenv in your user profile.

Second, Pipenv will install the requested packages to the virtual environment. When the installation is done, Pipenv will report back on all that it did, including a path to the virtual environment if it had to create one.

You generally don’t need to know the path to the virtual environment Pipenv creates. To activate the environment, just navigate to your project directory and use pipenv shell to launch a new shell session or use pipenv run <command> to run a command directly. For example, use pipenv run mypy to run the command-line tool version of mypy (assuming the mypy tool was installed in the virtual environment), or pipenv run python -m <module> to run a Python module available in the virtual environment.

Pipenv and lockfiles

Peek inside the project directory after you’ve installed packages with Pipenv, and you’ll see two files, Pipfile and Pipfile.lock. Both are auto-generated by Pipenv, and should not be edited directly, as they describe the state of the packages in the project.

Pipfile is the simpler of the two. It just lists the packages needed for the project, where they’re installed from (the default is PyPI), and which version of Python is needed to run everything. In other words, Pipfile is akin to requirements.txt.

Pipfile.lock is more complex. It lists each package along with version details and SHA-256 hashes generated from the package. The hashes are used to ensure that the installed packages match exactly what’s specified — not just the version number, but the obtained contents as well.

When you work on a project that uses Pipenv for package management, you’ll want to add the Pipfile and Pipfile.lock files to the version control repository for the project. Any changes made to the packages for your project will in turn alter those files, so those changes should be tracked and versioned.

Use a Pipenv project

If you download a source repository for a project that uses Pipenv for package management, all you need to do is unpack the contents of the repository into a directory and run pipenv install (no package names needed). Pipenv will read the Pipfile and Pipfile.lock files for the project, create the virtual environment, and install all of the dependencies as needed.

Finally, if you want to use Pipenv to manage a project that currently uses a requirements.txt file, just navigate to the project’s directory and run pipenv install. Pipenv will detect the requirements.txt (or you can use the -r flag to point to it) and migrate all of the requirements into a Pipfile.