Tutorial

How To Manage Python with Pyenv and Direnv

Published on December 12, 2019
author

Johnny M

How To Manage Python with Pyenv and Direnv

Introduction

Whether you’re just getting started or you’re a seasoned Python developer, you may have found managing your Python environments to be tedious and painful. Managing Python versions, libraries, and various dependencies is like playing shuffleboard where the object is to not hit any other puck; if you do, the probability of a cascading effect of pucks flying everywhere you don’t want them to be will soon follow.

In this tutorial, you’ll install pyenv for managing python environments, install direnv to auto configure and source the virtualenv for projects, set a global Python version and a local Python version for projects, and configure a virtualenv and auto source the venv when moving into your project directories. By the end of this tutorial, you will be able to install any valid version of python, set up and configure virtual environments for each project, and bring sanity from chaos.

Prerequisites

We’re going to install both pyenv and direnv via homebrew to ease the install process. Before getting started, install homebrew if you don’t have it already.

Working with pyenv

To start, you’ll see how to work with pyenv.

Installing pyenv

To get things started, we’re going to install pyenv with homebrew and add the required pyenv init to our ~/.bashrc file.

  1. brew install pyenv
  2. echo 'eval "$(pyenv init -)"' >> ~/.bashrc # initialize pyenv on new shells
  3. source ~/.bashrc # reinitialize bashrc to reflect changes in your current shell

Once it’s installed, take a moment to examine what our environment looks like. On a fresh system, you’ll see something similar to this:

  1. which pyenv
Output
/usr/local/bin/pyenv
  1. pyenv versions
Output
* system
  1. which pip
Output
/usr/local/bin/pip
  1. which python
Output
/usr/local/bin/python

This is a pretty standard snapshot. When you execute python version, you’ll notice * system, as the reference implies, this is your system’s python version. The asterisk denotes the current python binary sourced in your ${PATH}. A good rule of thumb is to leave the system binary alone. If you want to see the list of python versions that pyenv can install for you, use pyenv install --list.

  1. pyenv install 2.7.15
  2. pyenv install 3.7.0
  3. pyenv versions
Output
* system (set by /Users/iamjohnnym/.pyenv/version) 2.7.15 3.7.0

Configuring the Base Requirements

Now, let’s upgrade pip since chances are, it installed an older version. The following command will loop through your installed versions and update pip to the latest.

for VERSION in $(pyenv versions --bare) ; do
  pyenv shell ${VERSION} ;
  pip install --upgrade pip ;
done

For the sake of our desired workflow, we want to install py2venv for our Python 2.x versions so we can mimic how python 3.x installs virtualenvs. python -m venv .venv.

for VERSION in $(pyenv versions --bare | egrep '^2.') ; do
  pyenv shell ${VERSION} ;
  pip install py2venv ;
done

Setting the Global Python Version

Even though we have pyenv installed, it will still default to system. To change this, set python 3.7.0 as our global version:

  1. pyenv global 3.7.0
  2. pyenv versions
Output
system 2.7.15 * 3.7.0 (set by /Users/iamjohnnym/.pyenv/version)
  1. which python
Output
/Users/iamjohnnym/.pyenv/shims/python

We’ve got pyenv setup and functional. Let’s move on to direnv.

Working with direnv

direnv is a handy utility that allows you to create a file that’s placed in any directory that you want that functions like .bashrc. Whenever you enter the directory with this file, your shell will automatically execute it. The capabilities are endless but for the purpose of this post, we’re going to use it to configure a Python virtualenv based on the file .python-version and then activate it for us. The purpose is to create a seamless development flow. No need to worry about manually configuring or activating virtualenvs; let your computer do the work for you.

Installing direnv

Installation is straightforward with homebrew:

  1. brew install direnv

direnv is now installed, and ready for exploitation. Let’s try a sample project.

Creating a New Project

Let’s start up a project. We’re going to create a new directory, set a local python version, set up our .envrc file, and activate it.

  1. mkdir -p ~/python-projects/pyenv-tutorial
  2. cd $_ # if you're unaware, $_ will execute the last argument of your command
  3. pwd
Output
/Users/iamjohnnym/ python-projects/pyenv-tutorial
  1. pyenv local 3.7.0
  2. cat .python-version
Output
3.7.0

Configuring the Environment File

At this point, we’re ready to create our .envrc file. With your favorite editor, create that file and add the following contents:

# check if python version is set in current dir
if [ -f ".python-version" ] ; then
    if [ ! -d ".venv" ] ; then
        echo "Installing virtualenv for $(python -V)"
        # if we didn't install `py2venv` for python 2.x, we would need to use
        # `virtualenv`, which you would have to install separately.
        python -m venv .venv
    fi
    echo "Activating $(python -V) virtualenv"
    source .venv/bin/activate
fi
# announce python version and show the path of the current python in ${PATH}
echo "Virtualenv has been activated for $(python -V)"
echo "$(which python)"

Save the file. If you did this via a shell editor, such as vim. You’ll see the following message, direnv: error .envrc is blocked. Run direnv allow to approve its content. Don’t be alarmed as this is a security feature to prevent auto-execution of the file. Whenever this file is changed, it requires manual approval before it will auto-execute again. To activate it, simply type direnv allow from the project dir.

  1. direnv allow
Output
direnv: loading .envrc Installing virtualenv for Python 3.7.0 Activating Python 3.7.0 virtualenv Virtualenv has been activated for Python 3.7.0 /Users/iamjohnnym/.personal/tutorials/pyenv-direnv/.venv/bin/python direnv: export +VIRTUAL_ENV ~PATH

Conclusion

You now have the tools you need to manage different python versions and project dependencies!

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about our products

About the authors
Default avatar
Johnny M

author

While we believe that this content benefits our community, we have not yet thoroughly reviewed it. If you have any suggestions for improvements, please let us know by clicking the “report an issue“ button at the bottom of the tutorial.

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
Leave a comment


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Featured on Community

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more