Virtual environments for Python are very handy for several reasons when you need to:
The source code of pyenv is available here: https://github.com/pyenv/pyenv
pyenv is a little bit different, it will allow you to change a used version of Python completely. I found it handy on few servers running Ubuntu 14.04, where I needed to upgrade salt-stack to a newer version and that version supports only Python version 2.7.15, but Ubuntu 14.04 latest version in official repositories is 2.7.6. The shell recipe in this section is about the installation of the python and with this new version available, you can install additional virtual environments with tools (pipenv, venv, virtualenv) described in next sections.
Pyenv is primarily for:
Here is the example of installation of specific version (2.7.15):
git clone https://github.com/pyenv/pyenv.git ~/.pyenv echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~./.bashrc echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc echo 'eval "$(pyenv init -)"' >> ~/.bashrc source ~/.bashrc apt-get install make # be sure you have make installed pyenv install 2.7.15 echo 2.7.15 > /root/.pyenv/version source ~/.bashrc python --version # check that version is enabled
sudo pip3 install venv # install venv system-wide # Create a new virtual environment. # This will copy current Python on your system to that directory: python3 -m venv directory # Activate virtual environment: source directory/bin/activate # Your shell environment is now using that directory for all Python # related operations. You may notice, that your PS1 variable contains the # name of that directory in brackets. # Few commands to check where you are: pip --version # will show a new path for package installation python3 -c 'import sys; print(sys.path)' # system paths are also updated # To exit virtual environment use this command, which is actually a shell # function sourced by the 'activate' script you have used before: deactivate # These commands will show default operating system values after deactivation: pip --version python3 -c 'import sys; print(sys.path)'
This is very similar to the previous section:
sudo pip3 install virtualenv # install virtualenv system-wide # Create a new virtual environment. # This will copy current Python on your system to that directory: virtualenv directory # this script will execute the same as # To select explicit interpreter use -p or --python= virtualenv -p python2.7 directory python3 -m virualenv directory # use python2 or pytho3 to select # specified version # Activate virtual environment: source directory/bin/activate # Your shell environment is now using that directory for all Python # related operations. You may notice, that your PS1 variable contains the # name of that directory. # Few commands to check where you are: pip --version # will show a new path for package installation python -c 'import sys; print(sys.path)' # system paths are also updated # To exit virtual environment use deactivate
This is officially recommended solution for installing python dependencies by python.org. The pipenv session may look like this:
sudo pip3 install pipenv # install it system wide mkdir project # create a directory for a new project cd project pipenv install # Your project directory now contains following files Pipfile Pipfile.lock # Install packages: pipenv install package1 package2 # install packages # You can check section [packages] in Pipfile to see installed stuff: less Pipfile pipenv graph # will show a graph of dependencies # You can execute commands in the virtual environment without activating: pipenv run pip --version pipenv run python3 -c 'import sys; print(sys.path)' pipenv --venv # will show where is located virtual installation of the python # Activate virtual environment pipenv shell # To exit virtual environment use simply exit
To be able to span your virtual environment of the project everywhere, you need to save a state of installed modules. Command pip freeze will do it. The default file to save the list is requirements.txt. It will save also versions off all pip packages.
# Save current installed modules pip freeze > requirements.txt # Install packages from requirements.txt pip install -r requirements.txt # When pipenv is used pipenv install -r requirements.txt
Handy utilities dealing with packages are part of the package pip-tools. You need to create an input file, usually requirements.in and specify manually what packages are necessary. See example below with specification of versions of packages:
dnspython # do not care about version (will find latest) elasticsearch>=5.0.0,<6.0.0 # use versions between errbot==5.1.2 # install exact version salt-pepper>=0.5.0,!=0.5.3,!=0.5.4 # install version higher then, but ignoreOnce you have requirements.in present in your project directory use:
pip install pip-tools # install pip-tools # Generate requirements.txt with all dependencies pip-compile requirements.in # Install packages specified in requirements.txt pip-sync
The last option, I am currently using, is running python in a Docker container. In this case, you need to build a Docker image, use this image locally or deploy it somewhere. You can already have prepared requirements files in the same directory as Dockerfile is located. The image is created from a Dockerfile which may look like this:
# Base image from a public repository FROM python:3.7.1 # Copy file from the local directory to the image COPY requirements.txt /build/requirements.txt # Just example of an environmental variable set inside the image ENV AWS_REGION="us-east-1" # Install dependencies from requirements.txt RUN pip3 install --upgrade -r /build/requirements.txt # Copy local files to the image COPY your_script.py /opt/your_script.sh COPY ./dep/*.py /opt/dep/ # Be sure that the script is executable RUN chmod +x /opt/your_script.sh # What will be executed ENTRYPOINT ["/opt/your_script.sh", "-c", "parameter" ]
Stencil of the Dockerfile is ready, now build and deploy:
# Build from Dockerfile located in the work directory docker build . -t image_name # Build from specific Dockerfile docker build -f ../other/Dockerfile -t image_name # Check new image in your local repository docker images | head # Deploy image to some server # pv is optional, it is for monitoring of the progress of data through a pipe docker save image_name | bzip2 | pv | ssh user@server 'bunzip2 | docker load' # Execute image, and name running container 'application' docker run --name="application" image_name