Tutorial

How To Package And Distribute Python Applications

Published on January 14, 2014
Default avatar

By O.S Tezer

How To Package And Distribute Python Applications

Introduction


All Python libraries (i.e. application packages) that you download using a package manager (e.g. pip) are distributed using a utility dedicated to do the job. These utilities create “Python distributions” which are basically versioned (and compressed) archives. All related elements to what’s being distributed, such as source files and resource files, are contained within it.

In this DigitalOcean article, we are going to talk about the necessary tools for distribution and go over the key steps to allow you to package your own useful libraries, modules, or applications – which should help you when deploying your project on a droplet or sharing on the internet.

Python Distributions and Packages


Even if you have worked only a little with Python, you will be familiar with the concept of using a package manager (e.g. pip, easy_install) to download modules and libraries (e.g. application development frameworks) which are then imported and used to create a new one.

These package management tools, operating locally, connect to a source (i.e. Python Package Index - PyPI) and perform a desired action (e.g. search and install) as they work these resources which are actually called Python distributions.

The way to distribute an application consists of wrapping its directory with some must-have files (along with a few recommended ones), specifying related elements (e.g. resources, dependencies etc.) and releasing it or using it elsewhere…that simple.

Note: You are highly encouraged to work with virtual environments to isolate Python downloads, modules, and applications you are working with.

Python Packages


In Python, a package [technically] is an importable directory (with __init__.py) containing source files (i.e. modules). This shall not be confused with operating-system packages, which are [technically] actual applications (i.e. a Debian package). However, it must be noted that Python distributions are indeed called packages as well.

Example package structure:

package
  |
  |-- __init__.py

Python Applications


Although anything from a single file to one with hundreds scattered across various packages can be considered an application in Python, in most realistic scenarios, an application will consist of multiple modules and a certain amount of external imports (from libraries).

Example application structure:

myapp
  |
  |-- __init__.py
  |-- amodule.py
  |-- anothermod.py
  |__ tests
  |     |
  |     |-- __init__.py
  |     |-- ..
  |     |-- .
  | ..

Python Distribution Tools and Libraries


Given the popular nature of Python and the rich amount of third-party libraries / applications written for it, a simpler and unified way of distributing has always been a necessity. There have been several different tools and libraries used for creating Python distributions.

In order to deal with the tasks of distribution, Python distribution utilities toolset distutils was created.

Python Package Index (PyPI)


Python Package Index, or PyPI, is a central [online] repository for projects (Python distributions). Package managing tools such as pip use this repository in order to host, find and install them.

Getting Started


Let"s begin with creating a simple, general Python flask application [structure] which we then can use to package.

Creating the Application Structure


We aim to create an example that resembles most real-world projects. Therefore, it will be best to imagine a scenario with modularised components.

Example structure:

/MyApplication
    |-- run.py
    |-- config.py
    |__ /app
         |-- __init__.py
         |-- /module_one
             |-- __init__.py
             |-- controllers.py
             |-- models.py                
         |__ /templates
             |-- module_one
                 |-- hello.html
         |__ /static
         |__ ..
         |__ .

Create the folders:


mkdir ~/MyApplication
cd    ~/MyApplication
touch run.py
touch config.py
mkdir app
cd    app
touch __init__.py
mkdir templates
mkdir static
mkdir module_one
cd    module_one
touch __init__.py
touch controllers.py
touch models.py
cd    ../templates
mkdir module_one
cd    module_one
touch hello.html

Edit run.py using nano:


nano ~/MyApplication/run.py

Place the contents:

# Run a test server.
from app import app
app.run(debug=True)

Save and exit using CTRL+X and confirm with with Y.

Edit config.py using nano:


nano ~/MyApplication/config.py

Place the contents:

DEBUG = True

THREADS_PER_PAGE = 4

CSRF_ENABLED     = True
CSRF_SESSION_KEY = "secret"

Save and exit using CTRL+X and confirm with with Y.

Edit app/init.py using nano:


nano ~/MyApplication/app/__init__.py

Place the contents:

from flask import Flask, render_template

app = Flask(__name__)
app.config.from_object("config")

from app.module_one.controllers import module_one

app.register_blueprint(module_one)

Save and exit using CTRL+X and confirm with with Y.

Edit app/module_one/controllers.py using nano:


nano app/module_one/controllers.py

Place the contents:

from flask import Blueprint, request, render_template

module_one = Blueprint("auth", __name__, url_prefix="/auth")

@module_one.route("/hello")
def hello():
    return render_template("module_one/hello.html")

Save and exit using CTRL+X and confirm with with Y.

Place the contents:

Edit app/templates/module_one/hello.html using nano:


nano app/templates/module_one/hello.html

Place the contents:

<pre> &lt!DOCTYPE html> &lthtml lang=“en”> &lthead> &lttitle>{% block title %}My Site{% endblock %}</title> {% block css %} {% endblock %} &ltmeta name=“viewport” content=“width=device-width, initial-scale=1.0”> &lt/head> &ltbody> Hello, world! &lt/body> &lt/html> </pre>

Save and exit using CTRL+X and confirm with with Y.

Beginning with Application Distribution / Packaging


Having created an exemplary application structure of a web site that uses flask, we can continue with taking the first step into preparing the distribution.

Altering the Folder Structure


In order to package our application well, we need to make some additions to our folder structure.

/MyApplication
    |-- run.py
    |__ /app
         |-- __init__.py
         |-- /module_one
             |-- __init__.py
             |-- controllers.py
             |-- models.py                
         |__ /templates
             |-- module_one
                 |-- hello.html
         |__ /static
         |__ ..
         |__ .
    |-- setup.py    # Distribution setup file
    |-- README.txt  # Read-me file
    |-- MANIFEST.in # Distribution manifest file
    |-- CHANGES.txt # Changes log

Alter the folder structure to create necessary files:

touch ~/MyApplication/setup.py
touch ~/MyApplication/README.py
touch ~/MyApplication/MANIFEST.py
touch ~/MyApplication/CHANGES.py
mv    ~/MyApplication/run.py ~/MyApplication/bin/run

Create the setup.py


nano ~/MyApplication/setup.py

Place the below self explanatory contents:

from distutils.core import setup

setup(
    # Application name:
    name="MyApplication",
    
    # Version number (initial):
    version="0.1.0",
    
    # Application author details:
    author="name surname",
    author_email="name@addr.ess",
    
    # Packages
    packages=["app"],
    
    # Include additional files into the package
    include_package_data=True,
    
    # Details
    url="http://pypi.python.org/pypi/MyApplication_v010/",
    
    #
    # license="LICENSE.txt",
    description="Useful towel-related stuff.",
    
    # long_description=open("README.txt").read(),
    
    # Dependent packages (distributions)
    install_requires=[
        "flask",
    ],
)

Save and exit using CTRL+X and confirm with with Y.

Create the MANIFEST.in


If you need to ship extra directories (e.g. static or templates), you need to explicitly state them in the manifest to be packaged. We will do this inside the MANIFEST.in.

nano ~/MyApplication/MANIFEST.in

Place the below self explanatory contents:

recursive-include app/templates *
recursive-include app/static *

Save and exit using CTRL+X and confirm with with Y.

And that’s it! Your Python distribution package is ready to be installed and shipped.

Additional Files


Please remember that in order to have a complete distribution, your file/directory must contain (and linked):

  • README.txt

  • MANIFEST.in

  • LICENSE.txt

Working With the Distribution Ready Application


As we have finalized creation of our application followed by making necessary amendments to the file structure to prepare it for a flawless distribution build, we can begin with going through the packaging operations.

How to Create The Distribution File


In order to generate a distribution file copy, run the following:

cd     ~/MyApplication
python setup.py sdist

This command will go through your setup, print out the operations being performed and generate a tar archive inside the newly created dist directory, similar to:

# root@hostname:~/MyApplication# ls dist
# MyApplication-0.1.0.tar.gz

Note: Since we did not populate all the sub-folders (i.e. static) and worked with additional files (e.g. README.txt), you might see some warnings during the creation process.

How to Install The Application


From now on, your application can be installed and used by others using the setup.py file created.

In order to install the application, run the following:

python setup.py install

If this installation is for development and the requirements are also to be installed, run the following:

python setup.py develop

How to Share Your Application


If you would like to share your code on the Python Packaging Index, you can do so by initiating the “register” procedure as per the following:

python setup.py register

You will need to complete the procedure by following the on-screen instructions.

If you have a registered login, in order to just upload, you can use the following:

python setup.py sdist upload

How to Create Packages of Your Application’s New Versions


  1. Edit the setup.py file with a text editor (e.g. nano) and set the new version number: version="0.1.1"

  2. Edit the CHANGES.txt to reflect the changes

  3. Make the necessary adjustments to the LICENSE.txt and README.txt

  4. Upload your code following the previous step.

<div class=“author”>Submitted by: <a href=“https://twitter.com/ostezer”>O.S. Tezer</a></div>

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


About the authors
Default avatar
O.S Tezer

author

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
5 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

I have two questions :

Note: You are highly encouraged to work with virtual environments to isolate Python downloads, modules, and applications you are working with.

Should I also apply this isolation principle for production environment? Regarding settings, configuration files etc. - how are they managed and selected at deployment (e.g. I would imagine to have one file for uat, another for prod etc.) ?

if you are facing issues while uploading your application using “python setup.py sdist upload” command, you can try twine as well.

  1. First, install twine by “pip install twine”.
  2. create the dist by “python setup.py sdist”
  3. twine upload dist/*

It will ask you for credentials, provide the same and your application will be uploaded to the central repository.

Has anyone ran into issues like this:

/usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'install_requires'

I’m having issues with config.py. How is ~/MyApplication/app/__init__.py supposed to be able to find it when it is a layer above it ~/MyApplication/config.py?

very help for us!

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel