How to Write an Installable Django App

 

Table of Contents

 

In the Django framework, a project refers to the collection of configuration files and code for a particular website. Django groups business logic into what it calls apps, which are the modules of the Django framework. There’s plenty of documentation on how to structure your projects and the apps within them, but when it comes time to package an installable Django app, information is harder to find.

In this tutorial, you’ll learn how to take an app out of a Django project and package it so that it’s installable. Once you’ve packaged your app, you can share it on PyPI so that others can fetch it through pip install.

In this tutorial, you’ll learn:

  • What the differences are between writing stand-alone apps and writing apps inside of projects
  • How to create a setup.cfg file for publishing your Django app
  • How to bootstrap Django outside of a Django project so you can test your app
  • How to test across multiple versions of Python and Django using tox
  • How to publish your installable Django app to PyPI using Twine

Even if you originally intend to make your Django app available as a package, you’re likely to start inside a project. To demonstrate the process of moving from Django project to installable Django app, I’ve made two branches available in the repo. The project branch is the starting state of an app inside of a Django project. The master branch is the finished installable app.

You can also download the finished app at the PyPI realpython-django-receipts package page. You can install the package by running pip install realpython-django-receipts.

The sample app is a short representation of the line items on a receipt. In the project branch, you’ll find a directory named sample_project that contains a working Django project. The directory looks like this:

sample_project/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   └── views.py
│
├── sample_project/
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
│
├── db.sqlite3
├── manage.py
├── resetdb.sh
└── runserver.sh

The most current version of Django at the time this tutorial was written was 3.0.4, and all testing was done with Python 3.7. None of the steps outlined in this tutorial should be incompatible with earlier versions of Django—I’ve used these techniques since Django 1.8. However, some changes are necessary if you’re using Python 2. To keep the examples simple, I’ve assumed Python 3.7 across the code base.

Creating the Django Project From Scratch

The sample project and receipts app were created using the Django admin command and some small edits. To start, run the following code inside of a clean virtual environment:

$ python -m pip install Django
$ django-admin startproject sample_project
$ cd sample_project
$ ./manage.py startapp receipts

This creates the sample_project project directory structure and a receipts app subdirectory with template files that you’ll use to create your installable Django app.

Next, the sample_project/settings.py file needs a few modifications:

  • Add '127.0.0.1' to the ALLOWED_HOSTS setting so you can test locally.
  • Add 'receipts' to the INSTALLED_APPS list.

You’ll also need to register the receipts app’s URLs in the sample_project/urls.py file. To do so, add path('receipts/', include('receipts.urls')) to the url_patterns list.

Exploring the Receipts Sample App

The app consists of two ORM model classes: Item and Receipt. The Item class contains database field declarations for a description and a cost. The cost is contained in a DecimalField. Using floating-point numbers to represent money is dangerous—you should always use fixed-point numbers when dealing with currencies.

The Receipt class is a collection point for Item objects. This is achieved with a ForeignKey on Item that points to ReceiptReceipt also includes total() for getting the total cost of Item objects contained in the Receipt:

# receipts/models.py
from decimal import Decimal
from django.db import models

class Receipt(models.Model):
    created = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f"Receipt(id={self.id})"

    def total(self) -> Decimal:
        return sum(item.cost for item in self.item_set.all())

class Item(models.Model):
    created = models.DateTimeField(auto_now_add=True)

    description = models.TextField()
    cost = models.DecimalField(max_digits=7, decimal_places=2)
    receipt = models.ForeignKey(Receipt, on_delete=models.CASCADE)

    def __str__(self):
        return f"Item(id={self.id}, description={self.description}, " \
            f"cost={self.cost})"

The model objects give you content for the database. A short Django view returns a JSON dictionary with all the Receipt objects and their Item objects in the database:

# receipts/views.py
from django.http import JsonResponse
from receipts.models import Receipt

def receipt_json(request):
    results = {
        "receipts":[],
    }

    for receipt in Receipt.objects.all():
        line = [str(receipt), []]
        for item in receipt.item_set.all():
            line[1].append(str(item))

        results["receipts"].append(line)

    return JsonResponse(results)

The receipt_json() view iterates over all the Receipt objects, creating a pair of the Receipt objects and a list of the Item objects contained within. All of this is put in a dictionary and returned through Django’s JsonResponse().

To make the models available in the Django admin interface, you use an admin.py file to register the models:

# receipts/admin.py
from django.contrib import admin

from receipts.models import Receipt, Item

@admin.register(Receipt)
class ReceiptAdmin(admin.ModelAdmin):
    pass

@admin.register(Item)
class ItemAdmin(admin.ModelAdmin):
    pass

This code creates a Django ModelAdmin for each of the Receipt and Item classes and registers them with the Django admin.

Finally, a urls.py file registers a single view in the app against a URL:

# receipts/urls.py
from django.urls import path

from receipts import views

urlpatterns = [
    path("receipt_json/", views.receipt_json),
]

You can now include receipts/urls.py in your project’s url.py file to make the receipt view available on your website.

With everything in place, you can run ./manage.py makemigrations receipts, use the Django admin to add data, and then visit /receipts/receipt_json/ to view the results:

$ curl -sS http://127.0.0.1:8000/receipts/receipt_json/ | python3.8 -m json.tool
{
    "receipts": [
        [
            "Receipt(id=1)",
            [
                "Item(id=1, description=wine, cost=15.25)",
                "Item(id=2, description=pasta, cost=22.30)"
            ]
        ],
        [
            "Receipt(id=2)",
            [
                "Item(id=3, description=beer, cost=8.50)",
                "Item(id=4, description=pizza, cost=12.80)"
            ]
        ]
    ]
}

In the block above, you use curl to visit the receipt_json view, resulting in a JSON response containing the Receipt objects and their corresponding Item objects.

Testing the App in the Project

Django augments the Python unittest package with its own testing capabilities, enabling you to preload fixtures into the database and run your tests. The receipts app defines a tests.py file and a fixture to test with. This test is by no means comprehensive, but it’s a good enough proof of concept:

# receipts/tests.py
from decimal import Decimal
from django.test import TestCase
from receipts.models import Receipt

class ReceiptTest(TestCase):
    fixtures = ["receipts.json", ]

    def test_receipt(self):
        receipt = Receipt.objects.get(id=1)
        total = receipt.total()

        expected = Decimal("37.55")
        self.assertEqual(expected, total)

The fixture creates two Receipt objects and four corresponding Item objects. Click on the collapsible section below for a closer look at the code for the fixture.

You can test the receipts app with the Django manage.py command:

$ ./manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.013s

OK
Destroying test database for alias 'default'...

Running manage.py test runs the single test defined in receipts/tests.py and displays the results.

Making Your Installable Django App

Your goal is to share the receipts app without a project and to make it reusable by others. You could zip up the receipts/ directory and hand it out, but that’s somewhat limiting. Instead, you want to separate the app into a package so it’s installable.

The biggest challenge in creating an installable Django app is that Django expects a project. An app without a project is just a directory containing code. Without a project, Django doesn’t know how to do anything with your code, including running tests.

Moving Your Django App Out of the Project

It’s a good idea to keep a sample project around so you can run the Django dev server and play with a live version of your app. You won’t include this sample project in the app package, but it can still live in your repository. Following this idea, you can get started with packaging your installable Django app by moving it up a directory:

$ mv receipts ..

The directory structure now looks something like this:

django-receipts/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   ├── views.py
│   ├── admin.py
│   └── apps.py
│
├── sample_project/
│   ├── sample_project/
│   │   ├── __init__.py
│   │   ├── asgi.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   │
│   ├── db.sqlite3
│   ├── manage.py
│   ├── resetdb.sh
│   └── runserver.sh
│
├── LICENSE
└── README.rst

To package your app, you need to pull it out of the project. Moving it is the first step. I typically keep the original project around to test with, but I don’t include it in the resulting package.

Bootstrapping Django Outside of a Project

Now that your app is outside of a Django project, you need to tell Django how to find it. If you want to test your app, then run a Django shell that can find your app or run your migrations. You’ll need to configure Django and make it available.

Django’s settings.configure() and django.setup() are key to interacting with your app outside of a project. More information on these calls is available in the Django documentation.

You’re likely to need this configuration of Django in several places, so it makes sense to define it in a function. Create a file called boot_django.py containing the following code:

 1# boot_django.py
 2#
 3# This file sets up and configures Django. It's used by scripts that need to
 4# execute as if running in a Django server.
 5import os
 6import django
 7from django.conf import settings
 8
 9BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), "receipts"))
10
11def boot_django():
12    settings.configure(
13        BASE_DIR=BASE_DIR,
14        DEBUG=True,
15        DATABASES={
16            "default":{
17                "ENGINE":"django.db.backends.sqlite3",
18                "NAME": os.path.join(BASE_DIR, "db.sqlite3"),
19            }
20        },
21        INSTALLED_APPS=(
22            "receipts",
23        ),
24        TIME_ZONE="UTC",
25        USE_TZ=True,
26    )
27    django.setup()

Lines 12 and 27 set up the Django environment. The settings.configure() call takes a list of arguments that are equivalent to the variables defined in a settings.py file. Anything you would need in your settings.py to make your app run gets passed into settings.configure().

The above code is a fairly stripped-down configuration. The receipts app doesn’t do anything with sessions or templates, so INSTALLED_APPS only needs "receipts", and you can skip over any middleware definitions. The USE_TZ=True value is necessary because the Receipt model contains a created timestamp. Otherwise, you would run into problems loading the test fixture.

Running Management Commands With Your Installable Django App

Now that you have boot_django.py, you can run any Django management command with a very short script:

#!/usr/bin/env python
# makemigrations.py

from django.core.management import call_command
from boot_django import boot_django

boot_django()
call_command("makemigrations", "receipts")

Django allows you to programmatically call management commands through call_command(). You can now run any management command by importing and calling boot_django() followed by call_command().

Your app is now outside the project, allowing you to do all sorts of Django-y things to it. I often define four utility scripts:

  1. load_tests.py to test your app
  2. makemigrations.py to create migration files
  3. migrate.py to perform table migrations
  4. djangoshell.py to spawn a Django shell that’s aware of your app

Testing Your Installable Django App

The load_test.py file could be as simple as the makemigrations.py script, but then it would only be able to run all the tests at once. With a few additional lines, you can pass command-line arguments to the test runner, allowing you to run selective tests:

 1#!/usr/bin/env python
 2# load_tests.py
 3import sys
 4from unittest import TestSuite
 5from boot_django import boot_django
 6
 7boot_django()
 8
 9default_labels = ["receipts.tests", ]
10
11def get_suite(labels=default_labels):
12    from django.test.runner import DiscoverRunner
13    runner = DiscoverRunner(verbosity=1)
14    failures = runner.run_tests(labels)
15    if failures:
16        sys.exit(failures)
17
18    # In case this is called from setuptools, return a test suite
19    return TestSuite()
20
21if __name__ == "__main__":
22    labels = default_labels
23    if len(sys.argv[1:]) > 0:
24        labels = sys.argv[1:]
25
26    get_suite(labels)

Django’s DiscoverRunner is a test discovery class compatible with Python’s unittest. It’s responsible for setting up the test environment, building the suite of tests, setting up the databases, running the tests, and then tearing it all down. Starting on line 11get_suite() takes a list of test labels and directly calls the DiscoverRunner on them.

This script is similar to what the Django management command test does. The __main__ block passes any command-line arguments to get_suite(), and if there are none, then it passes in the test suite for the app, receipts.tests. You can now call load_tests.py with a test label argument and run a single test.

Line 19 is a special case to help when testing with tox. You’ll learn more about tox in a later section. You can also check out a potential substitute for DiscoverRunner in the collapsible section below.

 

Defining Your Installable Package With setup.cfg

To put your installable Django app on PyPI, you need to first put it in a package. PyPI expects an eggwheel, or source distribution. These are built using setuptools. To do this, you need to create a setup.cfg file and a setup.py file at the same directory level as your receipts directory.

Before digging into that, though, you want to make sure you have some documentation. You can include a project description in setup.cfg, which is automatically displayed on the PyPI project page. Make sure to write a README.rst or something similar with information about your package.

PyPI supports the reStructuredText format by default, but it can also handle Markdown with extra parameters:

 1# setup.cfg
 2[metadata]
 3name = realpython-django-receipts
 4version = 1.0.3
 5description = Sample installable django app
 6long_description = file:README.rst
 7url = https://github.com/realpython/django-receipts
 8license = MIT
 9classifiers =
10    Development Status :: 4 - Beta
11    Environment :: Web Environment
12    Intended Audience :: Developers
13    License :: OSI Approved :: MIT License
14    Operating System :: OS Independent
15    Programming Language :: Python :: 3 :: Only
16    Programming Language :: Python :: 3.7
17    Programming Language :: Python :: Implementation :: CPython
18    Topic :: Software Development :: Libraries :: Application Frameworks
19    Topic :: Software Development :: Libraries :: Python Modules
20
21[options]
22include_package_data = true
23python_requires = >=3.6
24setup_requires =
25    setuptools >= 38.3.0
26install_requires =
27    Django>=2.2

This setup.cfg file describes the package that you’ll build. Line 6 uses the file: directive to read in your README.rst file. This saves you from having to write a long description in two places.

The install_requires entry on line 26 tells any installers, such as pip install, about the dependencies your app has. You will always want to tie your installable Django app to its minimum supported version of Django.

If your code has any dependencies that are only required to run the tests, then you can add a tests_require = entry. For example, before mock became part of the standard Python library, it was common to see tests_require = mock>=2.0.0 in setup.cfg.

It’s considered best practice to include a pyproject.toml file with your package. Brett Cannon’s excellent article on the subject can run you through the details. A pyproject.toml file is also included with the sample code.

You’re almost ready to build the package for your installable Django app. The easiest way to test it is with your sample project— another good reason to keep a sample project around. The pip install command supports locally defined packages. This can be used to make sure that your app still works with a project. However, one caveat is that setup.cfg won’t work on its own in this case. You’ll also have to create a shim version of setup.py:

#!/usr/bin/env python

if __name__ == "__main__":
    import setuptools
    setuptools.setup()

This script will automatically use your setup.cfg file. You can now install a local editable version of the package to test it from within sample_project. To be doubly sure, it’s best to start with a brand-new virtual environment. Add the following requirements.txt file inside the sample_project directory:

# requirements.txt
-e ../../django-receipts

The -e tells pip that this is a local editable installation. You’re now ready to install:

$ pip install -r requirements.txt
Obtaining django-receipts (from -r requirements.txt (line 1))
Collecting Django>=3.0
  Using cached Django-3.0.4-py3-none-any.whl (7.5 MB)
Collecting asgiref~=3.2
  Using cached asgiref-3.2.7-py2.py3-none-any.whl (19 kB)
Collecting pytz
  Using cached pytz-2019.3-py2.py3-none-any.whl (509 kB)
Collecting sqlparse>=0.2.2
  Using cached sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
Installing collected packages: asgiref, pytz, sqlparse, Django, realpython-django-receipts
  Running setup.py develop for realpython-django-receipts
Successfully installed Django-3.0.4 asgiref-3.2.7 pytz-2019.3 realpython-django-receipts sqlparse-0.3.1

The install_requires list in setup.cfg tells pip install that it needs Django. Django needs asgirefpytz, and sqlparse. All the dependencies are taken care of and you should now be able to run your sample_project Django dev server. Congratulations—your app is now packaged and referenced from within the sample project!

Testing Multiple Versions With tox

Django and Python are both constantly advancing. If you’re going to share your installable Django app with the world, then you’re probably going to need to test in multiple environments. The tox tool needs a little help to be able to test your Django app. Go ahead and make the following change inside of setup.cfg:

 1# setup.cfg
 2[metadata]
 3name = realpython-django-receipts
 4version = 1.0.3
 5description = Sample installable django app
 6long_description = file:README.rst
 7url = https://github.com/realpython/django-receipts
 8license = MIT
 9classifiers =
10    Development Status :: 4 - Beta
11    Environment :: Web Environment
12    Intended Audience :: Developers
13    License :: OSI Approved :: MIT License
14    Operating System :: OS Independent
15    Programming Language :: Python :: 3 :: Only
16    Programming Language :: Python :: 3.7
17    Programming Language :: Python :: Implementation :: CPython
18    Topic :: Software Development :: Libraries :: Application Frameworks
19    Topic :: Software Development :: Libraries :: Python Modules
20
21[options]
22include_package_data = true
23python_requires = >=3.6
24setup_requires =
25    setuptools >= 38.3.0
26install_requires =
27    Django>=2.2
28test_suite = load_tests.get_suite

Line 28 tells the package manager to use the load_tests.py script to get its test suite. The tox utility uses this to run its tests. Recall get_suite() in load_tests.py:

 1# Defined inside load_tests.py
 2def get_suite(labels=default_labels):
 3    from django.test.runner import DiscoverRunner
 4    runner = DiscoverRunner(verbosity=1)
 5    failures = runner.run_tests(labels)
 6    if failures:
 7        sys.exit(failures)
 8
 9    # If this is called from setuptools, then return a test suite
10    return TestSuite()

What is happening here is admittedly a little bit weird. Normally, the test_suite field in setup.cfg points to a method that returns a suite of tests. When tox calls setup.py, it reads the test_suite parameter and runs load_tests.get_suite().

If this call didn’t return a TestSuite object, then tox would complain. The weird part is that you don’t actually want tox to get a suite of tests because tox is not aware of the Django test environment. Instead, get_suite() creates a DiscoverRunner and returns an empty TestSuite object on line 10.

You can’t simply have DiscoverRunner return a suite of tests because you have to call DiscoverRunner.run_tests() for the setup and teardown of the Django test environment to execute correctly. Merely passing the correct tests to tox wouldn’t work because the database wouldn’t be created. get_suite() runs all the tests, but as a side effect of the function call rather than as the normal case of returning a test suite for tox to execute.

The tox tool allows you to test multiple combinations. A tox.ini file determines which combinations of environments to test. Here’s an example:

[tox]
envlist = py{36,37}-django220, py{36,37}-django300

[testenv]
deps =
    django220: Django>=2.2,<3
    django300: Django>=3
commands=
    python setup.py test

This file states that tests should be run for Python 3.6 and 3.7 combined with Django 2.2 and 3.0. That’s a total of four test environments. The commands= section is where you tell tox to call the test through setup.py. This is how you invoke the test_suite = load_tests.get_suite hook in setup.cfg.

Publishing to PyPI

Finally, it’s time to share your installable Django app on PyPI. There are multiple tools for uploading a package, but you’ll focus on Twine in this tutorial. The following code builds the packages and invokes Twine:

$ python -m pip install -U wheel twine setuptools
$ python setup.py sdist
$ python setup.py bdist_wheel
$ twine upload dist/*

The first two commands build the source and binary distributions of your package. The call to twine uploads to PyPI. If you have a .pypirc file in your home directory, then you can preset your username so the only thing you’re prompted for is your password:

[disutils]
index-servers =
    pypi

[pypi]
username: <YOUR_USERNAME>

I often use a small shell script to grep the version number from the code. Then I call git tag to tag the repo with the version number, remove the old build/ and dist/ directories, and call the above three commands.

For more details on using Twine, see How to Publish an Open-Source Python Package to PyPI. Two popular alternatives to Twine are Poetry and Flit. Package management in Python is changing rapidly. PEP 517 and PEP 518 are redefining how to describe Python packages and dependencies.

Conclusion

Django apps rely on the Django project structure, so packaging them separately requires extra steps. You’ve seen how to make an installable Django app by extracting it from a project, packaging it, and sharing it on PyPI. Be sure to download the sample code at the link below:

In this tutorial, you’ve learned how to:

  • Use the Django framework outside of a project
  • Call Django management commands on an app that is independent of a project
  • Write a script that invokes Django tests, optionally using a single test label
  • Build a setup.py file to define your package
  • Modify the setup.py script to accommodate tox
  • Use Twine to upload your installable Django app

You’re all set to share your next app with the world. Happy coding!

Further Reading

Django, packaging, and testing are all very deep topics. There’s lots of information out there. To dig in deeper, check out the following resources:

PyPI has loads of installable Django apps that are worth trying out. Here are some of the most popular:

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *