Django Demo Project
Flow:
HTTP Request -> urls.py -> forward request to view: views.py -> HTTP Response (HTML)
models.py <-> read / write data views.py
Template (
-
View: A view is a request handler function, which receives HTTP requests and returns HTTP responses
-
Templates: A template is a text file defining the structure or layout of a file (such as an HTML page), with placeholders used to represent actual content
Create the project
django-admin startproject locallibrary
It has the following structure:
locallibrary/
manage.py
locallibrary/
__init__.py
settings.py
urls.py
wsgi.py
asgi.py
Create the catalog application
Make sure to run this command from the same folder as your project’s manage.py:
py -3 manage.py startapp catalog
update structure:
locallibrary/
manage.py
locallibrary/
catalog/
admin.py
apps.py
models.py
tests.py
views.py
__init__.py
migrations/
Register the catalog application
We have to register it with the project so that it will be included when any tools are run (like adding models to the database for example).
Applications are registered by adding them to the INSTALLED_APPS list in the project settings.
# Add our new application
'catalog.apps.CatalogConfig', # This object was created for us in /catalog/apps.py
A django App is a fancy name for a python package. Really, that’s it. The only thing that would distinguish a django app from other python packages is that it makes sense for it to appear in the INSTALLED_APPS list in settings.py, because it contains things like templates, models, or other features that can be auto-discovered by other django features.
from What exactly are Django Apps
Specify the database
You can see how this database is configured in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}
Other project settings
The settings.py file is also used for configuring a number of other settings, but at this point, you probably only want to change the TIME_ZONE
TIME_ZONE = 'Hongkong'
-
SECRET_KEY. This is a secret key that is used as part of Django’s website security strategy. If you’re not protecting this code in development, you’ll need to use a different code (perhaps read from an environment variable or file) when putting it into production.
-
DEBUG. This enables debugging logs to be displayed on error, rather than HTTP status code responses. This should be set to False in production as debug information is useful for attackers, but for now we can keep it set to True.
Hook up the URL mapper
The website is created with a URL mapper file (urls.py) in the project folder. While you can use this file to manage all your URL mappings, it is more usual to defer mappings to the associated application.
The `urlpatterns` list routes URLs to views. For more information please see:
https://docs.djangoproject.com/en/5.0/topics/http/urls/
Examples:
Function views
1. Add an import: from my_app import views
2. Add a URL to urlpatterns: path('', views.home, name='home')
Class-based views
1. Add an import: from other_app.views import Home
2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
Including another URLconf
1. Import the include() function: from django.urls import include, path
2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
This new item includes a path() that forwards requests with the pattern catalog/ to the module catalog.urls (the file with the relative URL catalog/urls.py).
# Use include() to add paths from the catalog application
from django.urls import include
urlpatterns += [
path('catalog/', include('catalog.urls')),
]
Now let’s redirect the root URL of our site (i.e. 127.0.0.1:8000) to the URL 127.0.0.1:8000/catalog/. This is the only app we’ll be using in this project.
To do this, we’ll use a special view function, RedirectView, which takes the new relative URL to redirect to (/catalog/) as its first argument when the URL pattern specified in the path() function is matched (the root URL, in this case).
# Add URL maps to redirect the base URL to our application
from django.views.generic import RedirectView
urlpatterns += [
path('', RedirectView.as_view(url='catalog/', permanent=True)),
]
Leave the first parameter of the path function empty to imply ‘/’. If you write the first parameter as ‘/’ Django will give you the following warning when you start the development server:
System check identified some issues:
WARNINGS:
?: (urls.W002) Your URL pattern '/' has a route beginning with a '/'.
Remove this slash as it is unnecessary.
If this pattern is targeted in an include(), ensure the include() pattern has a trailing '/'.
Django does not serve static files like CSS, JavaScript, and images by default, but it can be useful for the development web server to do so while you’re creating your site. As a final addition to this URL mapper, you can enable the serving of static files during development by appending the following lines.
Django does not serve static files like CSS, JavaScript, and images by default, but it can be useful for the development web server to do so while you’re creating your site. You can enable the serving of static files during development by appending the following lines.
# Use static() to add URL mapping to serve static files during development (only)
from django.conf import settings
from django.conf.urls.static import static
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
As a final step, create a file inside your catalog folder called urls.py, and add the following text to define the (empty) imported urlpatterns. This is where we’ll add our patterns as we build the application.
from django.urls import path
from . import views
urlpatterns = [
]
Test the website framework
Before we run the server, we should first run a database migration. This updates our database (to include any models in our installed applications) and removes some build warnings.
Run database migrations
Django uses an Object-Relational-Mapper (ORM) to map model definitions in the Django code to the data structure used by the underlying database. As we change our model definitions, Django tracks the changes and can create database migration scripts (in /django-locallibrary-tutorial/catalog/migrations/) to automatically migrate the underlying data structure in the database to match the model.
make sure you are in the directory that contains manage.py
py -3 manage.py makemigrations
py -3 manage.py migrate
Warning: You’ll need to run these commands every time your models change in a way that will affect the structure of the data that needs to be stored (including both addition and removal of whole models and individual fields).
Run the website
During development, you can serve the website first using the development web server, and then viewing it on your local web browser.
Note: The development web server is not robust or performant enough for production use, but for a convenient quick test. By default it will serve the site to your local computer (http://127.0.0.1:8000/), but you can also specify other computers on your network to serve to.
py -3 manage.py runserver
Design the LocalLibrary models
Django web applications access and manage data through Python objects referred to as models. Models define the structure of stored data, including the field types and possibly also their maximum size, default values, selection list options, help text for documentation, label text for forms, etc. The definition of the model is independent of the underlying database — you can choose one of several as part of your project settings. Once you’ve chosen what database you want to use, you don’t need to talk to it directly at all — you just write your model structure and other code, and Django handles all the dirty work of communicating with the database for you.
Model primer
Model definition
Models are usually defined in an application’s models.py file. They are implemented as subclasses of django.db.models.Model, and can include fields, methods and metadata.
Fields
A model can have an arbitrary number of fields, of any type — each one represents a column of data that we want to store in one of our database tables.
my_field_name = models.CharField(max_length=20, help_text='Enter field documentation')
-
max_length=20 — States that the maximum length of a value in this field is 20 characters.
-
help_text=’Enter field documentation’ — helpful text that may be displayed in a form to help users understand how the field is used.
The field name is used to refer to it in queries and templates. Fields also have a label, which is specified using the verbose_name argument (with a default value of None). If verbose_name is not set, the label is created from the field name by replacing any underscores with a space, and capitalizing the first letter (for example, the field my_field_name would have a default label of My field name when used in forms).
COMMON FIELD ARGUMENTS
-
default: The default value for the field. This can be a value or a callable object, in which case the object will be called every time a new record is created.
-
null: If True, Django will store blank values as NULL in the database for fields where this is appropriate (a CharField will instead store an empty string). The default is False.
-
blank: If True, the field is allowed to be blank in your forms. The default is False, which means that Django’s form validation will force you to enter a value. This is often used with null=True, because if you’re going to allow blank values, you also want the database to be able to represent them appropriately.
-
choices: A group of choices for this field. If this is provided, the default corresponding form widget will be a select box with these choices instead of the standard text field.
-
unique: If True, ensures that the field value is unique across the database. This can be used to prevent duplication of fields that can’t have the same values. The default is False.
-
primary_key: If True, sets the current field as the primary key for the model (A primary key is a special database column designated to uniquely identify all the different table records). If no field is specified as the primary key, Django will automatically add a field for this purpose. The type of auto-created primary key fields can be specified for each app in AppConfig.default_auto_field or globally in the DEFAULT_AUTO_FIELD setting.
COMMON FIELD TYPES
-
CharField is used to define short-to-mid sized fixed-length strings. You must specify the max_length of the data to be stored.
-
TextField is used for large arbitrary-length strings. You may specify a max_length for the field, but this is used only when the field is displayed in forms (it is not enforced at the database level).
-
IntegerField is a field for storing integer (whole number) values, and for validating entered values as integers in forms.
-
DateField and DateTimeField are used for storing/representing dates and date/time information (as Python datetime.date and datetime.datetime objects, respectively). These fields can additionally declare the (mutually exclusive) parameters auto_now=True (to set the field to the current date every time the model is saved), auto_now_add (to only set the date when the model is first created), and default (to set a default date that can be overridden by the user).
-
EmailField is used to store and validate email addresses.
-
FileField and ImageField are used to upload files and images respectively (the ImageField adds additional validation that the uploaded file is an image). These have parameters to define how and where the uploaded files are stored.
-
AutoField is a special type of IntegerField that automatically increments. A primary key of this type is automatically added to your model if you don’t explicitly specify one.
-
ForeignKey is used to specify a one-to-many relationship to another database model. The “one” side of the relationship is the model that contains the “key” (models containing a “foreign key” referring to that “key”, are on the “many” side of such a relationship).
-
ManyToManyField is used to specify a many-to-many relationship. In our library app we will use these very similarly to ForeignKeys, but they can be used in more complicated ways to describe the relationships between groups. These have the parameter on_delete to define what happens when the associated record is deleted (e.g. a value of models.SET_NULL would set the value to NULL).
Metadata
You can declare model-level metadata for your Model by declaring class Meta, as shown.
class Meta:
ordering = ['title', '-pubdate']
As shown above, you can prefix the field name with a minus symbol (-) to reverse the sorting order.
Many of the other metadata options control what database must be used for the model and how the data is stored (these are really only useful if you need to map a model to an existing database).
Methods
Minimally, in every model you should define the standard Python class method __str__() to return a human-readable string for each object. This string is used to represent individual records in the administration site (and anywhere else you need to refer to a model instance). Often this will return a title or name field from the model.
def __str__(self):
return self.my_field_name
Another common method to include in Django models is get_absolute_url(), which returns a URL for displaying individual model records on the website (if you define this method then Django will automatically add a “View on Site” button to the model’s record editing screens in the Admin site). A typical pattern for get_absolute_url() is shown below.
def get_absolute_url(self):
"""Returns the URL to access a particular instance of the model."""
return reverse('model-detail-view', args=[str(self.id)])
The reverse() function above is able to “reverse” your URL mapper (in the above case named ‘model-detail-view’) in order to create a URL of the right format.
You can also define any other methods you like, and call them from your code or templates (provided that they don’t take any parameters).
Model management
Create and modify records
To create a record you can define an instance of the model and then call save().
# Create a new record using the model's constructor.
record = MyModelName(my_field_name="Instance #1")
# Save the object into the database.
record.save()
Note: If you haven’t declared any field as a primary_key, the new record will be given one automatically, with the field name id. You could query this field after saving the above record, and it would have a value of 1.
You can access the fields in this new record using the dot syntax, and change the values. You have to call save() to store modified values to the database.
# Access model field values using Python attributes.
print(record.id) # should return 1 for the first record.
print(record.my_field_name) # should print 'Instance #1'
# Change record by modifying the fields, then calling save().
record.my_field_name = "New Instance Name"
record.save()
Search for records
You can search for records that match certain criteria using the model’s objects attribute (provided by the base class).
We can get all records for a model as a QuerySet, using objects.all(). The QuerySet is an iterable object, meaning that it contains a number of objects that we can iterate/loop through.
all_books = Book.objects.all()
wild_books = Book.objects.filter(title__contains='wild')
number_wild_books = wild_books.count()
The fields to match and the type of match are defined in the filter parameter name, using the format: field_name__match_type (note the double underscore between title and contains above). Above we’re filtering title with a case-sensitive match. There are many other types of matches you can do: icontains (case insensitive), iexact (case-insensitive exact match), exact (case-sensitive exact match) and in, gt (greater than), startswith, etc.
In some cases, you’ll need to filter on a field that defines a one-to-many relationship to another model (e.g. a ForeignKey). In this case, you can “index” to fields within the related model with additional double underscores. So for example to filter for books with a specific genre pattern, you will have to index to the name through the genre field, as shown below:
# Will match on: Fiction, Science fiction, non-fiction etc.
books_containing_genre = Book.objects.filter(genre__name__icontains='fiction')
Note: You can use underscores (__) to navigate as many levels of relationships (ForeignKey/ManyToManyField) as you like. For example, a Book that had different types, defined using a further “cover” relationship might have a parameter name: type__cover__name__exact=’hard’.
Define the LocalLibrary Models
Genre model
from django.urls import reverse # Used in get_absolute_url() to get URL for specified ID
from django.db.models import UniqueConstraint # Constrains fields to unique values
from django.db.models.functions import Lower # Returns lower cased value of field
class Genre(models.Model):
"""Model representing a book genre."""
name = models.CharField(
max_length=200,
unique=True,
help_text="Enter a book genre (e.g. Science Fiction, French Poetry etc.)"
)
def __str__(self):
"""String for representing the Model object."""
return self.name
def get_absolute_url(self):
"""Returns the url to access a particular genre instance."""
return reverse('genre-detail', args=[str(self.id)])
class Meta:
constraints = [
UniqueConstraint(
Lower('name'),
name='genre_name_case_insensitive_unique',
violation_error_message = "Genre already exists (case insensitive match)"
),
]
After the field, we declare a ` str()` method, which returns the name of the genre defined by a particular record. No verbose name has been defined, so the field label will be Name when it is used in forms. Then we declare the get_absolute_url() method, which returns a URL that can be used to access a detail record for this model (for this to work, we will have to define a URL mapping that has the name genre-detail, and define an associated view and template).
verbose_name is a human-readable name for the field. If the verbose name isn’t given, Django will automatically create it using the field’s attribute name, converting underscores to spaces.
Setting unique=True on the field above prevents genres being created with exactly the same name, but not variations such as “fantasy”, “Fantasy”, or even “FaNtAsY”. The last part of the model definition uses a constraints option on the model’s metadata to specify that the lower case of the value in the name field must be unique in the database, and display the violation_error_message string if it isn’t. Here we don’t need to do anything else, but you can define multiple constrainst a field or fields.
Book model
The model uses a CharField to represent the book’s title and isbn. For isbn, note how the first unnamed parameter explicitly sets the label as “ISBN” (otherwise, it would default to “Isbn”).
class Book(models.Model):
"""Model representing a book (but not a specific copy of a book)."""
title = models.CharField(max_length=200)
author = models.ForeignKey('Author', on_delete=models.RESTRICT, null=True)
# Foreign Key used because book can only have one author, but authors can have multiple books.
# Author as a string rather than object because it hasn't been declared yet in file.
summary = models.TextField(
max_length=1000, help_text="Enter a brief description of the book")
isbn = models.CharField('ISBN', max_length=13,
unique=True,
help_text='13 Character <a href="https://www.isbn-international.org/content/what-isbn'
'">ISBN number</a>')
# ManyToManyField used because genre can contain many books. Books can cover many genres.
# Genre class has already been defined so we can specify the object above.
genre = models.ManyToManyField(
Genre, help_text="Select a genre for this book")
def __str__(self):
"""String for representing the Model object."""
return self.title
def get_absolute_url(self):
"""Returns the URL to access a detail record for this book."""
return reverse('book-detail', args=[str(self.id)])
The other parameters of interest in the author field are null=True, which allows the database to store a Null value if no author is selected, and on_delete=models.RESTRICT, which will prevent the book’s associated author being deleted if it is referenced by any book.
Warning: By default on_delete=models.CASCADE, which means that if the author was deleted, this book would be deleted too! We use RESTRICT here, but we could also use PROTECT to prevent the author being deleted while any book uses it or SET_NULL to set the book’s author to Null if the record is deleted.
BookInstance model
import uuid # Required for unique book instances
class BookInstance(models.Model):
"""Model representing a specific copy of a book (i.e. that can be borrowed from the library)."""
id = models.UUIDField(primary_key=True, default=uuid.uuid4,
help_text="Unique ID for this particular book across whole library")
book = models.ForeignKey('Book', on_delete=models.RESTRICT, null=True)
imprint = models.CharField(max_length=200)
due_back = models.DateField(null=True, blank=True)
LOAN_STATUS = (
('m', 'Maintenance'),
('o', 'On loan'),
('a', 'Available'),
('r', 'Reserved'),
)
status = models.CharField(
max_length=1,
choices=LOAN_STATUS,
blank=True,
default='m',
help_text='Book availability',
)
class Meta:
ordering = ['due_back']
def __str__(self):
"""String for representing the Model object."""
return f'{self.id} ({self.book.title})'
-
UUIDField is used for the id field to set it as the primary_key for this model. This type of field allocates a globally unique value for each instance (one for every book you can find in the library).
-
DateField is used for the due_back date (at which the book is expected to become available after being borrowed or in maintenance). This value can be blank or null (needed for when the book is available). The model metadata (Class Meta) uses this field to order records when they are returned in a query.
-
status is a CharField that defines a choice/selection list. As you can see, we define a tuple containing tuples of key-value pairs and pass it to the choices argument. The value in a key/value pair is a display value that a user can select, while the keys are the values that are actually saved if the option is selected. We’ve also set a default value of ‘m’ (maintenance) as books will initially be created unavailable before they are stocked on the shelves.
Author model
class Author(models.Model):
"""Model representing an author."""
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
date_of_birth = models.DateField(null=True, blank=True)
date_of_death = models.DateField('Died', null=True, blank=True)
class Meta:
ordering = ['last_name', 'first_name']
def get_absolute_url(self):
"""Returns the URL to access a particular author instance."""
return reverse('author-detail', args=[str(self.id)])
def __str__(self):
"""String for representing the Model object."""
return f'{self.last_name}, {self.first_name}'
Rerun the database migrations
All your models have now been created. Now rerun your database migrations to add them to your database.
py -3 manage.py makemigrations
py -3 manage.py migrate
Django admin site
The Django admin application can use your models to automatically build a site area that you can use to create, view, update, and delete records. This can save you a lot of time during development, making it very easy to test your models and get a feel for whether you have the right data. The admin application can also be useful for managing data in production, depending on the type of website. The Django project recommends it only for internal data management (i.e. just for use by admins, or people internal to your organization), as the model-centric approach is not necessarily the best possible interface for all users, and exposes a lot of unnecessary detail about the models.
All the configuration required to include the admin application in your website was done automatically when you created the skeleton project. As a result, all you must do to add your models to the admin application is to register them.
Register models
First, open admin.py in the catalog application (/django-locallibrary-tutorial/catalog/admin.py). It currently looks like this — note that it already imports django.contrib.admin:
Register the models by copying the following text into the bottom of the file. This code imports the models and then calls admin.site.register to register each of them.
from .models import Author, Genre, Book, BookInstance, Language
admin.site.register(Book)
admin.site.register(Author)
admin.site.register(Genre)
admin.site.register(BookInstance)
admin.site.register(Language)
Create a superuser
In order to log into the admin site, we need a user account with Staff status enabled. In order to view and create records we also need this user to have permissions to manage all our objects. You can create a “superuser” account that has full access to the site and all needed permissions using manage.py.
Call the following command, in the same directory as manage.py, to create the superuser. You will be prompted to enter a username, email address, and strong password.
py -3 manage.py createsuperuser
Once this command completes a new superuser will have been added to the database. Now restart the development server so we can test the login:
python3 manage.py runserver
Log in and use the site
To login to the site, open the /admin URL (e.g. http://127.0.0.1:8000/admin) and enter your new superuser userid and password credentials (you’ll be redirected to the login page, and then back to the /admin URL after you’ve entered your details).
This part of the site displays all our models, grouped by installed application. You can click on a model name to go to a screen that lists all its associated records, and you can further click on those records to edit them. You can also directly click the Add link next to each model to start creating a record of that type.
Advanced configuration
You can further customize the interface to make it even easier to use. Some of the things you can do are:
-
List views:
- Add additional fields/information displayed for each record.
- Add filters to select which records are listed, based on date or some other selection value (e.g. Book loan status).
- Add additional options to the actions menu in list views and choose where this menu is displayed on the form.
-
Detail views
- Choose which fields to display (or exclude), along with their order, grouping, whether they are editable, the widget used, orientation etc.
- Add related fields to a record to allow inline editing (e.g. add the ability to add and edit book records while you’re creating their author record).
Register a ModelAdmin class
To change how a model is displayed in the admin interface you define a ModelAdmin class (which describes the layout) and register it with the model.
Let’s start with the Author model. Open admin.py in the catalog application (/django-locallibrary-tutorial/catalog/admin.py). Comment out your original registration (prefix it with a #) for the Author model:
Now add a new AuthorAdmin and registration as shown below.
# Define the admin class
class AuthorAdmin(admin.ModelAdmin):
pass
# Register the admin class with the associated model
admin.site.register(Author, AuthorAdmin)
Now to create and register the new models; for the purpose of this demonstration, we’ll instead use the @register decorator to register the models (this does exactly the same thing as the admin.site.register() syntax):
# Register the Admin classes for Book using the decorator
@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
pass
# Register the Admin classes for BookInstance using the decorator
@admin.register(BookInstance)
class BookInstanceAdmin(admin.ModelAdmin):
pass
Configure list views
The LocalLibrary currently lists all authors using the object name generated from the model __str__() method. This is fine when you only have a few authors, but once you have many you may end up having duplicates. To differentiate them, or just because you want to show more interesting information about each author, you can use list_display to add additional fields to the view.
Replace your AuthorAdmin class with the code below. The field names to be displayed in the list are declared in a tuple in the required order, as shown (these are the same names as specified in your original model).
class AuthorAdmin(admin.ModelAdmin):
list_display = ('last_name', 'first_name', 'date_of_birth', 'date_of_death')
For our Book model we’ll additionally display the author and genre. The author is a ForeignKey field (one-to-many) relationship, and so will be represented by the __str__() value for the associated record. Replace the BookAdmin class with the version below.
class BookAdmin(admin.ModelAdmin):
list_display = ('title', 'author', 'display_genre')
Unfortunately we can’t directly specify the genre field in list_display because it is a ManyToManyField (Django prevents this because there would be a large database access “cost” in doing so). Instead we’ll define a display_genre function to get the information as a string (this is the function we’ve called above; we’ll define it below).
Note: Getting the genre may not be a good idea here, because of the “cost” of the database operation. We’re showing you how because calling functions in your models can be very useful for other reasons — for example to add a Delete link next to every item in the list.
Add the following code into your Book model (models.py). This creates a string from the first three values of the genre field (if they exist) and creates a short_description that can be used in the admin site for this method.
def display_genre(self):
"""Create a string for the Genre. This is required to display genre in Admin."""
return ', '.join(genre.name for genre in self.genre.all()[:3])
display_genre.short_description = 'Genre'
Add list filters
Once you’ve got a lot of items in a list, it can be useful to be able to filter which items are displayed. This is done by listing fields in the list_filter attribute. Replace your current BookInstanceAdmin class with the code fragment below.
class BookInstanceAdmin(admin.ModelAdmin):
list_filter = ('status', 'due_back')
Organize detail view layout
By default, the detail views lay out all fields vertically, in their order of declaration in the model. You can change the order of declaration, which fields are displayed (or excluded), whether sections are used to organize the information, whether fields are displayed horizontally or vertically, and even what edit widgets are used in the admin forms.
Controlling which fields are displayed and laid out
Update your AuthorAdmin class to add the fields line, as shown below:
class AuthorAdmin(admin.ModelAdmin):
list_display = ('last_name', 'first_name', 'date_of_birth', 'date_of_death')
fields = ['first_name', 'last_name', ('date_of_birth', 'date_of_death')]
The fields attribute lists just those fields that are to be displayed on the form, in order. Fields are displayed vertically by default, but will display horizontally if you further group them in a tuple (as shown in the “date” fields above).
Sectioning the detail view
You can add “sections” to group related model information within the detail form, using the fieldsets attribute.
In the BookInstance model we have information related to what the book is (i.e. name, imprint, and id) and when it will be available (status, due_back). We can add these to our BookInstanceAdmin class as shown below, using the fieldsets property.
@admin.register(BookInstance)
class BookInstanceAdmin(admin.ModelAdmin):
list_filter = ('status', 'due_back')
fieldsets = (
(None, {
'fields': ('book', 'imprint', 'id')
}),
('Availability', {
'fields': ('status', 'due_back')
}),
)
Each section has its own title (or None, if you don’t want a title) and an associated tuple of fields in a dictionary — the format is complicated to describe, but fairly easy to understand if you look at the code fragment immediately above.
Inline editing of associated records
Sometimes it can make sense to be able to add associated records at the same time. For example, it may make sense to have both the book information and information about the specific copies you’ve got on the same detail page.
You can do this by declaring inlines, of type TabularInline (horizontal layout) or StackedInline (vertical layout, just like the default model layout). You can add the BookInstance information inline to our Book detail by specifying inlines in your BookAdmin:
class BooksInstanceInline(admin.TabularInline):
model = BookInstance
@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
list_display = ('title', 'author', 'display_genre')
inlines = [BooksInstanceInline]
Create our home page
Create the index page
The first page we’ll create is the index page (catalog/). The index page will include some static HTML, along with generated “counts” of different records in the database. To make this work we’ll create a URL mapping, a view, and a template.
URL mapping
When we created the skeleton website, we updated the locallibrary/urls.py file to ensure that whenever a URL that starts with catalog/ is received, the URLConf module catalog.urls will process the remaining substring.
The following code snippet from locallibrary/urls.py includes the catalog.urls module:
urlpatterns += [
path('catalog/', include('catalog.urls')),
]
We also created a placeholder file for the URLConf module, named /catalog/urls.py. Add the following lines to that file:
urlpatterns = [
path('', views.index, name='index'),
]
The path() function defines the following:
-
A url pattern, which is an empty string: ‘’. We’ll discuss url patterns in detail when working on the other views.
-
A view function that will be called if the URL pattern is detected: views.index, which is the function named index() in the views.py file.
The path() function also specifies a name parameter, which is a unique identifier for this particular URL mapping. You can use the name to “reverse” the mapper, i.e. to dynamically create a URL that points to the resource that the mapper is designed to handle. For example, we can use the name parameter to link to our home page from any other page by adding the following link in a template:
<a href="{\% url 'index' \%}">Home</a>.
Note: We can hard code the link as in
<a href="/catalog/">Home</a>), but if we change the pattern for our home page, for example, to/catalog/index)the templates will no longer link correctly. Using a reversed URL mapping is more robust.
View (function-based)
A view is a function that processes an HTTP request, fetches the required data from the database, renders the data in an HTML page using an HTML template, and then returns the generated HTML in an HTTP response to display the page to the user. The index view follows this model — it fetches information about the number of Book, BookInstance, available BookInstance and Author records that we have in the database, and passes that information to a template for display.
Open catalog/views.py and note that the file already imports the render() shortcut function to generate an HTML file using a template and data:
from django.shortcuts import render
# Create your views here.
Paste the following lines at the bottom of the file:
from .models import Book, Author, BookInstance, Genre
def index(request):
"""View function for home page of site."""
# Generate counts of some of the main objects
num_books = Book.objects.all().count()
num_instances = BookInstance.objects.all().count()
# Available books (status = 'a')
num_instances_available = BookInstance.objects.filter(status__exact='a').count()
# The 'all()' is implied by default.
num_authors = Author.objects.count()
context = {
'num_books': num_books,
'num_instances': num_instances,
'num_instances_available': num_instances_available,
'num_authors': num_authors,
}
# Render the HTML template index.html with the data in the context variable
return render(request, 'index.html', context=context)
The first line imports the model classes that we’ll use to access data in all our views.
The first part of the view function fetches the number of records using the objects.all() attribute on the model classes. It also gets a list of BookInstance objects that have a value of ‘a’ (Available) in the status field.
At the end of the view function we call the render() function to create an HTML page and return the page as a response. This shortcut function wraps a number of other functions to simplify a very common use case. The render() function accepts the following parameters:
-
the original request object, which is an HttpRequest.
-
an HTML template with placeholders for the data.
-
a context variable, which is a Python dictionary, containing the data to insert into the placeholders.
Template
A template is a text file that defines the structure or layout of a file (such as an HTML page), it uses placeholders to represent actual content.
A Django application created using startapp (like the skeleton of this example) will look for templates in a subdirectory named ‘templates’ of your applications. For example, in the index view that we just added, the render() function will expect to find the file index.html in /django-locallibrary-tutorial/catalog/templates/ and will raise an error if the file is not present.
You can check this by saving the previous changes and accessing 127.0.0.1:8000 in your browser - it will display a fairly intuitive error message: “TemplateDoesNotExist at /catalog/”, and other details.
Note: Based on your project’s settings file, Django will look for templates in a number of places, searching in your installed applications by default. You can find out more about how Django finds templates and what template formats it supports in the Templates section of the Django documentation.
Extending templates
The index template will need standard HTML markup for the head and body, along with navigation sections to link to the other pages of the site (which we haven’t created yet), and to sections that display introductory text and book data.
Much of the HTML and navigation structure will be the same in every page of our site. Instead of duplicating boilerplate code on every page, you can use the Django templating language to declare a base template, and then extend it to replace just the bits that are different for each specific page.
The following code snippet is a sample base template from a base_generic.html file. We’ll be creating the template for LocalLibrary shortly. The sample below includes common HTML with sections for a title, a sidebar, and main contents marked with the named block and endblock template tags. You can leave the blocks empty, or include default content to use when rendering pages derived from the template.
Note: Template tags are functions that you can use in a template to loop through lists, perform conditional operations based on the value of a variable, and so on. In addition to template tags, the template syntax allows you to reference variables that are passed into the template from the view, and use template filters to format variables (for example, to convert a string to lower case).
<!DOCTYPE html>
<html lang="en">
<head>
{\% block title \%}
<title>Local Library</title>
{\% endblock \%}
</head>
<body>
{\% block sidebar \%}
<!-- insert default navigation text for every page -->
{\% endblock \%}
{\% block content \%}
<!-- default content text (typically empty) -->
{\% endblock \%}
</body>
</html>
When defining a template for a particular view, we first specify the base template using the extends template tag — see the code sample below. Then we declare what sections from the template we want to replace (if any), using block/endblock sections as in the base template.
For example, the code snippet below shows how to use the extends template tag and override the content block. The generated HTML will include the code and structure defined in the base template, including the default content you defined in the title block, but the new content block in place of the default one.
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Local Library Home</h1>
<p>
Welcome to LocalLibrary, a website developed by
<em>Mozilla Developer Network</em>!
</p>
{\% endblock \%}
The LocalLibrary base template
We will use the following code snippet as the base template for the LocalLibrary website. As you can see, it contains some HTML code and defines blocks for title, sidebar, and content. We have a default title and a default sidebar with links to lists of all books and authors, both enclosed in blocks to be easily changed in the future.
Note: We also introduce two additional template tags: url and load static. These tags will be explained in following sections.
Create a new file base_generic.html in /django-locallibrary-tutorial/catalog/templates/ and paste the following code to the file:
<!DOCTYPE html>
<html lang="en">
<head>
{\% block title \%}
<title>Local Library</title>
{\% endblock \%}
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css"
rel="stylesheet"
integrity="sha384-QWTKZyjpPEjISv5WaRU9OFeRpok6YctnYmDr5pNlyT2bRjXh0JMhjY6hW+ALEwIH"
crossorigin="anonymous">
<!-- Add additional CSS in static file -->
{\% load static \%}
<link rel="stylesheet" href="{\% static 'css/styles.css' \%}" />
</head>
<body>
<div class="container-fluid">
<div class="row">
<div class="col-sm-2">
{\% block sidebar \%}
<ul class="sidebar-nav">
<li><a href="{\% url 'index' \%}">Home</a></li>
<li><a href="">All books</a></li>
<li><a href="">All authors</a></li>
</ul>
{\% endblock \%}
</div>
<div class="col-sm-10 ">{\% block content \%}{\% endblock \%}</div>
</div>
</div>
</body>
</html>
The template includes CSS from Bootstrap to improve the layout and presentation of the HTML page. Using Bootstrap (or another client-side web framework) is a quick way to create an attractive page that displays well on different screen sizes.
The base template also references a local CSS file (styles.css) that provides additional styling. Create a styles.css file in /django-locallibrary-tutorial/catalog/static/css/ and paste the following code in the file:
.sidebar-nav {
margin-top: 20px;
padding: 0;
list-style: none;
}
The index template
Create a new HTML file index.html in /django-locallibrary-tutorial/catalog/templates/ and paste the following code in the file. This code extends our base template on the first line, and then replaces the default content block for the template.
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Local Library Home</h1>
<p>
Welcome to LocalLibrary, a website developed by
<em>Mozilla Developer Network</em>!
</p>
<h2>Dynamic content</h2>
<p>The library has the following record counts:</p>
<ul>
<li><strong>Books:</strong> </li>
<li><strong>Copies:</strong> </li>
<li><strong>Copies available:</strong> </li>
<li><strong>Authors:</strong> </li>
</ul>
{\% endblock \%}
In the Dynamic content section we declare placeholders (template variables) for the information from the view that we want to include. The variables are enclosed with double brace (handlebars).
Note: You can easily recognize template variables and template tags (functions) - variables are enclosed in double braces (), and tags are enclosed in single braces with percentage signs ({\% extends “base_generic.html” \%}).
The important thing to note here is that variables are named with the keys that we pass into the context dictionary in the render() function of our view (see sample below). Variables will be replaced with their associated values when the template is rendered.
Reference static files in templates
Your project is likely to use static resources, including JavaScript, CSS, and images. Because the location of these files might not be known (or might change), Django allows you to specify the location in your templates relative to the STATIC_URL global setting. The default skeleton website sets the value of STATIC_URL to ‘/static/’, but you might choose to host these on a content delivery network or elsewhere.
Within the template you first call the load template tag specifying “static” to add the template library, as shown in the code sample below. You can then use the static template tag and specify the relative URL to the required file.
<!-- Add additional CSS in static file -->
{\% load static \%}
<link rel="stylesheet" href="{\% static 'css/styles.css' \%}" />
You can add an image into the page in a similar way, for example:
{\% load static \%}
<img
src="{\% static 'catalog/images/local_library_model_uml.png' \%}"
alt="UML diagram"
style="width:555px;height:540px;" />
Link to URLs
The base template above introduced the url template tag.
<li><a href="{\% url 'index' \%}">Home</a></li>
This tag accepts the name of a path() function called in your urls.py and the values for any arguments that the associated view will receive from that function, and returns a URL that you can use to link to the resource.
Configure where to find the templates
The location where Django searches for templates is specified in the TEMPLATES object in the settings.py file. The default settings.py (as created for this tutorial) looks something like this:
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
The setting of ‘APP_DIRS’: True, is the most important, as it tells Django to search for templates in a subdirectory of each application in the project, named “templates” (this makes it easier to group templates with their associated application for easy re-use).
We can also specify specific locations for Django to search for directories using ‘DIRS’: [] (but that isn’t needed yet).
Generic list and detail views
Book list page
The book list page will display a list of all the available book records in the page, accessed using the URL: catalog/books/. The page will display a title and author for each record, with the title being a hyperlink to the associated book detail page. The page will have the same structure and navigation as all other pages in the site, and we can, therefore, extend the base template (base_generic.html) we created in the previous tutorial.
URL mapping
Open /catalog/urls.py and copy in the line setting the path for ‘books/’, as shown below. Just as for the index page, this path() function defines a pattern to match against the URL (‘books/’), a view function that will be called if the URL matches (views.BookListView.as_view()), and a name for this particular mapping.
urlpatterns = [
path('', views.index, name='index'),
path('books/', views.BookListView.as_view(), name='books'),
]
As discussed in the previous tutorial the URL must already have matched /catalog, so the view will actually be called for the URL: /catalog/books/.
The view function has a different format than before — that’s because this view will actually be implemented as a class. We will be inheriting from an existing generic view function that already does most of what we want this view function to do, rather than writing our own from scratch.
For Django class-based views we access an appropriate view function by calling the class method as_view(). This does all the work of creating an instance of the class, and making sure that the right handler methods are called for incoming HTTP requests.
View (class-based)
We could quite easily write the book list view as a regular function (just like our previous index view), which would query the database for all books, and then call render() to pass the list to a specified template. Instead, however, we’re going to use a class-based generic list view (ListView) — a class that inherits from an existing view. Because the generic view already implements most of the functionality we need and follows Django best-practice, we will be able to create a more robust list view with less code, less repetition, and ultimately less maintenance.
Open catalog/views.py, and copy the following code into the bottom of the file:
from django.views import generic
class BookListView(generic.ListView):
model = Book
That’s it! The generic view will query the database to get all records for the specified model (Book) then render a template located at /django-locallibrary-tutorial/catalog/templates/catalog/book_list.html (which we will create below). Within the template you can access the list of books with the template variable named object_list OR book_list (i.e. generically “<the model name>_list”).
Note: This awkward path for the template location isn’t a misprint — the generic views look for templates in /application_name/the_model_name_list.html (catalog/book_list.html in this case) inside the application’s /application_name/templates/ directory (/catalog/templates/).
You can add attributes to change the default behavior above. For example, you can specify another template file if you need to have multiple views that use this same model, or you might want to use a different template variable name if book_list is not intuitive for your particular template use-case. Possibly the most useful variation is to change/filter the subset of results that are returned — so instead of listing all books you might list top 5 books that were read by other users.
class BookListView(generic.ListView):
model = Book
context_object_name = 'book_list' # your own name for the list as a template variable
queryset = Book.objects.filter(title__icontains='war')[:5] # Get 5 books containing the title war
template_name = 'books/my_arbitrary_template_name_list.html' # Specify your own template name/location
Overriding methods in class-based views
While we don’t need to do so here, you can also override some of the class methods.
For example, we can override the get_queryset() method to change the list of records returned. This is more flexible than just setting the queryset attribute as we did in the preceding code fragment (though there is no real benefit in this case):
class BookListView(generic.ListView):
model = Book
def get_queryset(self):
return Book.objects.filter(title__icontains='war')[:5] # Get 5 books containing the title war
We might also override get_context_data() in order to pass additional context variables to the template (e.g. the list of books is passed by default). The fragment below shows how to add a variable named “some_data” to the context (it would then be available as a template variable).
class BookListView(generic.ListView):
model = Book
def get_context_data(self, **kwargs):
# Call the base implementation first to get the context
context = super(BookListView, self).get_context_data(**kwargs)
# Create any data and add it to the context
context['some_data'] = 'This is just some data'
return context
When doing this it is important to follow the pattern used above:
-
First get the existing context from our superclass.
-
Then add your new context information.
-
Then return the new (updated) context.
Creating the List View template
Create the HTML file /django-locallibrary-tutorial/catalog/templates/catalog/book_list.html and copy in the text below. As discussed above, this is the default template file expected by the generic class-based list view (for a model named Book in an application named catalog).
Templates for generic views are just like any other templates (although of course the context/information passed to the template may differ). As with our index template, we extend our base template in the first line and then replace the block named content.
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Book List</h1>
{\% if book_list \%}
<ul>
{\% for book in book_list \%}
<li>
<a href=""></a>
()
</li>
{\% endfor \%}
</ul>
{\% else \%}
<p>There are no books in the library.</p>
{\% endif \%}
{\% endblock \%}
The view passes the context (list of books) by default as object_list and book_list aliases; either will work.
You might also use the {\% empty \%} template tag to define what happens if the book list is empty (although our template chooses to use a conditional instead):
<ul>
{\% for book in book_list \%}
<li><!-- code here get information from each book item --></li>
{\% empty \%}
<p>There are no books in the library.</p>
{\% endfor \%}
</ul>
Update the base template
Open the base template (/django-locallibrary-tutorial/catalog/templates/base_generic.html) and insert {\% url ‘books’ \%} into the URL link for All books, as shown below. This will enable the link in all pages (we can successfully put this in place now that we’ve created the “books” URL mapper).
<li><a href="{\% url 'index' \%}">Home</a></li>
<li><a href="{\% url 'books' \%}">All books</a></li>
<li><a href="">All authors</a></li>
What does it look like?
You won’t be able to build the book list yet, because we’re still missing a dependency — the URL map for the book detail pages, which is needed to create hyperlinks to individual books. We’ll show both list and detail views after the next section.
Book detail page
The book detail page will display information about a specific book, accessed using the URL catalog/book/
URL mapping
Open /catalog/urls.py and add the path named ‘book-detail’ shown below. This path() function defines a pattern, associated generic class-based detail view, and a name.
urlpatterns = [
path('', views.index, name='index'),
path('books/', views.BookListView.as_view(), name='books'),
path('book/<int:pk>', views.BookDetailView.as_view(), name='book-detail'),
]
For the book-detail path the URL pattern uses a special syntax to capture the specific id of the book that we want to see. The syntax is very simple: angle brackets define the part of the URL to be captured, enclosing the name of the variable that the view can use to access the captured data. For example,
In this case we use ‘
Warning: The generic class-based detail view expects to be passed a parameter named pk. If you’re writing your own function view you can use whatever parameter name you like, or indeed pass the information in an unnamed argument.
Advanced path matching/regular expression primer
The pattern matching provided by path() is simple and useful for the (very common) cases where you just want to capture any string or integer. If you need more refined filtering (for example, to filter only strings that have a certain number of characters) then you can use the re_path() method.
This method is used just like path() except that it allows you to specify a pattern using a Regular expression.
Passing additional options in your URL maps
One feature that we haven’t used here, but which you may find valuable, is that you can pass a dictionary containing additional options to the view (using the third un-named argument to the path() function). This approach can be useful if you want to use the same view for multiple resources, and pass data to configure its behavior in each case.
For example, given the path shown below, for a request to /myurl/halibut/ Django will call views.my_view(request, fish=’halibut’, my_template_name=’some_path’).
path('myurl/<fish>', views.my_view, {'my_template_name': 'some_path'}, name='aurl'),
Note: Both named captured patterns and dictionary options are passed to the view as named arguments. If you use the same name for both a capture pattern and a dictionary key, then the dictionary option will be used.
View (class-based)
Open catalog/views.py, and copy the following code into the bottom of the file:
class BookDetailView(generic.DetailView):
model = Book
That’s it! All you need to do now is create a template called /django-locallibrary-tutorial/catalog/templates/catalog/book_detail.html, and the view will pass it the database information for the specific Book record extracted by the URL mapper. Within the template you can access the book’s details with the template variable named object OR book (i.e. generically “the_model_name”).
If you need to, you can change the template used and the name of the context object used to reference the book in the template. You can also override methods to, for example, add additional information to the context.
What happens if the record doesn’t exist?
If a requested record does not exist then the generic class-based detail view will raise an Http404 exception for you automatically — in production, this will automatically display an appropriate “resource not found” page, which you can customize if desired.
Just to give you some idea of how this works, the code fragment below demonstrates how you would implement the class-based view as a function if you were not using the generic class-based detail view.
def book_detail_view(request, primary_key):
try:
book = Book.objects.get(pk=primary_key)
except Book.DoesNotExist:
raise Http404('Book does not exist')
return render(request, 'catalog/book_detail.html', context={'book': book})
The view first tries to get the specific book record from the model. If this fails the view should raise an Http404 exception to indicate that the book is “not found”. The final step is then, as usual, to call render() with the template name and the book data in the context parameter (as a dictionary).
Another way you could do this if you were not using a generic view would be to call the get_object_or_404() function. This is a shortcut to raise an Http404 exception if the record is not found.
from django.shortcuts import get_object_or_404
def book_detail_view(request, primary_key):
book = get_object_or_404(Book, pk=primary_key)
return render(request, 'catalog/book_detail.html', context={'book': book})
Creating the Detail View template
Create the HTML file /django-locallibrary-tutorial/catalog/templates/catalog/book_detail.html and give it the below content. As discussed above, this is the default template file name expected by the generic class-based detail view (for a model named Book in an application named catalog).
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Title: </h1>
<p><strong>Author:</strong> <a href=""></a></p>
<!-- author detail link not yet defined -->
<p><strong>Summary:</strong> </p>
<p><strong>ISBN:</strong> </p>
<p><strong>Language:</strong> </p>
<p><strong>Genre:</strong> </p>
<div style="margin-left:20px;margin-top:20px">
<h4>Copies</h4>
{\% for copy in book.bookinstance_set.all \%}
<hr />
<p
class="{\% if copy.status == 'a' \%}text-success{\% elif copy.status == 'm' \%}text-danger{\% else \%}text-warning{\% endif \%}">
</p>
{\% if copy.status != 'a' \%}
<p><strong>Due to be returned:</strong> </p>
{\% endif \%}
<p><strong>Imprint:</strong> </p>
<p class="text-muted"><strong>Id:</strong> </p>
{\% endfor \%}
</div>
{\% endblock \%}
Note: The author link in the template above has an empty URL because we’ve not yet created an author detail page to link to. Once the detail page exists we can get its URL with either of these two approaches:
- Use the url template tag to reverse the ‘author-detail’ URL (defined in the URL mapper), passing it the author instance for the book:
<a href="{\% url 'author-detail' book.author.pk \%}"></a>
- Call the author model’s get_absolute_url() method (this performs the same reversing operation):
<a href=""></a>
While both methods effectively do the same thing, get_absolute_url() is preferred because it helps you write more consistent and maintainable code (any changes only need to be done in one place: the author model).
The first interesting thing we haven’t seen before is the function book.bookinstance_set.all(). This method is “automagically” constructed by Django in order to return the set of BookInstance records associated with a particular Book.
{\% for copy in book.bookinstance_set.all \%}
<!-- code to iterate across each copy/instance of a book -->
{\% endfor \%}
This method is needed because you declare a ForeignKey (one-to many) field only in the “many” side of the relationship (the BookInstance). Since you don’t do anything to declare the relationship in the other (“one”) model, it (the Book) doesn’t have any field to get the set of associated records. To overcome this problem, Django constructs an appropriately named “reverse lookup” function that you can use. The name of the function is constructed by lower-casing the model name where the ForeignKey was declared, followed by _set (i.e. so the function created in Book is bookinstance_set()).
Note: Here we use all() to get all records (the default). While you can use the filter() method to get a subset of records in code, you can’t do this directly in templates because you can’t specify arguments to functions.
Beware also that if you don’t define an order (on your class-based view or model), you will also see errors from the development server like this one:
[29/May/2017 18:37:53] "GET /catalog/books/?page=1 HTTP/1.1" 200 1637
/foo/local_library/venv/lib/python3.5/site-packages/django/views/generic/list.py:99: UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list: <QuerySet [<Author: Ortiz, David>, <Author: H. McRaven, William>, <Author: Leigh, Melinda>]>
allow_empty_first_page=allow_empty_first_page, **kwargs)
That happens because the paginator object expects to see some ORDER BY being executed on your underlying database. Without it, it can’t be sure the records being returned are actually in the right order!
This tutorial hasn’t covered Pagination (yet!), but since you can’t use sort_by() and pass a parameter (the same with filter() described above) you will have to choose between three choices:
-
Add a ordering inside a class Meta declaration on your model.
-
Add a queryset attribute in your custom class-based view, specifying an order_by().
-
Adding a get_queryset method to your custom class-based view and also specify the order_by().
If you decide to go with a class Meta for the Author model (probably not as flexible as customizing the class-based view, but easy enough), you will end up with something like this:
class Author(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
date_of_birth = models.DateField(null=True, blank=True)
date_of_death = models.DateField('Died', null=True, blank=True)
def get_absolute_url(self):
return reverse('author-detail', args=[str(self.id)])
def __str__(self):
return f'{self.last_name}, {self.first_name}'
class Meta:
ordering = ['last_name']
Of course, the field doesn’t need to be last_name: it could be any other.
Last but not least, you should sort by an attribute/column that actually has an index (unique or not) on your database to avoid performance issues. Of course, this will not be necessary here (we are probably getting ahead of ourselves with so few books and users), but it is something worth keeping in mind for future projects.
The second interesting (and non-obvious) thing in the template is where we display the status text for each book instance (“available”, “maintenance”, etc.). Astute readers will note that the method BookInstance.get_status_display() that we use to get the status text does not appear elsewhere in the code.
<p class="{\% if copy.status == 'a' \%}text-success{\% elif copy.status == 'm' \%}text-danger{\% else \%}text-warning{\% endif \%}">
</p>
This function is automatically created because BookInstance.status is a choices field. Django automatically creates a method get_FOO_display() for every choices field “Foo” in a model, which can be used to get the current value of the field.
Pagination
If you’ve just got a few records, our book list page will look fine. However, as you get into the tens or hundreds of records the page will take progressively longer to load (and have far too much content to browse sensibly). The solution to this problem is to add pagination to your list views, reducing the number of items displayed on each page.
Django has excellent inbuilt support for pagination. Even better, this is built into the generic class-based list views so you don’t have to do very much to enable it!
Views
Open catalog/views.py, and add the paginate_by line shown below.
class BookListView(generic.ListView):
model = Book
paginate_by = 10
With this addition, as soon as you have more than 10 records the view will start paginating the data it sends to the template. The different pages are accessed using GET parameters — to access page 2 you would use the URL /catalog/books/?page=2.
Templates
Now that the data is paginated, we need to add support to the template to scroll through the results set. Because we might want paginate all list views, we’ll add this to the base template.
Open /django-locallibrary-tutorial/catalog/templates/base_generic.html and find the “content block” (as shown below).
{\% block content \%}{\% endblock \%}
Copy in the following pagination block immediately following the {\% endblock \%}. The code first checks if pagination is enabled on the current page. If so, it adds next and previous links as appropriate (and the current page number).
{\% block pagination \%}
{\% if is_paginated \%}
<div class="pagination">
<span class="page-links">
{\% if page_obj.has_previous \%}
<a href="?page=">previous</a>
{\% endif \%}
<span class="page-current">
Page of .
</span>
{\% if page_obj.has_next \%}
<a href="?page=">next</a>
{\% endif \%}
</span>
</div>
{\% endif \%}
{\% endblock \%}
The page_obj is a Paginator object that will exist if pagination is being used on the current page. It allows you to get all the information about the current page, previous pages, how many pages there are, etc.
We use to get the current page URL for creating the pagination links. This is useful because it is independent of the object that we’re paginating.
Sessions framework
In a “real” library you may wish to provide individual users with a customized experience, based on their previous use of the site, preferences, etc. For example, you could hide warning messages that the user has previously acknowledged next time they visit the site, or store and respect their preferences (such as, the number of search results that they want to be displayed on each page).
The session framework lets you implement this sort of behavior, allowing you to store and retrieve arbitrary data on a per-site-visitor basis.
What are sessions?
All communication between web browsers and servers is via HTTP, which is stateless. The fact that the protocol is stateless means that messages between the client and server are completely independent of each other — there is no notion of “sequence” or behavior based on previous messages. As a result, if you want to have a site that keeps track of the ongoing relationships with a client, you need to implement that yourself.
Sessions are the mechanism used by Django (and most of the Internet) for keeping track of the “state” between the site and a particular browser. Sessions allow you to store arbitrary data per browser, and have this data available to the site whenever the browser connects. Individual data items associated with the session are then referenced by a “key”, which is used both to store and retrieve the data.
Django uses a cookie containing a special session id to identify each browser and its associated session with the site. The actual session data is stored in the site database by default (this is more secure than storing the data in a cookie, where they are more vulnerable to malicious users). You can configure Django to store the session data in other places (cache, files, “secure” cookies), but the default location is a good and relatively secure option.
Enable sessions
Sessions were enabled automatically when we created the skeleton website (in tutorial 2).
«««< HEAD The configuration is set up in the INSTALLED_APPS and MIDDLEWARE sections of the project file (django-locallibrary-tutorial/locallibrary/settings.py), as shown below:
INSTALLED_APPS = [
# …
'django.contrib.sessions',
# …
MIDDLEWARE = [
# …
'django.contrib.sessions.middleware.SessionMiddleware',
# …
Use sessions
You can access the session attribute within a view from the request parameter (an HttpRequest passed in as the first argument to the view). This session attribute represents the specific connection to the current user (or to be more precise, the connection to the current browser, as identified by the session id in the browser’s cookie for this site).
The session attribute is a dictionary-like object that you can read and write as many times as you like in your view, modifying it as wished. You can do all the normal dictionary operations, including clearing all data, testing if a key is present, looping through data, etc. Most of the time though, you’ll just use the standard “dictionary” API to get and set values.
The code fragments below show how you can get, set, and delete some data with the key “my_car”, associated with the current session (browser).
Note: One of the great things about Django is that you don’t need to think about the mechanisms that tie the session to your current request in your view. If we were to use the fragments below in our view, we’d know that the information about my_car is associated only with the browser that sent the current request.
# Get a session value by its key (e.g. 'my_car'), raising a KeyError if the key is not present
my_car = request.session['my_car']
# Get a session value, setting a default if it is not present ('mini')
my_car = request.session.get('my_car', 'mini')
# Set a session value
request.session['my_car'] = 'mini'
# Delete a session value
del request.session['my_car']
Save session data
By default, Django only saves to the session database and sends the session cookie to the client when the session has been modified (assigned) or deleted. If you’re updating some data using its session key as shown in the previous section, then you don’t need to worry about this! For example:
# This is detected as an update to the session, so session data is saved.
request.session['my_car'] = 'mini'
If you’re updating some information within session data, then Django will not recognize that you’ve made a change to the session and save the data (for example, if you were to change “wheels” data inside your “my_car” data, as shown below). In this case you will need to explicitly mark the session as having been modified.
# Session object not directly modified, only data within the session. Session changes not saved!
request.session['my_car']['wheels'] = 'alloy'
# Set session as modified to force data updates/cookie to be saved.
request.session.modified = True
Note: You can change the behavior so the site will update the database/send cookie on every request by adding SESSION_SAVE_EVERY_REQUEST = True into your project settings (django-locallibrary-tutorial/locallibrary/settings.py).
Simple example — getting visit counts
As a simple real-world example we’ll update our library to tell the current user how many times they have visited the LocalLibrary home page.
Open /django-locallibrary-tutorial/catalog/views.py, and add the lines that contain num_visits into index() (as shown below).
def index(request):
# …
num_authors = Author.objects.count() # The 'all()' is implied by default.
# Number of visits to this view, as counted in the session variable.
num_visits = request.session.get('num_visits', 0)
request.session['num_visits'] = num_visits + 1
context = {
'num_books': num_books,
'num_instances': num_instances,
'num_instances_available': num_instances_available,
'num_authors': num_authors,
'num_visits': num_visits,
}
# Render the HTML template index.html with the data in the context variable.
return render(request, 'index.html', context=context)
Here we first get the value of the ‘num_visits’ session key, setting the value to 0 if it has not previously been set. Each time a request is received, we then increment the value and store it back in the session (for the next time the user visits the page). The num_visits variable is then passed to the template in our context variable.
Add the line shown at the bottom of the following block to your main HTML template (/django-locallibrary-tutorial/catalog/templates/index.html) at the bottom of the “Dynamic content” section to display the num_visits context variable.
<h2>Dynamic content</h2>
<p>The library has the following record counts:</p>
<ul>
<li><strong>Books:</strong> </li>
<li><strong>Copies:</strong> </li>
<li><strong>Copies available:</strong> </li>
<li><strong>Authors:</strong> </li>
</ul>
<p>
You have visited this page time.
</p>
Note that we use the Django built-in template tag pluralize to add an “s” when the page has been visited multiple times.
User authentication and permissions
Django provides an authentication and authorization (“permission”) system, built on top of the session framework discussed in the previous tutorial, that allows you to verify user credentials and define what actions each user is allowed to perform. The framework includes built-in models for Users and Groups (a generic way of applying permissions to more than one user at a time), permissions/flags that designate whether a user may perform a task, forms and views for logging in users, and view tools for restricting content.
Note: According to Django the authentication system aims to be very generic, and so does not provide some features provided in other web authentication systems. Solutions for some common problems are available as third-party packages. For example, throttling of login attempts and authentication against third parties (e.g. OAuth).
Enable authentication
The authentication was enabled automatically when we created the skeleton website (in tutorial 2) so you don’t need to do anything more at this point.
Note: The necessary configuration was all done for us when we created the app using the django-admin startproject command. The database tables for users and model permissions were created when we first called python manage.py migrate.
The configuration is set up in the INSTALLED_APPS and MIDDLEWARE sections of the project file (django-locallibrary-tutorial/locallibrary/settings.py), as shown below:
INSTALLED_APPS = [
# …
'django.contrib.auth', # Core authentication framework and its default models.
'django.contrib.contenttypes', # Django content type system (allows permissions to be associated with models).
# …
MIDDLEWARE = [
# …
'django.contrib.sessions.middleware.SessionMiddleware', # Manages sessions across requests
# …
'django.contrib.auth.middleware.AuthenticationMiddleware', # Associates users with requests using sessions.
# …
Createt users and groups
You already created your first user when we looked at the Django admin site in tutorial 4 (this was a superuser, created with the command python manage.py createsuperuser). Our superuser is already authenticated and has all permissions, so we’ll need to create a test user to represent a normal site user. We’ll be using the admin site to create our locallibrary groups and website logins, as it is one of the quickest ways to do so.
Note: You can also create users programmatically as shown below. You would have to do this, for example, if developing an interface to allow “ordinary” users to create their own logins (you shouldn’t give most users access to the admin site).
from django.contrib.auth.models import User
# Create user and save to the database
user = User.objects.create_user('myusername', 'myemail@crazymail.com', 'mypassword')
# Update fields and then save again
user.first_name = 'Tyrone'
user.last_name = 'Citizen'
user.save()
Note however that it is highly recommended to set up a custom user model when starting a project, as you’ll be able to easily customize it in the future if the need arises. If using a custom user model the code to create the same user would look like this:
# Get current user model from settings
from django.contrib.auth import get_user_model
User = get_user_model()
# Create user from model and save to the database
user = User.objects.create_user('myusername', 'myemail@crazymail.com', 'mypassword')
# Update fields and then save again
user.first_name = 'Tyrone'
user.last_name = 'Citizen'
user.save()
Setting up your authentication views
Django provides almost everything you need to create authentication pages to handle login, log out, and password management “out of the box”. This includes a URL mapper, views and forms, but it does not include the templates — we have to create our own!
In this section, we show how to integrate the default system into the LocalLibrary website and create the templates. We’ll put them in the main project URLs.
Note: You don’t have to use any of this code, but it is likely that you’ll want to because it makes things a lot easier. You’ll almost certainly need to change the form handling code if you change your user model, but even so, you would still be able to use the stock view functions.
Note: In this case, we could reasonably put the authentication pages, including the URLs and templates, inside our catalog application. However, if we had multiple applications it would be better to separate out this shared login behavior and have it available across the whole site, so that is what we’ve shown here!
Project URLs
Add the following to the bottom of the project urls.py file (django-locallibrary-tutorial/locallibrary/urls.py) file:
# Add Django site authentication urls (for login, logout, password management)
urlpatterns += [
path('accounts/', include('django.contrib.auth.urls')),
]
Navigate to the http://127.0.0.1:8000/accounts/ URL (note the trailing forward slash!). Django will show an error that it could not find a mapping for this URL, and list all the URLs that it tried. From this you can see the URLs that will work once we have created templates.
Note: Adding the accounts/ path as shown above adds the following URLs, along with names (given in square brackets) that can be used to reverse the URL mappings. You don’t have to implement anything else — the above URL mapping automatically maps the below mentioned URLs.
accounts/ login/ [name='login']
accounts/ logout/ [name='logout']
accounts/ password_change/ [name='password_change']
accounts/ password_change/done/ [name='password_change_done']
accounts/ password_reset/ [name='password_reset']
accounts/ password_reset/done/ [name='password_reset_done']
accounts/ reset/<uidb64>/<token>/ [name='password_reset_confirm']
accounts/ reset/done/ [name='password_reset_complete']
Now try to navigate to the login URL (http://127.0.0.1:8000/accounts/login/). This will fail again, but with an error that tells you that we’re missing the required template (registration/login.html) on the template search path. You’ll see the following lines listed in the yellow section at the top:
Exception Type: TemplateDoesNotExist
Exception Value: registration/login.html
Template directory
The URLs (and implicitly, views) that we just added expect to find their associated templates in a directory /registration/ somewhere in the templates search path.
For this site, we’ll put our HTML pages in the templates/registration/ directory. This directory should be in your project root directory, that is, the same directory as the catalog and locallibrary folders. Please create these folders now.
Note: Your folder structure should now look like the below:
django-locallibrary-tutorial/ # Django top level project folder catalog/ locallibrary/ templates/ registration/ To make the templates directory visible to the template loader we need to add it in the template search path. Open the project settings (/django-locallibrary-tutorial/locallibrary/settings.py).
Then import the os module (add the following line near the top of the file if it isn’t already present).
PYTHON Copy to Clipboard import os # needed by code below Update the TEMPLATES section’s ‘DIRS’ line as shown:
PYTHON Copy to Clipboard # … TEMPLATES = [ { # … ‘DIRS’: [os.path.join(BASE_DIR, ‘templates’)], ‘APP_DIRS’: True, # … ======= The configuration is set up in the INSTALLED_APPS and MIDDLEWARE sections of the project file (django-locallibrary-tutorial/locallibrary/settings.py), as shown below:
INSTALLED_APPS = [
# …
'django.contrib.sessions',
# …
MIDDLEWARE = [
# …
'django.contrib.sessions.middleware.SessionMiddleware',
# …
Use sessions
You can access the session attribute within a view from the request parameter (an HttpRequest passed in as the first argument to the view). This session attribute represents the specific connection to the current user (or to be more precise, the connection to the current browser, as identified by the session id in the browser’s cookie for this site).
The session attribute is a dictionary-like object that you can read and write as many times as you like in your view, modifying it as wished. You can do all the normal dictionary operations, including clearing all data, testing if a key is present, looping through data, etc. Most of the time though, you’ll just use the standard “dictionary” API to get and set values.
The code fragments below show how you can get, set, and delete some data with the key “my_car”, associated with the current session (browser).
Note: One of the great things about Django is that you don’t need to think about the mechanisms that tie the session to your current request in your view. If we were to use the fragments below in our view, we’d know that the information about my_car is associated only with the browser that sent the current request.
# Get a session value by its key (e.g. 'my_car'), raising a KeyError if the key is not present
my_car = request.session['my_car']
# Get a session value, setting a default if it is not present ('mini')
my_car = request.session.get('my_car', 'mini')
# Set a session value
request.session['my_car'] = 'mini'
# Delete a session value
del request.session['my_car']
Save session data
By default, Django only saves to the session database and sends the session cookie to the client when the session has been modified (assigned) or deleted. If you’re updating some data using its session key as shown in the previous section, then you don’t need to worry about this! For example:
# This is detected as an update to the session, so session data is saved.
request.session['my_car'] = 'mini'
If you’re updating some information within session data, then Django will not recognize that you’ve made a change to the session and save the data (for example, if you were to change “wheels” data inside your “my_car” data, as shown below). In this case you will need to explicitly mark the session as having been modified.
# Session object not directly modified, only data within the session. Session changes not saved!
request.session['my_car']['wheels'] = 'alloy'
# Set session as modified to force data updates/cookie to be saved.
request.session.modified = True
Note: You can change the behavior so the site will update the database/send cookie on every request by adding SESSION_SAVE_EVERY_REQUEST = True into your project settings (django-locallibrary-tutorial/locallibrary/settings.py).
Simple example — getting visit counts
As a simple real-world example we’ll update our library to tell the current user how many times they have visited the LocalLibrary home page.
Open /django-locallibrary-tutorial/catalog/views.py, and add the lines that contain num_visits into index() (as shown below).
def index(request):
# …
num_authors = Author.objects.count() # The 'all()' is implied by default.
# Number of visits to this view, as counted in the session variable.
num_visits = request.session.get('num_visits', 0)
request.session['num_visits'] = num_visits + 1
context = {
'num_books': num_books,
'num_instances': num_instances,
'num_instances_available': num_instances_available,
'num_authors': num_authors,
'num_visits': num_visits,
}
# Render the HTML template index.html with the data in the context variable.
return render(request, 'index.html', context=context)
Here we first get the value of the ‘num_visits’ session key, setting the value to 0 if it has not previously been set. Each time a request is received, we then increment the value and store it back in the session (for the next time the user visits the page). The num_visits variable is then passed to the template in our context variable.
Add the line shown at the bottom of the following block to your main HTML template (/django-locallibrary-tutorial/catalog/templates/index.html) at the bottom of the “Dynamic content” section to display the num_visits context variable.
<h2>Dynamic content</h2>
<p>The library has the following record counts:</p>
<ul>
<li><strong>Books:</strong> </li>
<li><strong>Copies:</strong> </li>
<li><strong>Copies available:</strong> </li>
<li><strong>Authors:</strong> </li>
</ul>
<p>
You have visited this page time.
</p>
Note that we use the Django built-in template tag pluralize to add an “s” when the page has been visited multiple times.
User authentication and permissions
Django provides an authentication and authorization (“permission”) system, built on top of the session framework discussed in the previous tutorial, that allows you to verify user credentials and define what actions each user is allowed to perform. The framework includes built-in models for Users and Groups (a generic way of applying permissions to more than one user at a time), permissions/flags that designate whether a user may perform a task, forms and views for logging in users, and view tools for restricting content.
Note: According to Django the authentication system aims to be very generic, and so does not provide some features provided in other web authentication systems. Solutions for some common problems are available as third-party packages. For example, throttling of login attempts and authentication against third parties (e.g. OAuth).
Enable authentication
The authentication was enabled automatically when we created the skeleton website (in tutorial 2) so you don’t need to do anything more at this point.
Note: The necessary configuration was all done for us when we created the app using the django-admin startproject command. The database tables for users and model permissions were created when we first called python manage.py migrate.
The configuration is set up in the INSTALLED_APPS and MIDDLEWARE sections of the project file (django-locallibrary-tutorial/locallibrary/settings.py), as shown below:
INSTALLED_APPS = [
# …
'django.contrib.auth', # Core authentication framework and its default models.
'django.contrib.contenttypes', # Django content type system (allows permissions to be associated with models).
# …
MIDDLEWARE = [
# …
'django.contrib.sessions.middleware.SessionMiddleware', # Manages sessions across requests
# …
'django.contrib.auth.middleware.AuthenticationMiddleware', # Associates users with requests using sessions.
# …
Createt users and groups
You already created your first user when we looked at the Django admin site in tutorial 4 (this was a superuser, created with the command python manage.py createsuperuser). Our superuser is already authenticated and has all permissions, so we’ll need to create a test user to represent a normal site user. We’ll be using the admin site to create our locallibrary groups and website logins, as it is one of the quickest ways to do so.
Note: You can also create users programmatically as shown below. You would have to do this, for example, if developing an interface to allow “ordinary” users to create their own logins (you shouldn’t give most users access to the admin site).
from django.contrib.auth.models import User
# Create user and save to the database
user = User.objects.create_user('myusername', 'myemail@crazymail.com', 'mypassword')
# Update fields and then save again
user.first_name = 'Tyrone'
user.last_name = 'Citizen'
user.save()
Note however that it is highly recommended to set up a custom user model when starting a project, as you’ll be able to easily customize it in the future if the need arises. If using a custom user model the code to create the same user would look like this:
# Get current user model from settings
from django.contrib.auth import get_user_model
User = get_user_model()
# Create user from model and save to the database
user = User.objects.create_user('myusername', 'myemail@crazymail.com', 'mypassword')
# Update fields and then save again
user.first_name = 'Tyrone'
user.last_name = 'Citizen'
user.save()
Setting up your authentication views
Django provides almost everything you need to create authentication pages to handle login, log out, and password management “out of the box”. This includes a URL mapper, views and forms, but it does not include the templates — we have to create our own!
In this section, we show how to integrate the default system into the LocalLibrary website and create the templates. We’ll put them in the main project URLs.
Note: You don’t have to use any of this code, but it is likely that you’ll want to because it makes things a lot easier. You’ll almost certainly need to change the form handling code if you change your user model, but even so, you would still be able to use the stock view functions.
Note: In this case, we could reasonably put the authentication pages, including the URLs and templates, inside our catalog application. However, if we had multiple applications it would be better to separate out this shared login behavior and have it available across the whole site, so that is what we’ve shown here!
Project URLs
Add the following to the bottom of the project urls.py file (django-locallibrary-tutorial/locallibrary/urls.py) file:
# Add Django site authentication urls (for login, logout, password management)
urlpatterns += [
path('accounts/', include('django.contrib.auth.urls')),
]
Navigate to the http://127.0.0.1:8000/accounts/ URL (note the trailing forward slash!). Django will show an error that it could not find a mapping for this URL, and list all the URLs that it tried. From this you can see the URLs that will work once we have created templates.
Note: Adding the accounts/ path as shown above adds the following URLs, along with names (given in square brackets) that can be used to reverse the URL mappings. You don’t have to implement anything else — the above URL mapping automatically maps the below mentioned URLs.
accounts/ login/ [name='login']
accounts/ logout/ [name='logout']
accounts/ password_change/ [name='password_change']
accounts/ password_change/done/ [name='password_change_done']
accounts/ password_reset/ [name='password_reset']
accounts/ password_reset/done/ [name='password_reset_done']
accounts/ reset/<uidb64>/<token>/ [name='password_reset_confirm']
accounts/ reset/done/ [name='password_reset_complete']
Now try to navigate to the login URL (http://127.0.0.1:8000/accounts/login/). This will fail again, but with an error that tells you that we’re missing the required template (registration/login.html) on the template search path. You’ll see the following lines listed in the yellow section at the top:
Exception Type: TemplateDoesNotExist
Exception Value: registration/login.html
Template directory
The URLs (and implicitly, views) that we just added expect to find their associated templates in a directory /registration/ somewhere in the templates search path.
For this site, we’ll put our HTML pages in the templates/registration/ directory. This directory should be in your project root directory, that is, the same directory as the catalog and locallibrary folders. Please create these folders now.
Note: Your folder structure should now look like the below:
django-locallibrary-tutorial/ # Django top level project folder
catalog/
locallibrary/
templates/
registration/
To make the templates directory visible to the template loader we need to add it in the template search path. Open the project settings (/django-locallibrary-tutorial/locallibrary/settings.py).
Then import the os module (add the following line near the top of the file if it isn’t already present).
import os # needed by code below
Update the TEMPLATES section’s ‘DIRS’ line as shown:
# …
TEMPLATES = [
{
# …
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
# …
Login template
Warning: The authentication templates provided in this article are a very basic/slightly modified version of the Django demonstration login templates. You may need to customize them for your own use!
Create a new HTML file called /django-locallibrary-tutorial/templates/registration/login.html and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
{\% if form.errors \%}
<p>Your username and password didn't match. Please try again.</p>
{\% endif \%}
{\% if next \%}
{\% if user.is_authenticated \%}
<p>Your account doesn't have access to this page. To proceed,
please login with an account that has access.</p>
{\% else \%}
<p>Please login to see this page.</p>
{\% endif \%}
{\% endif \%}
<form method="post" action="{\% url 'login' \%}">
{\% csrf_token \%}
<table>
<tr>
<td></td>
<td></td>
</tr>
<tr>
<td></td>
<td></td>
</tr>
</table>
<input type="submit" value="login">
<input type="hidden" name="next" value="">
</form>
{# Assumes you set up the password_reset view in your URLconf #}
<p><a href="{\% url 'password_reset' \%}">Lost password?</a></p>
{\% endblock \%}
This template shares some similarities with the ones we’ve seen before — it extends our base template and overrides the content block. The rest of the code is fairly standard form handling code, which we will discuss in a later tutorial. All you need to know for now is that this will display a form in which you can enter your username and password, and that if you enter invalid values you will be prompted to enter correct values when the page refreshes.
If you log in using valid credentials, you’ll be redirected to another page (by default this will be http://127.0.0.1:8000/accounts/profile/). The problem is that, by default, Django expects that upon logging in you will want to be taken to a profile page, which may or may not be the case. As you haven’t defined this page yet, you’ll get another error!
Open the project settings (/django-locallibrary-tutorial/locallibrary/settings.py) and add the text below to the bottom. Now when you log in you should be redirected to the site homepage by default.
Logout template
If you navigate to the logout URL (http://127.0.0.1:8000/accounts/logout/) then you’ll get an error because Django 5 does not allow logout using GET, only POST. We’ll add a form you can use to logout in a minute, but first we’ll create the page that users are taken to after logging out.
Create and open /django-locallibrary-tutorial/templates/registration/logged_out.html. Copy in the text below:
{\% extends "base_generic.html" \%}
{\% block content \%}
<p>Logged out!</p>
<a href="{\% url 'login'\%}">Click here to login again.</a>
{\% endblock \%}
This template is very simple. It just displays a message informing you that you have been logged out, and provides a link that you can press to go back to the login screen.
Password reset templates
The default password reset system uses email to send the user a reset link. You need to create forms to get the user’s email address, send the email, allow them to enter a new password, and to note when the whole process is complete.
The following templates can be used as a starting point.
Password reset form
This is the form used to get the user’s email address (for sending the password reset email). Create /django-locallibrary-tutorial/templates/registration/password_reset_form.html, and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
<form action="" method="post">
{\% csrf_token \%}
{\% if form.email.errors \%}
{\% endif \%}
<p></p>
<input type="submit" class="btn btn-default btn-lg" value="Reset password">
</form>
{\% endblock \%}
Password reset done
This form is displayed after your email address has been collected. Create /django-locallibrary-tutorial/templates/registration/password_reset_done.html, and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
<p>We've emailed you instructions for setting your password. If they haven't arrived in a few minutes, check your spam folder.</p>
{\% endblock \%}
Password reset email
This template provides the text of the HTML email containing the reset link that we will send to users. Create /django-locallibrary-tutorial/templates/registration/password_reset_email.html, and give it the following contents:
Someone asked for password reset for email . Follow the link below:
://{\% url 'password_reset_confirm' uidb64=uid token=token \%}
Password reset confirm
This page is where you enter your new password after clicking the link in the password reset email. Create /django-locallibrary-tutorial/templates/registration/password_reset_confirm.html, and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
{\% if validlink \%}
<p>Please enter (and confirm) your new password.</p>
<form action="" method="post">
{\% csrf_token \%}
<table>
<tr>
<td>
<label for="id_new_password1">New password:</label></td>
<td></td>
</tr>
<tr>
<td>
<label for="id_new_password2">Confirm password:</label></td>
<td></td>
</tr>
<tr>
<td></td>
<td><input type="submit" value="Change my password"></td>
</tr>
</table>
</form>
{\% else \%}
<h1>Password reset failed</h1>
<p>The password reset link was invalid, possibly because it has already been used. Please request a new password reset.</p>
{\% endif \%}
{\% endblock \%}
Password reset complete
This is the last password-reset template, which is displayed to notify you when the password reset has succeeded. Create /django-locallibrary-tutorial/templates/registration/password_reset_complete.html, and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>The password has been changed!</h1>
<p><a href="{\% url 'login' \%}">log in again?</a></p>
{\% endblock \%}
Testing the new authentication pages
Now that you’ve added the URL configuration and created all these templates, the authentication pages (other than logout) should now just work!
You can test the new authentication pages by first attempting to log in to your superuser account using the URL http://127.0.0.1:8000/accounts/login/. You’ll be able to test the password reset functionality from the link in the login page. Be aware that Django will only send reset emails to addresses (users) that are already stored in its database!
Note that you won’t be able to test account logout yet, because logout requests must be sent as a POST rather than a GET request.
Note: The password reset system requires that your website supports email, which is beyond the scope of this article, so this part won’t work yet. To allow testing, put the following line at the end of your settings.py file. This logs any emails sent to the console (so you can copy the password reset link from the console).
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
Testing against authenticated users
This section looks at what we can do to selectively control content the user sees based on whether they are logged in or not.
Testing in templates
You can get information about the currently logged in user in templates with the template variable (this is added to the template context by default when you set up the project as we did in our skeleton).
Typically you will first test against the template variable to determine whether the user is eligible to see specific content. To demonstrate this, next we’ll update our sidebar to display a “Login” link if the user is logged out, and a “Logout” link if they are logged in.
Open the base template (/django-locallibrary-tutorial/catalog/templates/base_generic.html) and copy the following text into the sidebar block, immediately before the endblock template tag.
<ul class="sidebar-nav">
…
{\% if user.is_authenticated \%}
<li>User: </li>
<li>
<form id="logout-form" method="post" action="{\% url 'logout' \%}">
{\% csrf_token \%}
<button type="submit" class="btn btn-link">Logout</button>
</form>
</li>
{\% else \%}
<li><a href="{\% url 'login' \%}?next=">Login</a></li>
{\% endif \%}
…
</ul>
As you can see, we use if / else / endif template tags to conditionally display text based on whether is true. If the user is authenticated then we know that we have a valid user, so we call to display their name.
We create the login link URL using the url template tag and the name of the login URL configuration. Note also how we have appended ?next= to the end of the URL. What this does is add a URL parameter next containing the address (URL) of the current page, to the end of the linked URL. After the user has successfully logged in, the view will use this “next” value to redirect the user back to the page where they first clicked the login link.
The logout template code is different, because from Django 5 to logout you must POST to the admin:logout URL, using a form with a button. By default this would render as a button, but you can style the button to display as a link. For this example we’re using Bootstrap, so we make the button look like a link by applying class=”btn btn-link”. You also need to append the following styles to /django-locallibrary-tutorial/catalog/static/css/styles.css in order to correctly position the logout link next to all the other sidebar links:
#logout-form {
display: inline;
}
#logout-form button {
padding: 0;
margin: 0;
}
Try it out by clicking the Login/Logout links in the sidebar. You should be taken to the logout/login pages that you defined in the Template directory above.
Testing in views
If you’re using function-based views, the easiest way to restrict access to your functions is to apply the login_required decorator to your view function, as shown below. If the user is logged in then your view code will execute as normal. If the user is not logged in, this will redirect to the login URL defined in the project settings (settings.LOGIN_URL), passing the current absolute path as the next URL parameter. If the user succeeds in logging in then they will be returned back to this page, but this time authenticated.
from django.contrib.auth.decorators import login_required
@login_required
def my_view(request):
# …
Note: You can do the same sort of thing manually by testing on request.user.is_authenticated, but the decorator is much more convenient!
Similarly, the easiest way to restrict access to logged-in users in your class-based views is to derive from LoginRequiredMixin. You need to declare this mixin first in the superclass list, before the main view class.
from django.contrib.auth.mixins import LoginRequiredMixin
class MyView(LoginRequiredMixin, View):
# …
This has exactly the same redirect behavior as the login_required decorator. You can also specify an alternative location to redirect the user to if they are not authenticated (login_url), and a URL parameter name instead of “next” to insert the current absolute path (redirect_field_name).
class MyView(LoginRequiredMixin, View):
login_url = '/login/'
redirect_field_name = 'redirect_to'
Example — listing the current user’s books
Now that we know how to restrict a page to a particular user, let’s create a view of the books that the current user has borrowed.
Unfortunately, we don’t yet have any way for users to borrow books! So before we can create the book list we’ll first extend the BookInstance model to support the concept of borrowing and use the Django Admin application to loan a number of books to our test user.
Models
First, we’re going to have to make it possible for users to have a BookInstance on loan, we already have a status and a due_back date, but we don’t yet have any association between this model and a particular user. We’ll create one using a ForeignKey (one-to-many) field. We also need an easy mechanism to test whether a loaned book is overdue.
Open catalog/models.py, and import the settings from django.conf (add this just below the previous import line at the top of the file, so the settings are available to subsequent code that makes use of them):
from django.conf import settings
Next, add the borrower field to the BookInstance model, setting the user model for the key as the value of the setting AUTH_USER_MODEL. Since we have not overridden the setting with a custom user model this maps to the default User model from django.contrib.auth.models.
borrower = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True, blank=True)
Note: Importing the model in this way reduces the work required if you later discover that you need a custom user model. This tutorial uses the default model, so you could instead import the User model directly with the following lines:
from django.contrib.auth.models import User
borrower = models.ForeignKey(User, on_delete=models.SET_NULL, null=True, blank=True)
null=True sets NULL (versus NOT NULL) on the column in your DB. Blank values for Django field types such as DateTimeField or ForeignKey will be stored as NULL in the DB.
blank determines whether the field will be required in forms. This includes the admin and your custom forms. If blank=True then the field will not be required, whereas if it’s False the field cannot be blank.
The combo of the two is so frequent because typically if you’re going to allow a field to be blank in your form, you’re going to also need your database to allow NULL values for that field. The exception is CharFields and TextFields, which in Django are never saved as NULL. Blank values are stored in the DB as an empty string (‘’).
While we’re here, let’s add a property that we can call from our templates to tell if a particular book instance is overdue. While we could calculate this in the template itself, using a property as shown below will be much more efficient.
Add this somewhere near the top of the file:
from datetime import date
Now add the following property definition to the BookInstance class:
Note: The following code uses Python’s bool() function, which evaluates an object or the resulting object of an expression, and returns True unless the result is “falsy”, in which case it returns False. In Python an object is falsy (evaluates as False) if it is: empty (like [], (), {}), 0, None or if it is False.
@property
def is_overdue(self):
"""Determines if the book is overdue based on due date and current date."""
return bool(self.due_back and date.today() > self.due_back)
Note: We first verify whether due_back is empty before making a comparison. An empty due_back field would cause Django to throw an error instead of showing the page: empty values are not comparable. This is not something we would want our users to experience!
Now that we’ve updated our models, we’ll need to make fresh migrations on the project and then apply those migrations:
py -3 manage.py makemigrations
py -3 manage.py migrate
On loan view
Now we’ll add a view for getting the list of all books that have been loaned to the current user. We’ll use the same generic class-based list view we’re familiar with, but this time we’ll also import and derive from LoginRequiredMixin, so that only a logged in user can call this view. We will also choose to declare a template_name, rather than using the default, because we may end up having a few different lists of BookInstance records, with different views and templates.
Add the following to catalog/views.py:
from django.contrib.auth.mixins import LoginRequiredMixin
class LoanedBooksByUserListView(LoginRequiredMixin,generic.ListView):
"""Generic class-based view listing books on loan to current user."""
model = BookInstance
template_name = 'catalog/bookinstance_list_borrowed_user.html'
paginate_by = 10
def get_queryset(self):
return (
BookInstance.objects.filter(borrower=self.request.user)
.filter(status__exact='o')
.order_by('due_back')
)
In order to restrict our query to just the BookInstance objects for the current user, we re-implement get_queryset() as shown above. Note that “o” is the stored code for “on loan” and we order by the due_back date so that the oldest items are displayed first.
URL conf for on loan books
Now open /catalog/urls.py and add a path() pointing to the above view (you can just copy the text below to the end of the file).
urlpatterns += [
path('mybooks/', views.LoanedBooksByUserListView.as_view(), name='my-borrowed'),
]
Template for on-loan books
Now, all we need to do for this page is add a template. First, create the template file /catalog/templates/catalog/bookinstance_list_borrowed_user.html and give it the following contents:
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Borrowed books</h1>
{\% if bookinstance_list \%}
<ul>
{\% for bookinst in bookinstance_list \%}
<li class="{\% if bookinst.is_overdue \%}text-danger{\% endif \%}">
<a href="{\% url 'book-detail' bookinst.book.pk \%}"></a> ()
</li>
{\% endfor \%}
</ul>
{\% else \%}
<p>There are no books borrowed.</p>
{\% endif \%}
{\% endblock \%}
This template is very similar to those we’ve created previously for the Book and Author objects. The only “new” thing here is that we check the method we added in the model (bookinst.is_overdue) and use it to change the color of overdue items.
When the development server is running, you should now be able to view the list for a logged in user in your browser at http://127.0.0.1:8000/catalog/mybooks/. Try this out with your user logged in and logged out (in the second case, you should be redirected to the login page).
Note: If you build wrong model, and wanna change the model, better delete all the data first and recreate. I deleted the database sqlite file, deleted the pycache folders inside each of the apps, deleted all files inside the migrations folder for each app , except the init.py file, and then ran python manage.py makemigrations and python manage.py migrate. Also note that because you deleted the database you will have to create a new superuser using python manage.py createsuperuser.
Add the list to the sidebar
The very last step is to add a link for this new page into the sidebar. We’ll put this in the same section where we display other information for the logged in user.
Open the base template (/django-locallibrary-tutorial/catalog/templates/base_generic.html) and add the “My Borrowed” line to the sidebar in the position shown below.
<ul class="sidebar-nav">
{\% if user.is_authenticated \%}
<li>User: </li>
<li><a href="{\% url 'my-borrowed' \%}">My Borrowed</a></li>
<li>
<form id="logout-form" method="post" action="{\% url 'admin:logout' \%}">
{\% csrf_token \%}
<button type="submit" class="btn btn-link">Logout</button>
</form>
</li>
{\% else \%}
<li><a href="{\% url 'login' \%}?next=">Login</a></li>
{\% endif \%}
</ul>
Permissions
Permissions are associated with models and define the operations that can be performed on a model instance by a user who has the permission. By default, Django automatically gives add, change, and delete permissions to all models, which allow users with the permissions to perform the associated actions via the admin site. You can define your own permissions to models and grant them to specific users. You can also change the permissions associated with different instances of the same model.
Testing on permissions in views and templates is then very similar to testing on the authentication status (and in fact, testing for a permission also tests for authentication).
Models
Defining permissions is done on the model “class Meta” section, using the permissions field. You can specify as many permissions as you need in a tuple, each permission itself being defined in a nested tuple containing the permission name and permission display value. For example, we might define a permission to allow a user to mark that a book has been returned as shown:
class BookInstance(models.Model):
# …
class Meta:
# …
permissions = (("can_mark_returned", "Set book as returned"),)
We could then assign the permission to a “Librarian” group in the Admin site.
Open the catalog/models.py, and add the permission as shown above. You will need to re-run your migrations (call python3 manage.py makemigrations and python3 manage.py migrate) to update the database appropriately.
Templates
The current user’s permissions are stored in a template variable called . You can check whether the current user has a particular permission using the specific variable name within the associated Django “app” — e.g. will be True if the user has this permission, and False otherwise. We typically test for the permission using the template {\% if \%} tag as shown:
{\% if perms.catalog.can_mark_returned \%}
<!-- We can mark a BookInstance as returned. -->
<!-- Perhaps add code to link to a "book return" view here. -->
{\% endif \%}
Views
Permissions can be tested in function view using the permission_required decorator or in a class-based view using the PermissionRequiredMixin. The pattern are the same as for login authentication, though of course, you might reasonably have to add multiple permissions.
Function view decorator:
from django.contrib.auth.decorators import permission_required
@permission_required('catalog.can_mark_returned')
@permission_required('catalog.can_edit')
def my_view(request):
# …
A permission-required mixin for class-based views.
from django.contrib.auth.mixins import PermissionRequiredMixin
class MyView(PermissionRequiredMixin, View):
permission_required = 'catalog.can_mark_returned'
# Or multiple permissions
permission_required = ('catalog.can_mark_returned', 'catalog.change_book')
# Note that 'catalog.change_book' is permission
# Is created automatically for the book model, along with add_book, and delete_book
ote: There is a small default difference in the behavior above. By default for a logged-in user with a permission violation:
@permission_required redirects to login screen (HTTP Status 302). PermissionRequiredMixin returns 403 (HTTP Status Forbidden). Normally you will want the PermissionRequiredMixin behavior: return 403 if a user is logged in but does not have the correct permission. To do this for a function view use @login_required and @permission_required with raise_exception=True as shown:
from django.contrib.auth.decorators import login_required, permission_required
@login_required
@permission_required('catalog.can_mark_returned', raise_exception=True)
def my_view(request):
# …
Work with forms
HTML Forms
First, a brief overview of HTML Forms. Consider a simple HTML form, with a single text field for entering the name of some “team”, and its associated label:
Simple name field example in HTML form
The form is defined in HTML as a collection of elements inside <form>…</form> tags, containing at least one input element of type=”submit”.
<form action="/team_name_url/" method="post">
<label for="team_name">Enter name: </label>
<input
id="team_name"
type="text"
name="name_field"
value="Default name for team." />
<input type="submit" value="OK" />
</form>
While here we just have one text field for entering the team name, a form may have any number of other input elements and their associated labels. The field’s type attribute defines what sort of widget will be displayed. The name and id of the field are used to identify the field in JavaScript/CSS/HTML, while value defines the initial value for the field when it is first displayed. The matching team label is specified using the label tag (see “Enter name” above), with a for field containing the id value of the associated input.
The submit input will be displayed as a button by default. This can be pressed to upload the data in all the other input elements in the form to the server (in this case, just the team_name field). The form attributes define the HTTP method used to send the data and the destination of the data on the server (action):
-
action: The resource/URL where data is to be sent for processing when the form is submitted. If this is not set (or set to an empty string), then the form will be submitted back to the current page URL.
-
method: The HTTP method used to send the data: post or get.
-
The POST method should always be used if the data is going to result in a change to the server’s database, because it can be made more resistant to cross-site forgery request attacks.
-
The GET method should only be used for forms that don’t change user data (for example, a search form). It is recommended for when you want to be able to bookmark or share the URL.
-
The role of the server is first to render the initial form state — either containing blank fields or pre-populated with initial values. After the user presses the submit button, the server will receive the form data with values from the web browser and must validate the information. If the form contains invalid data, the server should display the form again, this time with user-entered data in “valid” fields and messages to describe the problem for the invalid fields. Once the server gets a request with all valid form data, it can perform an appropriate action (such as: saving the data, returning the result of a search, uploading a file, etc.) and then notify the user.
As you can imagine, creating the HTML, validating the returned data, re-displaying the entered data with error reports if needed, and performing the desired operation on valid data can all take quite a lot of effort to “get right”. Django makes this a lot easier by taking away some of the heavy lifting and repetitive code!
Django form handling process
Django’s form handling uses all of the same techniques that we learned about in previous tutorials (for displaying information about our models): the view gets a request, performs any actions required including reading data from the models, then generates and returns an HTML page (from a template, into which we pass a context containing the data to be displayed). What makes things more complicated is that the server also needs to be able to process data provided by the user, and redisplay the page if there are any errors.
- Display the default form the first time it is requested by the user.
-
The form may contain blank fields if you’re creating a new record, or it may be pre-populated with initial values (for example, if you are changing a record, or have useful default initial values).
-
The form is referred to as unbound at this point, because it isn’t associated with any user-entered data (though it may have initial values).
- Receive data from a submit request and bind it to the form.
- Binding data to the form means that the user-entered data and any errors are available when we need to redisplay the form.
- Clean and validate the data.
-
Cleaning the data performs sanitization of the input fields, such as removing invalid characters that might be used to send malicious content to the server, and converts them into consistent Python types.
-
Validation checks that the values are appropriate for the field (for example, that they are in the right date range, aren’t too short or too long, etc.)
-
If any data is invalid, re-display the form, this time with any user populated values and error messages for the problem fields.
-
If all data is valid, perform required actions (such as save the data, send an email, return the result of a search, upload a file, and so on).
-
Once all actions are complete, redirect the user to another page.
Django provides a number of tools and approaches to help you with the tasks detailed above. The most fundamental is the Form class, which simplifies both generation of form HTML and data cleaning/validation.
Renew-book form using a Form and function view
Next, we’re going to add a page to allow librarians to renew borrowed books. To do this we’ll create a form that allows users to enter a date value. We’ll seed the field with an initial value 3 weeks from the current date (the normal borrowing period), and add some validation to ensure that the librarian can’t enter a date in the past or a date too far in the future. When a valid date has been entered, we’ll write it to the current record’s BookInstance.due_back field.
The example will use a function-based view and a Form class. The following sections explain how forms work, and the changes you need to make to our ongoing LocalLibrary project.
Form
The Form class is the heart of Django’s form handling system. It specifies the fields in the form, their layout, display widgets, labels, initial values, valid values, and (once validated) the error messages associated with invalid fields. The class also provides methods for rendering itself in templates using predefined formats (tables, lists, etc.) or for getting the value of any element (enabling fine-grained manual rendering).
Declaring a Form
The declaration syntax for a Form is very similar to that for declaring a Model, and shares the same field types (and some similar parameters). This makes sense because in both cases we need to ensure that each field handles the right types of data, is constrained to valid data, and has a description for display/documentation.
Form data is stored in an application’s forms.py file, inside the application directory. Create and open the file django-locallibrary-tutorial/catalog/forms.py. To create a Form, we import the forms library, derive from the Form class, and declare the form’s fields. A very basic form class for our library book renewal form is shown below — add this to your new file:
from django import forms
class RenewBookForm(forms.Form):
renewal_date = forms.DateField(help_text="Enter a date between now and 4 weeks (default 3).")
Form fields
In this case, we have a single DateField for entering the renewal date that will render in HTML with a blank value, the default label “Renewal date:”, and some helpful usage text: “Enter a date between now and 4 weeks (default 3 weeks).” As none of the other optional arguments are specified the field will accept dates using the input_formats: YYYY-MM-DD (2024-11-06), MM/DD/YYYY (02/26/2024), MM/DD/YY (10/25/24), and will be rendered using the default widget: DateInput.
The arguments that are common to most fields are listed below (these have sensible default values):
-
required: If True, the field may not be left blank or given a None value. Fields are required by default, so you would set required=False to allow blank values in the form.
-
label: The label to use when rendering the field in HTML. If a label is not specified, Django will create one from the field name by capitalizing the first letter and replacing underscores with spaces (e.g. Renewal date).
-
label_suffix: By default, a colon is displayed after the label (e.g. Renewal date:). This argument allows you to specify a different suffix containing other character(s).
-
initial: The initial value for the field when the form is displayed.
-
widget: The display widget to use.
-
help_text (as seen in the example above): Additional text that can be displayed in forms to explain how to use the field.
-
error_messages: A list of error messages for the field. You can override these with your own messages if needed.
-
validators: A list of functions that will be called on the field when it is validated.
-
localize: Enables the localization of form data input (see link for more information).
-
disabled: The field is displayed but its value cannot be edited if this is True. The default is False.
Validation
Django provides numerous places where you can validate your data. The easiest way to validate a single field is to override the method clean_<fieldname>() for the field you want to check. So for example, we can validate that entered renewal_date values are between now and 4 weeks by implementing clean_renewal_date() as shown below.
Update your forms.py file so it looks like this:
import datetime
from django import forms
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
class RenewBookForm(forms.Form):
renewal_date = forms.DateField(help_text="Enter a date between now and 4 weeks (default 3).")
def clean_renewal_date(self):
data = self.cleaned_data['renewal_date']
# Check if a date is not in the past.
if data < datetime.date.today():
raise ValidationError(_('Invalid date - renewal in past'))
# Check if a date is in the allowed range (+4 weeks from today).
if data > datetime.date.today() + datetime.timedelta(weeks=4):
raise ValidationError(_('Invalid date - renewal more than 4 weeks ahead'))
# Remember to always return the cleaned data.
return data
There are two important things to note. The first is that we get our data using self.cleaned_data['renewal_date'] and that we return this data whether or not we change it at the end of the function. This step gets us the data “cleaned” and sanitized of potentially unsafe input using the default validators, and converted into the correct standard type for the data (in this case a Python datetime.datetime object).
The second point is that if a value falls outside our range we raise a ValidationError, specifying the error text that we want to display in the form if an invalid value is entered. The example above also wraps this text in one of Django’s translation functions, gettext_lazy() (imported as _()), which is good practice if you want to translate your site later.
Note: There are numerous other methods and examples for validating forms in Form and field validation (Django docs). For example, in cases where you have multiple fields that depend on each other, you can override the Form.clean() function and again raise a ValidationError.
URL configuration
Before we create our view, let’s add a URL configuration for the renew-books page. Copy the following configuration to the bottom of django-locallibrary-tutorial/catalog/urls.py:
urlpatterns += [
path('book/<uuid:pk>/renew/', views.renew_book_librarian, name='renew-book-librarian'),
]
The URL configuration will redirect URLs with the format /catalog/book/<bookinstance_id>/renew/ to the function named renew_book_librarian() in views.py, and send the BookInstance id as the parameter named pk. The pattern only matches if pk is a correctly formatted uuid.
Note: We can name our captured URL data “pk” anything we like, because we have complete control over the view function (we’re not using a generic detail view class that expects parameters with a certain name). However, pk short for “primary key”, is a reasonable convention to use!
View
As discussed in the Django form handling process above, the view has to render the default form when it is first called and then either re-render it with error messages if the data is invalid, or process the data and redirect to a new page if the data is valid. In order to perform these different actions, the view has to be able to know whether it is being called for the first time to render the default form, or a subsequent time to validate data.
For forms that use a POST request to submit information to the server, the most common pattern is for the view to test against the POST request type (if request.method == ‘POST’:) to identify form validation requests and GET (using an else condition) to identify the initial form creation request. If you want to submit your data using a GET request, then a typical approach for identifying whether this is the first or subsequent view invocation is to read the form data (e.g. to read a hidden value in the form).
The book renewal process will be writing to our database, so, by convention, we use the POST request approach. The code fragment below shows the (very standard) pattern for this sort of function view.
import datetime
from django.shortcuts import render, get_object_or_404
from django.http import HttpResponseRedirect
from django.urls import reverse
from catalog.forms import RenewBookForm
def renew_book_librarian(request, pk):
book_instance = get_object_or_404(BookInstance, pk=pk)
# If this is a POST request then process the Form data
if request.method == 'POST':
# Create a form instance and populate it with data from the request (binding):
form = RenewBookForm(request.POST)
# Check if the form is valid:
if form.is_valid():
# process the data in form.cleaned_data as required (here we just write it to the model due_back field)
book_instance.due_back = form.cleaned_data['renewal_date']
book_instance.save()
# redirect to a new URL:
return HttpResponseRedirect(reverse('all-borrowed'))
# If this is a GET (or any other method) create the default form.
else:
proposed_renewal_date = datetime.date.today() + datetime.timedelta(weeks=3)
form = RenewBookForm(initial={'renewal_date': proposed_renewal_date})
context = {
'form': form,
'book_instance': book_instance,
}
return render(request, 'catalog/book_renew_librarian.html', context)
First, we import our form (RenewBookForm) and a number of other useful objects/methods used in the body of the view function:
-
get_object_or_404(): Returns a specified object from a model based on its primary key value, and raises an Http404 exception (not found) if the record does not exist.
-
HttpResponseRedirect: This creates a redirect to a specified URL (HTTP status code 302).
-
reverse(): This generates a URL from a URL configuration name and a set of arguments. It is the Python equivalent of the url tag that we’ve been using in our templates.
-
datetime: A Python library for manipulating dates and times.
In the view, we first use the pk argument in get_object_or_404() to get the current BookInstance (if this does not exist, the view will immediately exit and the page will display a “not found” error). If this is not a POST request (handled by the else clause) then we create the default form passing in an initial value for the renewal_date field, 3 weeks from the current date.
After creating the form, we call render() to create the HTML page, specifying the template and a context that contains our form. In this case, the context also contains our BookInstance, which we’ll use in the template to provide information about the book we’re renewing.
However, if this is a POST request, then we create our form object and populate it with data from the request. This process is called “binding” and allows us to validate the form.
We then check if the form is valid, which runs all the validation code on all of the fields — including both the generic code to check that our date field is actually a valid date and our specific form’s clean_renewal_date() function to check the date is in the right range.
If the form is not valid we call render() again, but this time the form value passed in the context will include error messages.
If the form is valid, then we can start to use the data, accessing it through the form.cleaned_data attribute (e.g. data = form.cleaned_data[‘renewal_date’]). Here, we just save the data into the due_back value of the associated BookInstance object.
Warning: While you can also access the form data directly through the request (for example, request.POST[‘renewal_date’] or request.GET[‘renewal_date’] if using a GET request), this is NOT recommended. The cleaned data is sanitized, validated, and converted into Python-friendly types.
The final step in the form-handling part of the view is to redirect to another page, usually a “success” page. In this case, we use HttpResponseRedirect and reverse() to redirect to the view named ‘all-borrowed’.
That’s everything needed for the form handling itself, but we still need to restrict access to the view to just logged-in librarians who have permission to renew books. We use @login_required to require that the user is logged in, and the @permission_required function decorator with our existing can_mark_returned permission to allow access (decorators are processed in order). Note that we probably should have created a new permission setting in BookInstance (“can_renew”), but we will reuse the existing one to keep the example simple.
The final view is therefore as shown below. Please copy this into the bottom of django-locallibrary-tutorial/catalog/views.py.
import datetime
from django.contrib.auth.decorators import login_required, permission_required
from django.shortcuts import get_object_or_404
from django.http import HttpResponseRedirect
from django.urls import reverse
from catalog.forms import RenewBookForm
@login_required
@permission_required('catalog.can_mark_returned', raise_exception=True)
def renew_book_librarian(request, pk):
"""View function for renewing a specific BookInstance by librarian."""
book_instance = get_object_or_404(BookInstance, pk=pk)
# If this is a POST request then process the Form data
if request.method == 'POST':
# Create a form instance and populate it with data from the request (binding):
form = RenewBookForm(request.POST)
# Check if the form is valid:
if form.is_valid():
# process the data in form.cleaned_data as required (here we just write it to the model due_back field)
book_instance.due_back = form.cleaned_data['renewal_date']
book_instance.save()
# redirect to a new URL:
return HttpResponseRedirect(reverse('all-borrowed'))
# If this is a GET (or any other method) create the default form.
else:
proposed_renewal_date = datetime.date.today() + datetime.timedelta(weeks=3)
form = RenewBookForm(initial={'renewal_date': proposed_renewal_date})
context = {
'form': form,
'book_instance': book_instance,
}
return render(request, 'catalog/book_renew_librarian.html', context)
The template
Create the template referenced in the view (/catalog/templates/catalog/book_renew_librarian.html) and copy the code below into it:
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Renew: </h1>
<p>Borrower: </p>
<p {\% if book_instance.is_overdue \%} class="text-danger"{\% endif \%} >Due date: </p>
<form action="" method="post">
{\% csrf_token \%}
<table>
</table>
<input type="submit" value="Submit">
</form>
{\% endblock \%}
Most of this will be completely familiar from previous tutorials.
We extend the base template and then redefine the content block. We are able to reference (and its variables) because it was passed into the context object in the render() function, and we use these to list the book title, borrower, and the original due date.
The form code is relatively simple. First, we declare the form tags, specifying where the form is to be submitted (action) and the method for submitting the data (in this case an “HTTP POST”) — if you recall the HTML Forms overview at the top of the page, an empty action as shown, means that the form data will be posted back to the current URL of the page (which is what we want). Inside the tags, we define the submit input, which a user can press to submit the data. The {\% csrf_token \%} added just inside the form tags is part of Django’s cross-site forgery protection.
Note: Add the {\% csrf_token \%} to every Django template you create that uses POST to submit data. This will reduce the chance of forms being hijacked by malicious users.
All that’s left is the template variable, which we passed to the template in the context dictionary. Perhaps unsurprisingly, when used as shown this provides the default rendering of all the form fields, including their labels, widgets, and help text — the rendering is as shown below:
<tr>
<th><label for="id_renewal_date">Renewal date:</label></th>
<td>
<input
id="id_renewal_date"
name="renewal_date"
type="text"
value="2023-11-08"
required />
<br />
<span class="helptext">
Enter date between now and 4 weeks (default 3 weeks).
</span>
</td>
</tr>
Note: It is perhaps not obvious because we only have one field, but, by default, every field is defined in its own table row. This same rendering is provided if you reference the template variable .
If you were to enter an invalid date, you’d additionally get a list of the errors rendered on the page (see errorlist below).
<tr>
<th><label for="id_renewal_date">Renewal date:</label></th>
<td>
<ul class="errorlist">
<li>Invalid date - renewal in past</li>
</ul>
<input
id="id_renewal_date"
name="renewal_date"
type="text"
value="2023-11-08"
required />
<br />
<span class="helptext">
Enter date between now and 4 weeks (default 3 weeks).
</span>
</td>
</tr>
Other ways of using form template variable
Using as shown above, each field is rendered as a table row. You can also render each field as a list item (using ) or as a paragraph (using ).
It is also possible to have complete control over the rendering of each part of the form, by indexing its properties using dot notation. So, for example, we can access a number of separate items for our renewal_date field:
-
: The whole field.
-
: The list of errors.
-
: The id of the label.
-
: The field help text.
Testing the page
If you accepted the “challenge” in Django Tutorial Part 8: User authentication and permissions you’ll have a view showing all books on loan in the library, which is only visible to library staff. The view might look similar to this:
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>All Borrowed Books</h1>
{\% if bookinstance_list \%}
<ul>
{\% for bookinst in bookinstance_list \%}
<li class="{\% if bookinst.is_overdue \%}text-danger{\% endif \%}">
<a href="{\% url 'book-detail' bookinst.book.pk \%}"></a> () {\% if user.is_staff \%}- {\% endif \%}
</li>
{\% endfor \%}
</ul>
{\% else \%}
<p>There are no books borrowed.</p>
{\% endif \%}
{\% endblock \%}
We can add a link to the book renew page next to each item by appending the following template code to the list item text above. Note that this template code can only run inside the {\% for \%} loop, because that is where the bookinst value is defined.
{\% if perms.catalog.can_mark_returned \%}- <a href="{\% url 'renew-book-librarian' bookinst.id \%}">Renew</a>{\% endif \%}
ModelForms
Creating a Form class using the approach described above is very flexible, allowing you to create whatever sort of form page you like and associate it with any model or models.
However, if you just need a form to map the fields of a single model then your model will already define most of the information that you need in your form: fields, labels, help text and so on. Rather than recreating the model definitions in your form, it is easier to use the ModelForm helper class to create the form from your model. This ModelForm can then be used within your views in exactly the same way as an ordinary Form.
A basic ModelForm containing the same field as our original RenewBookForm is shown below. All you need to do to create the form is add class Meta with the associated model (BookInstance) and a list of the model fields to include in the form.
from django.forms import ModelForm
from catalog.models import BookInstance
class RenewBookModelForm(ModelForm):
class Meta:
model = BookInstance
fields = ['due_back']
Note: You can also include all fields in the form using
fields = '__all__', or you can use exclude (instead of fields) to specify the fields not to include from the model.
Neither approach is recommended because new fields added to the model are then automatically included in the form (without the developer necessarily considering possible security implications).
Note: This might not look all that much simpler than just using a Form (and it isn’t in this case, because we just have one field). However, if you have a lot of fields, it can considerably reduce the amount of code required!
The rest of the information comes from the model field definitions (e.g. labels, widgets, help text, error messages). If these aren’t quite right, then we can override them in our class Meta, specifying a dictionary containing the field to change and its new value. For example, in this form, we might want a label for our field of “Renewal date” (rather than the default based on the field name: Due Back), and we also want our help text to be specific to this use case. The Meta below shows you how to override these fields, and you can similarly set widgets and error_messages if the defaults aren’t sufficient.
class Meta:
model = BookInstance
fields = ['due_back']
labels = {'due_back': _('New renewal date')}
help_texts = {'due_back': _('Enter a date between now and 4 weeks (default 3).')}
To add validation you can use the same approach as for a normal Form — you define a function named clean_<field_name>() and raise ValidationError exceptions for invalid values. The only difference with respect to our original form is that the model field is named due_back and not “renewal_date”. This change is necessary since the corresponding field in BookInstance is called due_back.
from django.forms import ModelForm
from catalog.models import BookInstance
class RenewBookModelForm(ModelForm):
def clean_due_back(self):
data = self.cleaned_data['due_back']
# Check if a date is not in the past.
if data < datetime.date.today():
raise ValidationError(_('Invalid date - renewal in past'))
# Check if a date is in the allowed range (+4 weeks from today).
if data > datetime.date.today() + datetime.timedelta(weeks=4):
raise ValidationError(_('Invalid date - renewal more than 4 weeks ahead'))
# Remember to always return the cleaned data.
return data
class Meta:
model = BookInstance
fields = ['due_back']
labels = {'due_back': _('Renewal date')}
help_texts = {'due_back': _('Enter a date between now and 4 weeks (default 3).')}
The class RenewBookModelForm above is now functionally equivalent to our original RenewBookForm. You could import and use it wherever you currently use RenewBookForm as long as you also update the corresponding form variable name from renewal_date to due_back as in the second form declaration: RenewBookModelForm(initial={‘due_back’: proposed_renewal_date}).
Generic editing views
The form handling algorithm we used in our function view example above represents an extremely common pattern in form editing views. Django abstracts much of this “boilerplate” for you, by creating generic editing views for creating, editing, and deleting views based on models. Not only do these handle the “view” behavior, but they automatically create the form class (a ModelForm) for you from the model.
Note: In addition to the editing views described here, there is also a FormView class, which lies somewhere between our function view and the other generic views in terms of “flexibility” vs. “coding effort”. Using FormView, you still need to create your Form, but you don’t have to implement all of the standard form-handling patterns. Instead, you just have to provide an implementation of the function that will be called once the submission is known to be valid.
In this section, we’re going to use generic editing views to create pages to add functionality to create, edit, and delete Author records from our library — effectively providing a basic reimplementation of parts of the Admin site (this could be useful if you need to offer admin functionality in a more flexible way than can be provided by the admin site).
Views
Open the views file (django-locallibrary-tutorial/catalog/views.py) and append the following code block to the bottom of it:
from django.views.generic.edit import CreateView, UpdateView, DeleteView
from django.urls import reverse_lazy
from .models import Author
class AuthorCreate(PermissionRequiredMixin, CreateView):
model = Author
fields = ['first_name', 'last_name', 'date_of_birth', 'date_of_death']
initial = {'date_of_death': '11/11/2023'}
permission_required = 'catalog.add_author'
class AuthorUpdate(PermissionRequiredMixin, UpdateView):
model = Author
# Not recommended (potential security issue if more fields added)
fields = '__all__'
permission_required = 'catalog.change_author'
class AuthorDelete(PermissionRequiredMixin, DeleteView):
model = Author
success_url = reverse_lazy('authors')
permission_required = 'catalog.delete_author'
def form_valid(self, form):
try:
self.object.delete()
return HttpResponseRedirect(self.success_url)
except Exception as e:
return HttpResponseRedirect(
reverse("author-delete", kwargs={"pk": self.object.pk})
)
As you can see, to create, update, or delete the views you need to derive from CreateView, UpdateView, and DeleteView (respectively) and then define the associated model. We also restrict calling these views to only logged in users with the add_author, change_author, and delete_author permissions, respectively.
For the “create” and “update” cases you also need to specify the fields to display in the form (using the same syntax as for ModelForm). In this case, we show how to list them individually and the syntax to list “all” fields. You can also specify initial values for each of the fields using a dictionary of field_name/value pairs (here we arbitrarily set the date of death for demonstration purposes — you might want to remove that). By default, these views will redirect on success to a page displaying the newly created/edited model item, which in our case will be the author detail view we created in a previous tutorial. You can specify an alternative redirect location by explicitly declaring parameter success_url.
The AuthorDelete class doesn’t need to display any of the fields, so these don’t need to be specified. We so set a success_url (as shown above), because there is no obvious default URL for Django to navigate to after successfully deleting the Author. Above we use the reverse_lazy() function to redirect to our author list after an author has been deleted — reverse_lazy() is a lazily executed version of reverse(), used here because we’re providing a URL to a class-based view attribute.
If deletion of authors should always succeed that would be it. Unfortunately deleting an Author will cause an exception if the author has an associated book, because our Book model specifies on_delete=models.RESTRICT for the author ForeignKey field. To handle this case the view overrides the form_valid() method so that if deleting the Author succeeds it redirects to the success_url, but if not, it just redirects back to the same form. We’ll update the template below to make clear that you can’t delete an Author instance that is used in any Book.
URL configurations
Open your URL configuration file (django-locallibrary-tutorial/catalog/urls.py) and add the following configuration to the bottom of the file:
urlpatterns += [
path('author/create/', views.AuthorCreate.as_view(), name='author-create'),
path('author/<int:pk>/update/', views.AuthorUpdate.as_view(), name='author-update'),
path('author/<int:pk>/delete/', views.AuthorDelete.as_view(), name='author-delete'),
]
There is nothing particularly new here! You can see that the views are classes, and must hence be called via .as_view(), and you should be able to recognize the URL patterns in each case. We must use pk as the name for our captured primary key value, as this is the parameter name expected by the view classes.
Templates
The “create” and “update” views use the same template by default, which will be named after your model: model_name_form.html (you can change the suffix to something other than _form using the template_name_suffix field in your view, for example, template_name_suffix = ‘_other_suffix’)
Create the template file django-locallibrary-tutorial/catalog/templates/catalog/author_form.html and copy the text below.
{\% extends "base_generic.html" \%}
{\% block content \%}
<form action="" method="post">
{\% csrf_token \%}
<table>
</table>
<input type="submit" value="Submit" />
</form>
{\% endblock \%}
This is similar to our previous forms and renders the fields using a table. Note also how again we declare the {\% csrf_token \%} to ensure that our forms are resistant to CSRF attacks.
The “delete” view expects to find a template named with the format _model_name_confirm_delete.html (again, you can change the suffix using template_name_suffix in your view). Create the template file django-locallibrary-tutorial/catalog/templates/catalog/author_confirm_delete.html and copy the text below.
{\% extends "base_generic.html" \%}
{\% block content \%}
<h1>Delete Author: </h1>
{\% if author.book_set.all \%}
<p>You can't delete this author until all their books have been deleted:</p>
<ul>
{\% for book in author.book_set.all \%}
<li><a href="{\% url 'book-detail' book.pk \%}"></a> ()</li>
{\% endfor \%}
</ul>
{\% else \%}
<p>Are you sure you want to delete the author?</p>
<form action="" method="POST">
{\% csrf_token \%}
<input type="submit" action="" value="Yes, delete.">
</form>
{\% endif \%}
{\% endblock \%}
The template should be familiar. It first checks if the author is used in any books, and if so displays the list of books that must be deleted before the author record can be deleted. If not, it displays a form asking the user to confirm they want to delete the author record.
The final step is to hook the pages into the sidebar. First we’ll add a link for creating the author into the base template, so that it is visible in all pages for logged in users who are considered “staff” and who have permission to create authors (catalog.add_author). Open /django-locallibrary-tutorial/catalog/templates/base_generic.html and add the lines that allow users with the permission to create the author (in the same block as the link that shows “All Borrowed” books). Remember to reference the URL using it’s name ‘author-create’ as shown below.
{\% if user.is_staff \%}
<hr>
<ul class="sidebar-nav">
<li>Staff</li>
<li><a href="{\% url 'all-borrowed' \%}">All borrowed</a></li>
{\% if perms.catalog.add_author \%}
<li><a href="{\% url 'author-create' \%}">Create author</a></li>
{\% endif \%}
</ul>
{\% endif \%}
We’ll add the links to update and delete authors to the author detail page. Open catalog/templates/catalog/author_detail.html and append the following code:
{\% block sidebar \%}
{\% if perms.catalog.change_author or perms.catalog.delete_author \%}
<hr>
<ul class="sidebar-nav">
{\% if perms.catalog.change_author \%}
<li><a href="{\% url 'author-update' author.id \%}">Update author</a></li>
{\% endif \%}
{\% if not author.book_set.all and perms.catalog.delete_author \%}
<li><a href="{\% url 'author-delete' author.id \%}">Delete author</a></li>
{\% endif \%}
</ul>
{\% endif \%}
{\% endblock \%}
This block overrides the sidebar block in the base template and then pulls in the original content using . It then appends links to update or delete the author, but when the user has the correct permissions and the author record isn’t associated with any books.
Test a Django web application
Types of testing
There are numerous types, levels, and classifications of tests and testing approaches. The most important automated tests are:
-
Unit tests: Verify functional behavior of individual components, often to class and function level.
-
Regression tests: Tests that reproduce historic bugs. Each test is initially run to verify that the bug has been fixed, and then re-run to ensure that it has not been reintroduced following later changes to the code.
-
Integration tests: Verify how groupings of components work when used together. Integration tests are aware of the required interactions between components, but not necessarily of the internal operations of each component. They may cover simple groupings of components through to the whole website.
What does Django provide for testing?
Testing a website is a complex task, because it is made of several layers of logic – from HTTP-level request handling, to model queries, to form validation and processing, and template rendering.
Django provides a test framework with a small hierarchy of classes that build on the Python standard unittest library. Despite the name, this test framework is suitable for both unit and integration tests. The Django framework adds API methods and tools to help test web and Django-specific behavior. These allow you to simulate requests, insert test data, and inspect your application’s output. Django also provides an API (LiveServerTestCase) and tools for using different testing frameworks, for example you can integrate with the popular Selenium framework to simulate a user interacting with a live browser.
To write a test you derive from any of the Django (or unittest) test base classes (SimpleTestCase, TransactionTestCase, TestCase, LiveServerTestCase) and then write separate methods to check that specific functionality works as expected (tests use “assert” methods to test that expressions result in True or False values, or that two values are equal, etc.) When you start a test run, the framework executes the chosen test methods in your derived classes. The test methods are run independently, with common setup and/or tear-down behavior defined in the class, as shown below.
class YourTestClass(TestCase):
def setUp(self):
# Setup run before every test method.
pass
def tearDown(self):
# Clean up run after every test method.
pass
def test_something_that_will_pass(self):
self.assertFalse(False)
def test_something_that_will_fail(self):
self.assertTrue(False)
The best base class for most tests is django.test.TestCase. This test class creates a clean database before its tests are run, and runs every test function in its own transaction. The class also owns a test Client that you can use to simulate a user interacting with the code at the view level. In the following sections we’re going to concentrate on unit tests, created using this TestCase base class.
Note: The django.test.TestCase class is very convenient, but may result in some tests being slower than they need to be (not every test will need to set up its own database or simulate the view interaction). Once you’re familiar with what you can do with this class, you may want to replace some of your tests with the available simpler test classes.
What should you test?
You should test all aspects of your own code, but not any libraries or functionality provided as part of Python or Django.
So for example, consider the Author model defined below. You don’t need to explicitly test that first_name and last_name have been stored properly as CharField in the database because that is something defined by Django (though of course in practice you will inevitably test this functionality during development). Nor do you need to test that the date_of_birth has been validated to be a date field, because that is again something implemented in Django.
However you should check the text used for the labels (First name, Last name, Date of birth, Died), and the size of the field allocated for the text (100 chars), because these are part of your design and something that could be broken/changed in future.
Similarly, you should check that the custom methods get_absolute_url() and __str__() behave as required because they are your code/business logic. In the case of get_absolute_url() you can trust that the Django reverse() method has been implemented properly, so what you’re testing is that the associated view has actually been defined.
Test structure overview
Before we go into the detail of “what to test”, let’s first briefly look at where and how tests are defined.
Django uses the unittest module’s built-in test discovery, which will discover tests under the current working directory in any file named with the pattern test*.py. Provided you name the files appropriately, you can use any structure you like. We recommend that you create a module for your test code, and have separate files for models, views, forms, and any other types of code you need to test. For example:
catalog/
/tests/
__init__.py
test_models.py
test_forms.py
test_views.py
Create a file structure as shown above in your LocalLibrary project. The __init__.py should be an empty file (this tells Python that the directory is a package). You can create the three test files by copying and renaming the skeleton test file /catalog/tests.py.
Note: The skeleton test file /catalog/tests.py was created automatically when we built the Django skeleton website. It is perfectly “legal” to put all your tests inside it, but if you test properly, you’ll quickly end up with a very large and unmanageable test file.
Delete the skeleton file as we won’t need it.
Open /catalog/tests/test_models.py. The file should import django.test.TestCase, as shown:
from django.test import TestCase
Often you will add a test class for each model/view/form you want to test, with individual methods for testing specific functionality. In other cases you may wish to have a separate class for testing a specific use case, with individual test functions that test aspects of that use-case (for example, a class to test that a model field is properly validated, with functions to test each of the possible failure cases). Again, the structure is very much up to you, but it is best if you are consistent.
Add the test class below to the bottom of the file. The class demonstrates how to construct a test case class by deriving from TestCase.
class YourTestClass(TestCase):
@classmethod
def setUpTestData(cls):
print("setUpTestData: Run once to set up non-modified data for all class methods.")
pass
def setUp(self):
print("setUp: Run once for every test method to set up clean data.")
pass
def test_false_is_false(self):
print("Method: test_false_is_false.")
self.assertFalse(False)
def test_false_is_true(self):
print("Method: test_false_is_true.")
self.assertTrue(False)
def test_one_plus_one_equals_two(self):
print("Method: test_one_plus_one_equals_two.")
self.assertEqual(1 + 1, 2)
The new class defines two methods that you can use for pre-test configuration (for example, to create any models or other objects you will need for the test):
-
setUpTestData() is called once at the beginning of the test run for class-level setup. You’d use this to create objects that aren’t going to be modified or changed in any of the test methods.
-
setUp() is called before every test function to set up any objects that may be modified by the test (every test function will get a “fresh” version of these objects).
Note: The test classes also have a tearDown() method which we haven’t used. This method isn’t particularly useful for database tests, since the TestCase base class takes care of database teardown for you.
Below those we have a number of test methods, which use Assert functions to test whether conditions are true, false or equal (AssertTrue, AssertFalse, AssertEqual). If the condition does not evaluate as expected then the test will fail and report the error to your console.
The AssertTrue, AssertFalse, AssertEqual are standard assertions provided by unittest. There are other standard assertions in the framework, and also Django-specific assertions to test if a view redirects (assertRedirects), to test if a particular template has been used (assertTemplateUsed), etc.
How to run the tests
The easiest way to run all the tests is to use the command:
py -3 manage.py test
This will discover all files named with the pattern test*.py under the current directory and run all tests defined using appropriate base classes (here we have a number of test files, but only /catalog/tests/test_models.py currently contains any tests.) By default the tests will individually report only on test failures, followed by a test summary.
Note: If you get errors similar to: ValueError: Missing staticfiles manifest entry… this may be because testing does not run collectstatic by default, and your app is using a storage class that requires it (see manifest_strict for more information). There are a number of ways you can overcome this problem - the easiest is to run collectstatic before running the tests:
python3 manage.py collectstatic
Show more test information
If you want to get more information about the test run you can change the verbosity. For example, to list the test successes as well as failures (and a whole bunch of information about how the testing database is set up) you can set the verbosity to “2” as shown:
python3 manage.py test --verbosity 2
The allowed verbosity levels are 0, 1, 2, and 3, with the default being “1”.
Speed things up
If your tests are independent, on a multiprocessor machine you can significantly speed them up by running them in parallel. The use of –parallel auto below runs one test process per available core. The auto is optional, and you can also specify a particular number of cores to use.
python3 manage.py test --parallel auto
Run specific tests
If you want to run a subset of your tests you can do so by specifying the full dot path to the package(s), module, TestCase subclass or method:
# Run the specified module
python3 manage.py test catalog.tests
# Run the specified module
python3 manage.py test catalog.tests.test_models
# Run the specified class
python3 manage.py test catalog.tests.test_models.YourTestClass
# Run the specified method
python3 manage.py test catalog.tests.test_models.YourTestClass.test_one_plus_one_equals_two
Other test runner options
The test runner provides many other options, including the ability to shuffle tests (–shuffle), run them in debug mode (–debug-mode), and use the Python logger to capture the results.
LocalLibrary tests
Now we know how to run our tests and what sort of things we need to test, let’s look at some practical examples.
Models As discussed above, we should test anything that is part of our design or that is defined by code that we have written, but not libraries/code that is already tested by Django or the Python development team.
For example, consider the Author model below. Here we should test the labels for all the fields, because even though we haven’t explicitly specified most of them, we have a design that says what these values should be. If we don’t test the values, then we don’t know that the field labels have their intended values. Similarly while we trust that Django will create a field of the specified length, it is worthwhile to specify a test for this length to ensure that it was implemented as planned.
Open our /catalog/tests/test_models.py, and replace any existing code with the following test code for the Author model.
Here you’ll see that we first import TestCase and derive our test class (AuthorModelTest) from it, using a descriptive name so we can easily identify any failing tests in the test output. We then call setUpTestData() to create an author object that we will use but not modify in any of the tests.
from django.test import TestCase
from catalog.models import Author
class AuthorModelTest(TestCase):
@classmethod
def setUpTestData(cls):
# Set up non-modified objects used by all test methods
Author.objects.create(first_name='Big', last_name='Bob')
def test_first_name_label(self):
author = Author.objects.get(id=1)
field_label = author._meta.get_field('first_name').verbose_name
self.assertEqual(field_label, 'first name')
def test_date_of_death_label(self):
author = Author.objects.get(id=1)
field_label = author._meta.get_field('date_of_death').verbose_name
self.assertEqual(field_label, 'died')
def test_first_name_max_length(self):
author = Author.objects.get(id=1)
max_length = author._meta.get_field('first_name').max_length
self.assertEqual(max_length, 100)
def test_object_name_is_last_name_comma_first_name(self):
author = Author.objects.get(id=1)
expected_object_name = f'{author.last_name}, {author.first_name}'
self.assertEqual(str(author), expected_object_name)
def test_get_absolute_url(self):
author = Author.objects.get(id=1)
# This will also fail if the urlconf is not defined.
self.assertEqual(author.get_absolute_url(), '/catalog/author/1')
The interesting things to note are:
-
We can’t get the verbose_name directly using author.first_name.verbose_name, because author.first_name is a string (not a handle to the first_name object that we can use to access its properties). Instead we need to use the author’s _meta attribute to get an instance of the field and use that to query for the additional information.
-
We chose to use assertEqual(field_label,’first name’) rather than assertTrue(field_label == ‘first name’). The reason for this is that if the test fails the output for the former tells you what the label actually was, which makes debugging the problem just a little easier.
Forms
The philosophy for testing your forms is the same as for testing your models; you need to test anything that you’ve coded or your design specifies, but not the behavior of the underlying framework and other third party libraries.
Generally this means that you should test that the forms have the fields that you want, and that these are displayed with appropriate labels and help text. You don’t need to verify that Django validates the field type correctly (unless you created your own custom field and validation) — i.e. you don’t need to test that an email field only accepts emails. However you would need to test any additional validation that you expect to be performed on the fields and any messages that your code will generate for errors.
Consider our form for renewing books. This has just one field for the renewal date, which will have a label and help text that we will need to verify.
Open our /catalog/tests/test_forms.py file and replace any existing code with the following test code for the RenewBookForm form. We start by importing our form and some Python and Django libraries to help test time-related functionality. We then declare our form test class in the same way as we did for models, using a descriptive name for our TestCase-derived test class.
import datetime
from django.test import TestCase
from django.utils import timezone
from catalog.forms import RenewBookForm
class RenewBookFormTest(TestCase):
def test_renew_form_date_field_label(self):
form = RenewBookForm()
self.assertTrue(form.fields['renewal_date'].label is None or form.fields['renewal_date'].label == 'renewal date')
def test_renew_form_date_field_help_text(self):
form = RenewBookForm()
self.assertEqual(form.fields['renewal_date'].help_text, 'Enter a date between now and 4 weeks (default 3).')
def test_renew_form_date_in_past(self):
date = datetime.date.today() - datetime.timedelta(days=1)
form = RenewBookForm(data={'renewal_date': date})
self.assertFalse(form.is_valid())
def test_renew_form_date_too_far_in_future(self):
date = datetime.date.today() + datetime.timedelta(weeks=4) + datetime.timedelta(days=1)
form = RenewBookForm(data={'renewal_date': date})
self.assertFalse(form.is_valid())
def test_renew_form_date_today(self):
date = datetime.date.today()
form = RenewBookForm(data={'renewal_date': date})
self.assertTrue(form.is_valid())
def test_renew_form_date_max(self):
date = timezone.localtime() + datetime.timedelta(weeks=4)
form = RenewBookForm(data={'renewal_date': date})
self.assertTrue(form.is_valid())
The first two functions test that the field’s label and help_text are as expected. We have to access the field using the fields dictionary (e.g. form.fields['renewal_date']). Note here that we also have to test whether the label value is None, because even though Django will render the correct label it returns None if the value is not explicitly set.
The rest of the functions test that the form is valid for renewal dates just inside the acceptable range and invalid for values outside the range. Note how we construct test date values around our current date (datetime.date.today()) using datetime.timedelta() (in this case specifying a number of days or weeks). We then just create the form, passing in our data, and test if it is valid.
Note: Here we don’t actually use the database or test client. Consider modifying these tests to use SimpleTestCase.
We also need to validate that the correct errors are raised if the form is invalid, however this is usually done as part of view processing, so we’ll take care of that in the next section.
Warning: If you use the ModelForm class RenewBookModelForm(forms.ModelForm) instead of class RenewBookForm(forms.Form), then the form field name would be ‘due_back’ instead of ‘renewal_date’.
That’s all for forms; we do have some others, but they are automatically created by our generic class-based editing views, and should be tested there! Run the tests and confirm that our code still passes!
Views
To validate our view behavior we use the Django test Client. This class acts like a dummy web browser that we can use to simulate GET and POST requests on a URL and observe the response. We can see almost everything about the response, from low-level HTTP (result headers and status codes) through to the template we’re using to render the HTML and the context data we’re passing to it. We can also see the chain of redirects (if any) and check the URL and status code at each step. This allows us to verify that each view is doing what is expected.
Let’s start with one of our simplest views, which provides a list of all Authors. This is displayed at URL /catalog/authors/ (a URL named ‘authors’ in the URL configuration).
As this is a generic list view almost everything is done for us by Django. Arguably if you trust Django then the only thing you need to test is that the view is accessible at the correct URL and can be accessed using its name. However if you’re using a test-driven development process you’ll start by writing tests that confirm that the view displays all Authors, paginating them in lots of 10.
Open the /catalog/tests/test_views.py file and replace any existing text with the following test code for AuthorListView. As before we import our model and some useful classes. In the setUpTestData() method we set up a number of Author objects so that we can test our pagination.
from django.test import TestCase
from django.urls import reverse
from catalog.models import Author
class AuthorListViewTest(TestCase):
@classmethod
def setUpTestData(cls):
# Create 13 authors for pagination tests
number_of_authors = 13
for author_id in range(number_of_authors):
Author.objects.create(
first_name=f'Dominique {author_id}',
last_name=f'Surname {author_id}',
)
def test_view_url_exists_at_desired_location(self):
response = self.client.get('/catalog/authors/')
self.assertEqual(response.status_code, 200)
def test_view_url_accessible_by_name(self):
response = self.client.get(reverse('authors'))
self.assertEqual(response.status_code, 200)
def test_view_uses_correct_template(self):
response = self.client.get(reverse('authors'))
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, 'catalog/author_list.html')
def test_pagination_is_ten(self):
response = self.client.get(reverse('authors'))
self.assertEqual(response.status_code, 200)
self.assertTrue('is_paginated' in response.context)
self.assertTrue(response.context['is_paginated'] == True)
self.assertEqual(len(response.context['author_list']), 10)
def test_lists_all_authors(self):
# Get second page and confirm it has (exactly) remaining 3 items
response = self.client.get(reverse('authors')+'?page=2')
self.assertEqual(response.status_code, 200)
self.assertTrue('is_paginated' in response.context)
self.assertTrue(response.context['is_paginated'] == True)
self.assertEqual(len(response.context['author_list']), 3)
All the tests use the client (belonging to our TestCase’s derived class) to simulate a GET request and get a response. The first version checks a specific URL (note, just the specific path without the domain) while the second generates the URL from its name in the URL configuration.
Once we have the response we query it for its status code, the template used, whether or not the response is paginated, the number of items returned, and the total number of items.
The most interesting variable we demonstrate above is response.context, which is the context variable passed to the template by the view. This is incredibly useful for testing, because it allows us to confirm that our template is getting all the data it needs. In other words we can check that we’re using the intended template and what data the template is getting, which goes a long way to verifying that any rendering issues are solely due to template.
Views that are restricted to logged in users
In some cases you’ll want to test a view that is restricted to just logged in users. For example our LoanedBooksByUserListView is very similar to our previous view but is only available to logged in users, and only displays BookInstance records that are borrowed by the current user, have the ‘on loan’ status, and are ordered “oldest first”.
Add the following test code to /catalog/tests/test_views.py. Here we first use SetUp() to create some user login accounts and BookInstance objects (along with their associated books and other records) that we’ll use later in the tests. Half of the books are borrowed by each test user, but we’ve initially set the status of all books to “maintenance”. We’ve used SetUp() rather than setUpTestData() because we’ll be modifying some of these objects later.
import datetime
from django.utils import timezone
# Get user model from settings
from django.contrib.auth import get_user_model
User = get_user_model()
from catalog.models import BookInstance, Book, Genre, Language
class LoanedBookInstancesByUserListViewTest(TestCase):
def setUp(self):
# Create two users
test_user1 = User.objects.create_user(username='testuser1', password='1X<ISRUkw+tuK')
test_user2 = User.objects.create_user(username='testuser2', password='2HJ1vRV0Z&3iD')
test_user1.save()
test_user2.save()
# Create a book
test_author = Author.objects.create(first_name='John', last_name='Smith')
test_genre = Genre.objects.create(name='Fantasy')
test_language = Language.objects.create(name='English')
test_book = Book.objects.create(
title='Book Title',
summary='My book summary',
isbn='ABCDEFG',
author=test_author,
language=test_language,
)
# Create genre as a post-step
genre_objects_for_book = Genre.objects.all()
test_book.genre.set(genre_objects_for_book) # Direct assignment of many-to-many types not allowed.
test_book.save()
# Create 30 BookInstance objects
number_of_book_copies = 30
for book_copy in range(number_of_book_copies):
return_date = timezone.localtime() + datetime.timedelta(days=book_copy%5)
the_borrower = test_user1 if book_copy % 2 else test_user2
status = 'm'
BookInstance.objects.create(
book=test_book,
imprint='Unlikely Imprint, 2016',
due_back=return_date,
borrower=the_borrower,
status=status,
)
def test_redirect_if_not_logged_in(self):
response = self.client.get(reverse('my-borrowed'))
self.assertRedirects(response, '/accounts/login/?next=/catalog/mybooks/')
def test_logged_in_uses_correct_template(self):
login = self.client.login(username='testuser1', password='1X<ISRUkw+tuK')
response = self.client.get(reverse('my-borrowed'))
# Check our user is logged in
self.assertEqual(str(response.context['user']), 'testuser1')
# Check that we got a response "success"
self.assertEqual(response.status_code, 200)
# Check we used correct template
self.assertTemplateUsed(response, 'catalog/bookinstance_list_borrowed_user.html')
To verify that the view will redirect to a login page if the user is not logged in we use assertRedirects, as demonstrated in test_redirect_if_not_logged_in(). To verify that the page is displayed for a logged in user we first log in our test user, and then access the page again and check that we get a status_code of 200 (success).
The rest of the tests verify that our view only returns books that are on loan to our current borrower. Copy the code below and paste it onto the end of the test class above.
def test_only_borrowed_books_in_list(self):
login = self.client.login(username='testuser1', password='1X<ISRUkw+tuK')
response = self.client.get(reverse('my-borrowed'))
# Check our user is logged in
self.assertEqual(str(response.context['user']), 'testuser1')
# Check that we got a response "success"
self.assertEqual(response.status_code, 200)
# Check that initially we don't have any books in list (none on loan)
self.assertTrue('bookinstance_list' in response.context)
self.assertEqual(len(response.context['bookinstance_list']), 0)
# Now change all books to be on loan
books = BookInstance.objects.all()[:10]
for book in books:
book.status = 'o'
book.save()
# Check that now we have borrowed books in the list
response = self.client.get(reverse('my-borrowed'))
# Check our user is logged in
self.assertEqual(str(response.context['user']), 'testuser1')
# Check that we got a response "success"
self.assertEqual(response.status_code, 200)
self.assertTrue('bookinstance_list' in response.context)
# Confirm all books belong to testuser1 and are on loan
for bookitem in response.context['bookinstance_list']:
self.assertEqual(response.context['user'], bookitem.borrower)
self.assertEqual(bookitem.status, 'o')
def test_pages_ordered_by_due_date(self):
# Change all books to be on loan
for book in BookInstance.objects.all():
book.status='o'
book.save()
login = self.client.login(username='testuser1', password='1X<ISRUkw+tuK')
response = self.client.get(reverse('my-borrowed'))
# Check our user is logged in
self.assertEqual(str(response.context['user']), 'testuser1')
# Check that we got a response "success"
self.assertEqual(response.status_code, 200)
# Confirm that of the items, only 10 are displayed due to pagination.
self.assertEqual(len(response.context['bookinstance_list']), 10)
last_date = 0
for book in response.context['bookinstance_list']:
if last_date == 0:
last_date = book.due_back
else:
self.assertTrue(last_date <= book.due_back)
last_date = book.due_back
Testing views with forms
Testing views with forms is a little more complicated than in the cases above, because you need to test more code paths: initial display, display after data validation has failed, and display after validation has succeeded. The good news is that we use the client for testing in almost exactly the same way as we did for display-only views.
To demonstrate, let’s write some tests for the view used to renew books (renew_book_librarian()):
We’ll need to test that the view is only available to users who have the can_mark_returned permission, and that users are redirected to an HTTP 404 error page if they attempt to renew a BookInstance that does not exist. We should check that the initial value of the form is seeded with a date three weeks in the future, and that if validation succeeds we’re redirected to the “all-borrowed books” view. As part of checking the validation-fail tests we’ll also check that our form is sending the appropriate error messages.
Add the first part of the test class (shown below) to the bottom of /catalog/tests/test_views.py. This creates two users and two book instances, but only gives one user the permission required to access the view.
import uuid
from django.contrib.auth.models import Permission # Required to grant the permission needed to set a book as returned.
class RenewBookInstancesViewTest(TestCase):
def setUp(self):
# Create a user
test_user1 = User.objects.create_user(username='testuser1', password='1X<ISRUkw+tuK')
test_user2 = User.objects.create_user(username='testuser2', password='2HJ1vRV0Z&3iD')
test_user1.save()
test_user2.save()
# Give test_user2 permission to renew books.
permission = Permission.objects.get(name='Set book as returned')
test_user2.user_permissions.add(permission)
test_user2.save()
# Create a book
test_author = Author.objects.create(first_name='John', last_name='Smith')
test_genre = Genre.objects.create(name='Fantasy')
test_language = Language.objects.create(name='English')
test_book = Book.objects.create(
title='Book Title',
summary='My book summary',
isbn='ABCDEFG',
author=test_author,
language=test_language,
)
# Create genre as a post-step
genre_objects_for_book = Genre.objects.all()
test_book.genre.set(genre_objects_for_book) # Direct assignment of many-to-many types not allowed.
test_book.save()
# Create a BookInstance object for test_user1
return_date = datetime.date.today() + datetime.timedelta(days=5)
self.test_bookinstance1 = BookInstance.objects.create(
book=test_book,
imprint='Unlikely Imprint, 2016',
due_back=return_date,
borrower=test_user1,
status='o',
)
# Create a BookInstance object for test_user2
return_date = datetime.date.today() + datetime.timedelta(days=5)
self.test_bookinstance2 = BookInstance.objects.create(
book=test_book,
imprint='Unlikely Imprint, 2016',
due_back=return_date,
borrower=test_user2,
status='o',
)
Add the following tests to the bottom of the test class. These check that only users with the correct permissions (testuser2) can access the view. We check all the cases: when the user is not logged in, when a user is logged in but does not have the correct permissions, when the user has permissions but is not the borrower (should succeed), and what happens when they try to access a BookInstance that doesn’t exist. We also check that the correct template is used.
def test_redirect_if_not_logged_in(self):
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}))
# Manually check redirect (Can't use assertRedirect, because the redirect URL is unpredictable)
self.assertEqual(response.status_code, 302)
self.assertTrue(response.url.startswith('/accounts/login/'))
def test_forbidden_if_logged_in_but_not_correct_permission(self):
login = self.client.login(username='testuser1', password='1X<ISRUkw+tuK')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}))
self.assertEqual(response.status_code, 403)
def test_logged_in_with_permission_borrowed_book(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance2.pk}))
# Check that it lets us login - this is our book and we have the right permissions.
self.assertEqual(response.status_code, 200)
def test_logged_in_with_permission_another_users_borrowed_book(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}))
# Check that it lets us login. We're a librarian, so we can view any users book
self.assertEqual(response.status_code, 200)
def test_HTTP404_for_invalid_book_if_logged_in(self):
# unlikely UID to match our bookinstance!
test_uid = uuid.uuid4()
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk':test_uid}))
self.assertEqual(response.status_code, 404)
def test_uses_correct_template(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}))
self.assertEqual(response.status_code, 200)
# Check we used correct template
self.assertTemplateUsed(response, 'catalog/book_renew_librarian.html')
Add the next test method, as shown below. This checks that the initial date for the form is three weeks in the future. Note how we are able to access the value of the initial value of the form field (response.context[‘form’].initial[‘renewal_date’]).
def test_form_renewal_date_initially_has_date_three_weeks_in_future(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
response = self.client.get(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}))
self.assertEqual(response.status_code, 200)
date_3_weeks_in_future = datetime.date.today() + datetime.timedelta(weeks=3)
self.assertEqual(response.context['form'].initial['renewal_date'], date_3_weeks_in_future)
The next test (add this to the class too) checks that the view redirects to a list of all borrowed books if renewal succeeds. What differs here is that for the first time we show how you can POST data using the client. The post data is the second argument to the post function, and is specified as a dictionary of key/values.
def test_redirects_to_all_borrowed_book_list_on_success(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
valid_date_in_future = datetime.date.today() + datetime.timedelta(weeks=2)
response = self.client.post(reverse('renew-book-librarian', kwargs={'pk':self.test_bookinstance1.pk,}), {'renewal_date':valid_date_in_future})
self.assertRedirects(response, reverse('all-borrowed'))
Warning: The all-borrowed view was added as a challenge, and your code may instead redirect to the home page ‘/’. If so, modify the last two lines of the test code to be like the code below. The follow=True in the request ensures that the request returns the final destination URL (hence checking /catalog/ rather than /).
response = self.client.post(reverse('renew-book-librarian', kwargs={'pk':self.test_bookinstance1.pk,}), {'renewal_date':valid_date_in_future}, follow=True)
self.assertRedirects(response, '/catalog/')
Copy the last two functions into the class, as seen below. These again test POST requests, but in this case with invalid renewal dates. We use assertFormError() to verify that the error messages are as expected.
def test_form_invalid_renewal_date_past(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
date_in_past = datetime.date.today() - datetime.timedelta(weeks=1)
response = self.client.post(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}), {'renewal_date': date_in_past})
self.assertEqual(response.status_code, 200)
self.assertFormError(response.context['form'], 'renewal_date', 'Invalid date - renewal in past')
def test_form_invalid_renewal_date_future(self):
login = self.client.login(username='testuser2', password='2HJ1vRV0Z&3iD')
invalid_date_in_future = datetime.date.today() + datetime.timedelta(weeks=5)
response = self.client.post(reverse('renew-book-librarian', kwargs={'pk': self.test_bookinstance1.pk}), {'renewal_date': invalid_date_in_future})
self.assertEqual(response.status_code, 200)
self.assertFormError(response.context['form'], 'renewal_date', 'Invalid date - renewal more than 4 weeks ahead')
Templates
Django provides test APIs to check that the correct template is being called by your views, and to allow you to verify that the correct information is being sent. There is however no specific API support for testing in Django that your HTML output is rendered as expected.
Other recommended test tools
Django’s test framework can help you write effective unit and integration tests — we’ve only scratched the surface of what the underlying unittest framework can do, let alone Django’s additions (for example, check out how you can use unittest.mock to patch third party libraries so you can more thoroughly test your own code).
While there are numerous other test tools that you can use, we’ll just highlight two:
-
Coverage: This Python tool reports on how much of your code is actually executed by your tests. It is particularly useful when you’re getting started, and you are trying to work out exactly what you should test.
-
Selenium is a framework to automate testing in a real browser. It allows you to simulate a real user interacting with the site, and provides a great framework for system testing your site (the next step up from integration testing).