DEV Community: Aidas Bendoraitis

Wagtail Routable Pages and Layout Configuration

Aidas Bendoraitis — Sat, 21 Mar 2026 11:07:32 +0000

If you are familiar with Wagtail CMS for Django, you know that you can create Wagtail pages and control their content and layout with blocks inside of stream fields. But what if you have entries coming from normal Django models through a routable page? In this article, I will explore how you can control the dynamic layout of a detail view in a routable page.

Routable pages in Wagtail are dynamic pages of your CMS page tree that can have their own URL subpaths and views. You can use them for filtered list and detail views, multi-step forms, multiple formats for the same data, etc. Here I will show you a routable ArticleIndexPage with a list and detail views for Article instances rendering the detail views based on the block layout in a detail_layout stream field.

1. Project Setup

Create a Wagtail project myproject and articles app:

pip install wagtail
wagtail start myproject
cd myproject
python manage.py startapp articles

Add to INSTALLED_APPS in your Django project settings:

INSTALLED_APPS = [
    ...
    "wagtail.contrib.routable_page",  # required for RoutablePage
    "myproject.apps.articles",
]

2. File Structure

The articles app:

myproject/apps/articles/
├── __init__.py
├── apps.py
├── models.py          # Article, Category, ArticleIndexPage
├── blocks.py          # All StreamField block definitions
└── admin.py           # Register Article and Category in Django admin

The articles templates:

myproject/templates/articles/
├── article_list.html           # List view
├── article_detail.html         # Detail view
└── blocks/
    ├── cover_image_block.html
    ├── description_block.html
    └── related_articles_block.html

3. Models

`myproject/apps/articles/models.py`

Create the Category and Article Django models, and the ArticleIndexPage routable Wagtail page with article list and detail views:

from django.core.paginator import EmptyPage, PageNotAnInteger, Paginator
from django.db import models
from django.shortcuts import get_object_or_404
from django.utils.translation import gettext_lazy as _

from wagtail.admin.panels import FieldPanel, ObjectList, TabbedInterface
from wagtail.contrib.routable_page.models import RoutablePageMixin, path
from wagtail.fields import StreamField
from wagtail.models import Page

from .blocks import article_detail_layout_blocks


class Category(models.Model):
    name = models.CharField(max_length=100, verbose_name=_("name"))
    slug = models.SlugField(unique=True, verbose_name=_("slug"))

    class Meta:
        verbose_name = _("category")
        verbose_name_plural = _("categories")

    def __str__(self):
        return self.name


class Article(models.Model):
    title = models.CharField(max_length=255, verbose_name=_("title"))
    slug = models.SlugField(unique=True, verbose_name=_("slug"))
    category = models.ForeignKey(
        Category,
        null=True,
        blank=True,
        on_delete=models.SET_NULL,
        related_name="articles",
        verbose_name=_("category"),
    )
    cover_image = models.ForeignKey(
        "wagtailimages.Image",
        null=True,
        blank=True,
        on_delete=models.SET_NULL,
        related_name="+",
        verbose_name=_("cover image"),
    )
    description = models.TextField(blank=True, verbose_name=_("description"))
    created_at = models.DateTimeField(auto_now_add=True, verbose_name=_("created at"))

    class Meta:
        verbose_name = _("article")
        verbose_name_plural = _("articles")

    def __str__(self):
        return self.title


class ArticleIndexPage(RoutablePageMixin, Page):
    """
    A single Wagtail page that owns:
      - /articles/          → paginated list of all Articles
      - /articles/<slug>/   → detail view for one Article

    The StreamField is edited once in the Wagtail admin and
    defines the layout for every detail view.
    """

    articles_per_page = models.IntegerField(default=10, verbose_name=_("articles per page"))

    detail_layout = StreamField(
        article_detail_layout_blocks(),
        blank=True,
        use_json_field=True,
        verbose_name=_("detail layout"),
        help_text=_(
            "Configure the layout for all article detail pages. "
            "Add, remove, and reorder blocks to change what appears "
            "on every article detail view."
        ),
    )

    # TabbedInterface gives List View and Detail View their own tabs.
    # promote_panels and settings_panels must be added explicitly here
    # because edit_handler takes full ownership of the admin UI structure.
    edit_handler = TabbedInterface([
        ObjectList(Page.content_panels + [FieldPanel("articles_per_page")], heading=_("List View")),
        ObjectList([FieldPanel("detail_layout")], heading=_("Detail View")),
        ObjectList(Page.promote_panels, heading=_("SEO / Promote")),
        ObjectList(Page.settings_panels, heading=_("Settings")),
    ])

    class Meta:
        verbose_name = _("article index page")
        verbose_name_plural = _("article index pages")

    @path("")
    def article_list(self, request):
        all_articles = Article.objects.select_related("category", "cover_image").order_by("-created_at")
        paginator = Paginator(all_articles, self.articles_per_page)
        page_number = request.GET.get("page")

        try:
            articles = paginator.page(page_number)
        except PageNotAnInteger:
            articles = paginator.page(1)
        except EmptyPage:
            articles = paginator.page(paginator.num_pages)

        return self.render(
            request,
            context_overrides={"articles": articles, "paginator": paginator},
            template="articles/article_list.html",
        )

    @path("<slug:article_slug>/")
    def article_detail(self, request, article_slug):
        article = get_object_or_404(
            Article.objects.select_related("category", "cover_image"),
            slug=article_slug,
        )
        return self.render(
            request,
            context_overrides={"article": article},
            template="articles/article_detail.html",
        )

4. StreamField Blocks

`myproject/apps/articles/blocks.py`

Create Wagtail stream-field blocks for the cover image, description, and the related articles of an actual article. Each block can have some settings on how to represent the content of the block.

from django.utils.translation import gettext_lazy as _

from wagtail import blocks


class CoverImageBlock(blocks.StructBlock):
    aspect_ratio = blocks.ChoiceBlock(
        choices=[
            ("16-9", _("16:9 Widescreen")),
            ("4-3", _("4:3 Standard")),
            ("1-1", _("1:1 Square")),
            ("3-1", _("3:1 Banner")),
        ],
        default="16-9",
        label=_("Aspect ratio"),
        help_text=_("Controls the cropping of the cover image."),
    )

    class Meta:
        template = "articles/blocks/cover_image_block.html"
        icon = "image"
        label = _("Cover Image")


class DescriptionBlock(blocks.StructBlock):
    max_lines = blocks.IntegerBlock(
        min_value=0,
        default=0,
        label=_("Maximum lines"),
        help_text=_("Clamp the description to this many lines. Set to 0 to show all."),
        required=False,
    )

    class Meta:
        template = "articles/blocks/description_block.html"
        icon = "pilcrow"
        label = _("Description")


class RelatedArticlesBlock(blocks.StructBlock):
    sort_order = blocks.ChoiceBlock(
        choices=[
            ("newest", _("Newest first")),
            ("oldest", _("Oldest first")),
            ("title_asc", _("Title A → Z")),
            ("title_desc", _("Title Z → A")),
        ],
        default="newest",
        label=_("Sort order"),
        help_text=_("Order in which related articles are listed."),
    )

    def get_context(self, value, parent_context=None):
        context = super().get_context(value, parent_context=parent_context)
        article = (parent_context or {}).get("article")
        if not article or not article.category_id:
            context["related_articles"] = []
            return context

        from .models import Article

        sort_map = {
            "newest": "-created_at",
            "oldest": "created_at",
            "title_asc": "title",
            "title_desc": "-title",
        }

        context["related_articles"] = (
            Article.objects.select_related("category", "cover_image")
            .filter(category=article.category)
            .exclude(pk=article.pk)
            .order_by(sort_map.get(value["sort_order"], "-created_at"))[:3]
        )
        return context

    class Meta:
        template = "articles/blocks/related_articles_block.html"
        icon = "list-ul"
        label = _("Related Articles")


def article_detail_layout_blocks():
    """
    Returns the list of (name, block) tuples used in ArticleIndexPage.detail_layout.
    Defined as a function so models.py can import it without circular issues.
    """
    return [
        ("cover_image", CoverImageBlock()),
        ("description", DescriptionBlock()),
        ("related_articles", RelatedArticlesBlock()),
    ]

The RelatedArticlesBlock here also has a customized context where we pass related_articles variable with 3 other articles of the same category sorted by the sorting order defined in the block.

5. Templates

`articles/article_list.html`

This will be the template for the paginated article list. Later you could augment it with a search form and filters.

{% extends "base.html" %}
{% load wagtailcore_tags wagtailimages_tags i18n wagtailroutablepage_tags %}

{% block content %}
<main class="article-index">
  <h1>{{ page.title }}</h1>
  <ul class="article-list">
    {% for article in articles %}
      <li class="article-card">
        {% if article.cover_image %}{% image article.cover_image width-400 as img %}
          <img src="{{ img.url }}" alt="{{ article.title }}">
        {% endif %}
        <h2>
          <a href="{% routablepageurl page "article_detail" article.slug %}">{{ article.title }}</a>
        </h2>
        {% if article.category %}<span class="badge">{{ article.category.name }}</span>{% endif %}
        <p>{{ article.description|truncatewords:30 }}</p>
      </li>
    {% empty %}
      <li>{% trans "No articles yet." %}</li>
    {% endfor %}
  </ul>
  {% if articles.has_other_pages %}
    <nav class="pagination" aria-label="{% trans 'Article pagination' %}">
      {% if articles.has_previous %}
        <a href="?page={{ articles.previous_page_number }}">{% trans "← Previous" %}</a>
      {% endif %}
      <span>{% blocktrans with num=articles.number total=articles.paginator.num_pages %}Page {{ num }} of {{ total }}{% endblocktrans %}</span>
      {% if articles.has_next %}
        <a href="?page={{ articles.next_page_number }}">{% trans "Next →" %}</a>
      {% endif %}
    </nav>
  {% endif %}
</main>
{% endblock %}

`articles/article_detail.html`

The detail page would use the {% include_block page.detail_layout with article=article page=page %} to pass the article to the context of each block:

{% extends "base.html" %}
{% load i18n wagtailcore_tags wagtailroutablepage_tags %}

{% block content %}
<article class="article-detail">
  <header>
    <h1>{{ article.title }}</h1>
    {% if article.category %}<span class="badge">{{ article.category.name }}</span>{% endif %}
  </header>
  {% include_block page.detail_layout with article=article page=page %}
  <p>
    <a href="{% routablepageurl page "article_list" %}">{% trans "← Back to all articles" %}</a>
  </p>
</article>
{% endblock %}

`articles/blocks/cover_image_block.html`

Cover image block would show the article cover image with the aspect ratio set in the block:

{% load wagtailimages_tags %}

{% if article.cover_image %}
  <div class="cover-image cover-image--{{ value.aspect_ratio }}">
    {% image article.cover_image width-1200 as img %}
    <img src="{{ img.url }}" alt="{{ article.title }}">
  </div>
{% endif %}

`articles/blocks/description_block.html`

Description block would hide the article description text overflow based on the max lines set in the block:

<section class="article-description">
  <p{% if value.max_lines > 0 %} class="line-clamp" style="-webkit-line-clamp: {{ value.max_lines }};"{% endif %}>
    {{ article.description }}
  </p>
</section>

`articles/blocks/related_articles_block.html`

The related articles block would list the related articles as defined in the extra context of the block:

{% load i18n wagtailimages_tags wagtailroutablepage_tags %}

{% if related_articles %}
  <section class="related-articles">
    <h2>{% trans "Related Articles" %}</h2>
    <ul class="related-articles__list">
      {% for rel in related_articles %}
        <li class="related-card">
          {% if rel.cover_image %}{% image rel.cover_image width-400 as img %}
            <img src="{{ img.url }}" alt="{{ rel.title }}">
          {% endif %}
          <div class="related-card__body">
            {% if rel.category %}<span class="badge">{{ rel.category.name }}</span>{% endif %}
            <h3>
              <a href="{% routablepageurl page "article_detail" rel.slug %}">{{ rel.title }}</a>
            </h3>
            <p>{{ rel.description|truncatewords:20 }}</p>
          </div>
        </li>
      {% endfor %}
    </ul>
  </section>
{% endif %}

6. Django Admin Registration

`articles/admin.py`

Let's not forget to register admin views for the categories and articles so that we can add some data there:

from django.contrib import admin
from .models import Article, Category


@admin.register(Category)
class CategoryAdmin(admin.ModelAdmin):
    list_display = ("name", "slug")
    prepopulated_fields = {"slug": ("name",)}


@admin.register(Article)
class ArticleAdmin(admin.ModelAdmin):
    list_display = ("title", "category", "created_at")
    list_filter = ("category",)
    search_fields = ("title", "description")
    prepopulated_fields = {"slug": ("title",)}

7. Migrations and Initial Data

python manage.py makemigrations articles
python manage.py migrate
python manage.py createsuperuser
python manage.py runserver

8. Wagtail Admin Setup

Open http://localhost:8000/cms/ and log in.
In the Pages explorer, create an Article Index Page as a child of the root page.
- Set the Slug to articles.
On the List View tab, set Articles per page (e.g. 24).
On the Detail View tab, open the Detail Layout StreamField and add blocks in your preferred order:
- Cover Image — choose an aspect ratio.
- Description — optionally set a maximum line count to clamp long descriptions.
- Related Articles — choose the sort order for the three related articles shown.
Publish the page.
In the Django admin (/django-admin/), create some Categories and Articles with cover images and descriptions.
Visit http://localhost:8000/articles/ for the paginated list.
Click any article to see the detail view rendered using the StreamField layout you configured in step 4.

Final words

Using stream fields we can render not only editorial content, for example, images or rich-text descriptions, but also dynamic content based on values from other models and/or the context of the given template.

The approach illustrated in this article allows us to create Wagtail pages where content editors have freedom to adjust the layouts of the pages or insert blocks, such as ads or info texts, into specific places based on real-time events.

How to Show a Waitlist Until Your Wagtail Site Is Ready

Aidas Bendoraitis — Sat, 21 Mar 2026 11:02:14 +0000

This year, I want to bring my centralized gamified donation platform www.make-impact.org to life (at least technically). Earlier I had the version I was developing separate from the waiting list, but I decided to merge them and have a switch between the waitlist and an early preview.

This allows me to have no data duplication, the possibility to create user accounts immediately, and saves hosting and maintenance costs.

This guide walks through a pattern that lets you ship a temporary waitlist page while your Wagtail site is still being built, with the ability to show your progress to chosen people. If you are building a Software as a Service (SaaS) or a web platform with Django, this article is for you.

The Concept

A custom start page view will check for a specific cookie value. If it is unset, the visitor will be redirected to a waitlist form at /waitlist/. If it is set, the visitor will be served the Wagtail home page.

All views under development will have a decorator that checks the cookie value and redirects to the start page if it is unset.

There will be a special view at /preview-access/ with a passphrase form that allows the visitor to gain preview access by setting the mentioned cookie. This view will also allow preview access to be deactivated.

These are the steps to implement this:

1. Generate and store two secrets

You will need two secret values, either set manually or generated with a cryptographically secure random generator (e.g. Python's secrets module):

PREVIEW_ACCESS_PASSPHRASE — the human-readable passphrase typed into the form. Share this with the people who need site access.
PREVIEW_ACCESS_TOKEN — the opaque random value stored in the cookie. Never exposed to users; only the server compares against it.

>>> import secrets
>>> print(secrets.token_urlsafe(16))  # passphrase
dI5nGNftZOBx8m-r0m6glg
>>> print(secrets.token_hex(32))      # cookie token
c1b7a76e3ad5cbfb1657fa4e9885a3c8baa6a5a869f49a136abd0e873a9be9ee

Add both to environment variables or a secrets file untracked by Git, and load them in the Django project settings:

# myproject/settings/_base.py
PREVIEW_ACCESS_PASSPHRASE = get_secret("PREVIEW_ACCESS_PASSPHRASE")
PREVIEW_ACCESS_TOKEN = get_secret("PREVIEW_ACCESS_TOKEN")

The get_secret() here is my custom function to retrieve a secret from the secrets source.

2. Create the access-control decorator

Create myproject/apps/misc/decorators.py. Every protected view will import from here.

# myproject/apps/misc/decorators.py
from functools import wraps

from django.conf import settings
from django.shortcuts import redirect


def preview_access_required(view_func):
    @wraps(view_func)
    def wrapper(request, *args, **kwargs):
        if request.COOKIES.get("preview_access") == settings.PREVIEW_ACCESS_TOKEN:
            return view_func(request, *args, **kwargs)
        return redirect("misc:home_page")
    return wrapper

The decorator compares the cookie against the opaque unguessable token from settings, so unless the token value is known, a random attacker cannot gain access by setting the cookie manually in DevTools.

3. Create the passphrase form

Create myproject/apps/misc/forms.py. The form will have a single required password field. Validation will reject any value that does not match the setting.

# myproject/apps/misc/forms.py
from django import forms
from django.conf import settings
from django.utils.translation import gettext_lazy as _


class PreviewAccessForm(forms.Form):
    passphrase = forms.CharField(
        label=_("Passphrase"),
        widget=forms.PasswordInput(
            attrs={"autocomplete": "current-password"}
        ),
        required=True,
    )

    def clean_passphrase(self):
        value = self.cleaned_data["passphrase"]
        if value != settings.PREVIEW_ACCESS_PASSPHRASE:
            raise forms.ValidationError(
                _("Incorrect passphrase.")
            )
        return value

4. Build the cookie toggle view

Point your browser to /preview-access/. When access is off it shows a passphrase form; when access is on it shows a disable button.

# myproject/apps/misc/views.py
from django.conf import settings
from django.shortcuts import redirect, render

from .forms import PreviewAccessForm


def preview_access(request):
    has_access = request.COOKIES.get("preview_access") == settings.PREVIEW_ACCESS_TOKEN

    if request.method == "POST":
        if has_access:
            response = redirect("misc:home_page")
            response.delete_cookie("preview_access")
            return response

        form = PreviewAccessForm(request.POST)
        if form.is_valid():
            response = redirect("misc:home_page")
            response.set_cookie(
                "preview_access",
                settings.PREVIEW_ACCESS_TOKEN,
                httponly=True,
                samesite="Strict",
            )
            return response
    else:
        form = PreviewAccessForm()

    return render(
        request, 
        "preview_access/preview_access.html", 
        {"has_access": has_access, "form": form}
    )

Key points:

Disabling never requires the passphrase — the cookie is already proof of prior access.
The cookie is set with httponly=True (not readable by JavaScript) and samesite="Strict" (not sent on cross-site requests).
The cookie value is the opaque token, not "1", so it cannot be guessed.

The template renders the passphrase input only when not has_access, and shows field-level errors from the form if the passphrase is wrong.

5. Wrap the Wagtail catch-all with the decorator

Replace the default Wagtail catch-all route handler with a thin wrapper that enforces the same cookie check.

# myproject/apps/misc/views.py
from myproject.apps.misc.decorators import preview_access_required
from wagtail.views import serve as wagtail_serve


@preview_access_required
def serve_wagtail_page(request, path=""):
    return wagtail_serve(request, path)

Without this, a visitor who knows any Wagtail page URL could bypass the gate by typing it directly into the browser.

6. Build the proxy home page view

This view is the only entry point to the site. It decides what every visitor sees first.

# myproject/apps/misc/views.py
from django.conf import settings

from wagtail.models import Site
from wagtail.views import serve as wagtail_serve


def home_page(request):
    if request.COOKIES.get("preview_access") == settings.PREVIEW_ACCESS_TOKEN:
        # serve Wagtail directly
        site = Site.find_for_request(request)
        return wagtail_serve(request, "")  

    # redirect to waiting list
    return redirect("waiting_list")

Key point: the waiting_list view and a Wagtail Site and page must exist and be matched to the request domain before wagtail_serve is called.

7. Wire up the URLs

Django project URL rules:

# myproject/urls.py
from django.conf.urls.i18n import i18n_patterns
from django.urls import re_path

from wagtail.coreutils import WAGTAIL_APPEND_SLASH

from myproject.apps.misc import views as misc_views

if WAGTAIL_APPEND_SLASH:
    wagtail_serve_pattern = r"^((?:[\w\-]+/)*)$"
else:
    wagtail_serve_pattern = r"^([\w\-/]*)$"

urlpatterns += i18n_patterns(
    # ... all your other app URLs above ...

    # Catch-all — must be last
    re_path(
        wagtail_serve_pattern,
        misc_views.serve_wagtail_page,
        name="wagtail_serve"
    ),
)

The misc app URLs:

# myproject/apps/misc/urls.py
from django.urls import path

from . import views

app_name = "misc"

urlpatterns = [
    path("", views.home_page, name="home_page"),
    path("preview-access/", views.preview_access, name="preview_access"),
]

The waiting_list app URLs:

# myproject/apps/waiting_list/urls.py
from django.urls import path

from . import views

urlpatterns = [
    path("waitlist/", views.show_waiting_list_form, name="waiting_list"),
]

8. Protect every other app view

Import and apply @preview_access_required to every view that belongs to the real site. Class-based views can be wrapped at assignment time:

from myproject.apps.misc.decorators import preview_access_required

# Function-based view
@preview_access_required
def event_list(request): 
    ...

# Class-based view
event_list = preview_access_required(
    EventListView.as_view()
)

Waiting-list views, API views, social authentication views, and static/legal pages (/imprint/, /privacy/, etc.) must not receive this decorator — they need to remain publicly accessible.

Final words

You get a lot of benefits from this setup. The waitlist measures demand for your website while you are still building. Invited test users can evaluate your progress at any time. While you are developing the website, you do not necessarily need multiple servers. Launching later is also easier — no hassle or delays with domain IP updates and SSL certificates.

Using tox to Test a Django App Across Multiple Django Versions

Aidas Bendoraitis — Fri, 27 Feb 2026 18:53:25 +0000

Recently, I developed a reusable Django app django-clearplaintext for normalizing plain text in Django templates. And to package and test it properly, I had a fresh look at Tox.

Tox is the standard testing tool that creates isolated virtual environments, installs the exact dependencies you specify, and runs your test suite in each one — all from a single command.

This post walks through a complete, working setup using a minimal example app called django-shorturl.

The Example App: django-shorturl

django-shorturl is a self-contained Django app with one model and one view.

shorturl/models.py

from django.db import models
from django.utils.translation import gettext_lazy as _

class ShortLink(models.Model):
    slug = models.SlugField(_("slug"), unique=True)
    target_url = models.URLField(_("target URL"))
    created_at = models.DateTimeField(_("created at"), auto_now_add=True)

    class Meta:
        verbose_name = _("short link")
        verbose_name_plural = _("short links")

    def __str__(self):
        return self.slug

shorturl/views.py

from django.shortcuts import get_object_or_404, redirect

from .models import ShortLink

def redirect_link(request, slug):
    link = get_object_or_404(ShortLink, slug=slug)
    return redirect(link.target_url)

shorturl/urls.py

from django.urls import path

from . import views

urlpatterns = [
    path("<slug:slug>/", views.redirect_link, name="redirect_link"),
]

shorturl/admin.py

from django.contrib import admin
from .models import ShortLink

admin.site.register(ShortLink)

Project Layout

django-shorturl/
├── src/
│   └── shorturl/
│       ├── __init__.py
│       ├── admin.py
│       ├── models.py
│       ├── views.py
│       └── urls.py
├── tests/
│   ├── __init__.py
│   └── test_views.py
├── pyproject.toml
├── test_settings.py
└── tox.ini

The source lives under src/ and the tests are at the top level, separate from the package. This separation prevents the tests from accidentally being shipped inside the installed package.

Packaging: pyproject.toml

Tox needs a properly packaged app to install into each environment. With isolated_build = true (more on that below), Tox builds a wheel from your pyproject.toml before running any tests.

pyproject.toml

[project]
name = "django-shorturl"
version = "1.0.0"
requires-python = ">=3.8"
dependencies = [
    "Django>=4.2",
]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]

The dependencies list here declares the runtime minimum — your app needs Django, but you don't pin a specific version because that is Tox's job during testing.

For the [build-system] section, we can also use uv_build to gain some performance improvements:

[build-system]
requires = ["uv_build >= 0.10.0, <0.11.0"]
build-backend = "uv_build"

[tool.uv.build-backend]
module-name = "shorturl"

Here module-name lets uv_build not to get confused between django-shorturl and shorturl.

Test Settings: test_settings.py

Django requires a settings module to run. As we don't have an associated project, we have to create a minimal one by defining project settings in the project's settings, create a minimal one dedicated to testing. It lives at the repo root so it's easy to point to from anywhere.

test_settings.py

SECRET_KEY = "test"

INSTALLED_APPS = [
    "shorturl",
]

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.sqlite3",
        "NAME": ":memory:",
    }
}

ROOT_URLCONF = "shorturl.urls"

DEFAULT_AUTO_FIELD = "django.db.models.AutoField"

A few deliberate choices here:

SECRET_KEY = "test" — A fixed value, fine for tests, but never use this in production.
INSTALLED_APPS — Only include apps that your tests actually need. No django.contrib.admin, no auth, nothing extra.
SQLite in-memory database — ":memory:" means the database is created fresh for every test run and disappears when the process exits. No files left behind, no teardown needed, and it is fast.
ROOT_URLCONF — The test client resolves URLs through this setting. Without it, reverse() raises NoReverseMatch and the test client has no URL configuration to dispatch against. Point it at your app's urls.py.
DEFAULT_AUTO_FIELD — Suppresses Django's system check warning about the implicit primary key type. Setting it explicitly keeps the test output clean and makes the expectation clear.

The Core: tox.ini

This is where Tox is configured.

tox.ini

[tox]
envlist =
    py{38,39,310,311,312}-django42,
    py{310,311,312}-django50,
    py{310,311,312,313}-django51,
    py{310,311,312,313,314}-django52,
    py{312,313,314}-django60

isolated_build = true

[testenv]
deps =
    django42: Django>=4.2,<4.3
    django50: Django>=5.0,<5.1
    django51: Django>=5.1,<5.2
    django52: Django>=5.2,<6.0
    django60: Django>=6.0,<6.1
commands =
    python -m django test
setenv =
    DJANGO_SETTINGS_MODULE = test_settings

envlist — the matrix

py{38,39,310,311,312}-django42 is a shortcut used in Tox.

The numbers inside {} are expanded automatically. Tox combines each Python version with django42, creating 5 environments:

py38-django42
py39-django42
py310-django42
py311-django42
py312-django42

The full envlist simply lists all Python and Django combinations you want to test, so you can check that your project works in each setup.

Each part separated by a dash in an environment name is called a "factor". You can have as many factors as you like, and they can be named anything. py* factors are a convention for Python versions. Others need to be defined in the [testenv] deps section.

isolated_build = true

This tells tox to build a proper wheel from your pyproject.toml before installing into each environment. Without it, tox would try to install your package with pip install -e ., which bypasses the build system and can hide packaging bugs. With it, each environment tests the package exactly as a user would receive it after pip install django-shorturl.

deps — conditional dependencies

The django42: prefix is a Tox factor condition: the dependency on that line is only installed when the environment name contains the django42 factor. This is how a single [testenv] block handles all Django versions without needing a separate section for each one.

Tox also installs your package itself into each environment (because of isolated_build), so you don't need to list it here.

commands

commands =
    python -m django test

python -m django test is Django's built-in test runner. It discovers tests by looking for files matching test*.py under the current directory, which picks up everything in your tests/ folder automatically.

setenv

setenv =
    DJANGO_SETTINGS_MODULE = test_settings

Django refuses to run without a settings module. This environment variable tells it where to find yours. Because test_settings.py is at the repo root and tox runs from the repo root, the module name test_settings resolves correctly without any path manipulation.

Writing the Tests

Create test cases for each (critical) component of your app. For example, if you have models, views, and template tags, create tests/test_models.py, tests/test_views.py, and tests/test_templatetags.py.

tests/test_views.py

from django.test import TestCase
from django.urls import reverse

from shorturl.models import ShortLink


class RedirectLinkViewTest(TestCase):
    def setUp(self):
        ShortLink.objects.create(
            slug="dt",
            target_url="https://www.djangotricks.com",
        )

    def test_redirects_to_target_url(self):
        response = self.client.get(
            reverse(
                "redirect_link", kwargs={"slug": "dt"}
            )
        )
        self.assertRedirects(
            response,
            "https://www.djangotricks.com",
            fetch_redirect_response=False,
        )

    def test_returns_404_for_unknown_slug(self):
        response = self.client.get(
            reverse(
                "redirect_link", kwargs={"slug": "nope"}
            )
        )
        self.assertEqual(response.status_code, 404)

Installing Python Versions with pyenv

Tox needs the actual Python binaries for every version in your envlist. If you try to run tox without them installed, it will fail immediately with an InterpreterNotFound error. pyenv is the standard way to install and manage multiple Python versions side by side.

Install pyenv

Use Homebrew on macOS (or follow the official instructions for Linux):

brew install pyenv

Add the following to your shell config (~/.zshrc, ~/.bashrc, etc.) and restart your shell:

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

Install each Python version

Install every version that appears in your envlist:

pyenv install 3.8
pyenv install 3.9
pyenv install 3.10
pyenv install 3.11
pyenv install 3.12
pyenv install 3.13
pyenv install 3.14

Make them all reachable at once

Tox resolves py312 by looking for a binary named python3.12 on PATH. The trick is pyenv global, which accepts multiple versions and places all of their binaries on your PATH simultaneously:

pyenv global 3.14 3.13 3.12 3.11 3.10 3.9 3.8

List the first (the one python3 and python resolve to) and work downward. After running this, confirm every interpreter is visible:

python3.8 --version   # Python 3.8.x
python3.9 --version   # Python 3.9.x
python3.10 --version   # Python 3.10.x
python3.11 --version   # Python 3.11.x
python3.12 --version   # Python 3.12.x
python3.13 --version   # Python 3.13.x
python3.14 --version   # Python 3.14.x

Now tox can find all of them and the full matrix will run without InterpreterNotFound errors.

Running tox

Run the full matrix:

tox

Or run a single environment:

tox -e py312-django52

tox will print a summary at the end showing which environments passed and which failed.

  py38-django42: OK (3.25=setup[2.32]+cmd[0.93] seconds)
  py39-django42: OK (2.88=setup[2.16]+cmd[0.72] seconds)
  py310-django42: OK (2.61=setup[2.02]+cmd[0.59] seconds)
  py311-django42: OK (2.70=setup[2.09]+cmd[0.61] seconds)
  py312-django42: OK (3.28=setup[2.46]+cmd[0.82] seconds)
  py310-django50: OK (2.67=setup[2.09]+cmd[0.58] seconds)
  py311-django50: OK (2.61=setup[2.02]+cmd[0.59] seconds)
  py312-django50: OK (2.85=setup[2.25]+cmd[0.60] seconds)
  py310-django51: OK (2.81=setup[2.27]+cmd[0.54] seconds)
  py311-django51: OK (2.85=setup[2.30]+cmd[0.55] seconds)
  py312-django51: OK (2.70=setup[2.09]+cmd[0.61] seconds)
  py313-django51: OK (2.97=setup[2.29]+cmd[0.68] seconds)
  py310-django52: OK (3.03=setup[2.31]+cmd[0.72] seconds)
  py311-django52: OK (2.88=setup[2.22]+cmd[0.66] seconds)
  py312-django52: OK (2.80=setup[2.13]+cmd[0.67] seconds)
  py313-django52: OK (4.70=setup[3.66]+cmd[1.04] seconds)
  py314-django52: OK (6.41=setup[5.18]+cmd[1.23] seconds)
  py312-django60: OK (5.13=setup[4.06]+cmd[1.07] seconds)
  py313-django60: OK (5.35=setup[4.15]+cmd[1.21] seconds)
  py314-django60: OK (6.01=setup[4.65]+cmd[1.37] seconds)
  congratulations :) (70.59 seconds)

Final Words

What makes this setup robust?

No shared state between environments. Each Tox environment is its own virtualenv with its own Django installation.
The package is built, not symlinked. isolated_build = true catches packaging mistakes before they reach users.
The database never persists between runs. SQLite in-memory means no stale data, no cleanup scripts, no CI-specific teardown.
The test settings are minimal by design. Fewer installed apps means faster startup, fewer implicit dependencies, and tests that fail for clear, local reasons rather than configuration noise from elsewhere in the project.

This setup is not the only way to test a Django app with Tox, but it is a solid starting point that balances comprehensiveness with maintainability. With a little effort upfront, you can ensure your app works across a wide range of Python and Django versions — and catch packaging bugs before they hit real users.

Understanding the Different POST Content Types

Aidas Bendoraitis — Sun, 16 Nov 2025 23:54:35 +0000

After more than 20 years of building for the web, this topic somehow kept slipping past me. It always felt obvious, so I never looked deeper. Recently I finally took the time to explore it properly, did some quick research, and now I’m sharing the results. Here’s a simple walkthrough of the different content types you can send in POST requests.

Standard Form Data

When you submit a basic HTML form like <form method="post" action=""></form>, for example a login form, the browser sends the data using the application/x-www-form-urlencoded content type. The body of the request looks like a URL-encoded query string, the same format typically used in GET requests.

Example: username=john_doe&password=pass123.

A POST request with this content type using the fetch API looks like this:

async function sendURLEncoded() {
    const params = new URLSearchParams();
    params.append('username', 'john_doe');
    params.append('email', 'john@example.com');
    params.append('password', 'Secret123');
    params.append('bio', 'This is a multi-line\nbio text.\nIt supports newlines!');

    const response = await fetch('/api/urlencoded/', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/x-www-form-urlencoded',
            'X-CSRFToken': getCSRFToken()
        },
        body: params
    });
    const result = await response.json();
    console.log(result);
}

On the Django side, validation with a Django form is the usual approach:

from django import forms
from django.http import JsonResponse
from django.views.decorators.http import require_http_methods
from .forms import URLEncodedForm


class URLEncodedForm(forms.Form):
    username = forms.CharField(
        max_length=150,
        required=True,
        min_length=3,
    )
    email = forms.EmailField(
        required=True,
        error_messages={
            "required": "Email is required",
            "invalid": "Please enter a valid email address",
        },
    )
    password = forms.CharField(
        required=True,
        min_length=8,
    )
    bio = forms.CharField(
        required=False,
        max_length=500,
        widget=forms.Textarea,
        error_messages={"max_length": "Bio cannot exceed 500 characters"},
    )


@require_http_methods(["POST"])
def handle_urlencoded(request):
    """Handle application/x-www-form-urlencoded requests"""
    form = URLEncodedForm(request.POST)

    if not form.is_valid():
        return JsonResponse(
            {
                "status": "error",
                "errors": form.errors,
                "content_type": request.content_type,
            },
            status=400,
        )

    return JsonResponse(
        {
            "status": "success",
            "form_data": form.cleaned_data,
            "content_type": request.content_type,
        }
    )

Form Data with Files

If your form includes file uploads, the browser switches to multipart/form-data. You enable it in HTML like this:

<form method="post" action="" enctype="multipart/form-data">

A fetch-based upload looks like:

async function sendMultipart() {
    const formData = new FormData();
    formData.append('username', 'john_doe');
    formData.append('email', 'john@example.com');

    const fileInput = document.getElementById('file-input');
    if (fileInput.files[0]) {
        formData.append('avatar', fileInput.files[0]);
    }

    const response = await fetch('/api/multipart/', {
        method: 'POST',
        headers: {
            'X-CSRFToken': getCSRFToken()
        },
        body: formData
    });
    const result = await response.json();
    console.log(result);
}

The Django view works the same way, just with request.FILES added.

from django import forms
from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


class MultipartForm(forms.Form):
    username = forms.CharField(
        max_length=150,
        required=True,
        min_length=3,
    )
    email = forms.EmailField(
        required=True,
    )
    avatar = forms.FileField(
        required=False,
    )

    def clean_avatar(self):
        avatar = self.cleaned_data.get("avatar")
        if avatar:
            if avatar.size > 5 * 1024 * 1024:
                raise forms.ValidationError("File size cannot exceed 5MB")

            allowed_types = ["image/jpeg", "image/png", "image/gif", "image/webp"]
            if avatar.content_type not in allowed_types:
                raise forms.ValidationError(
                    f'Invalid file type. Allowed types: {", ".join(allowed_types)}'
                )
        return avatar



@require_http_methods(["POST"])
def handle_multipart(request):
    """Handle multipart/form-data requests (forms with files)"""
    form = MultipartForm(request.POST, request.FILES)

    if not form.is_valid():
        return JsonResponse(
            {
                "status": "error",
                "errors": form.errors,
                "content_type": request.content_type,
            },
            status=400,
        )

    form_data = {}

    for key, value in form.cleaned_data.items():
        if hasattr(value, "read"):
            form_data[key] = {
                "name": value.name,
                "size": value.size,
                "content_type": value.content_type,
            }
        else:
            form_data[key] = value

    return JsonResponse(
        {
            "status": "success",
            "form_data": form_data,
            "content_type": request.content_type,
        }
    )

JSON String

A very common modern content type is application/json. Single-Page Applications and most JavaScript-heavy frontends rely on it.

Frontend example:

async function sendJSON() {
    const data = { 
        name: "John Doe", 
        email: "john@example.com", 
        age: 30
    };
    const response = await fetch('/api/json/', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'X-CSRFToken': getCSRFToken()
        },
        body: JSON.stringify(data)
    });
    const result = await response.json();
    console.log(result)
}

For validation, you can use Pydantic, which is a nice alternative to Django-REST-Framework serializers.

from django.http import JsonResponse
from django.views.decorators.http import require_http_methods
from pydantic import BaseModel, EmailStr, Field, ConfigDict, ValidationError


class JSONDataSchema(BaseModel):
    model_config = ConfigDict(
        json_schema_extra={
            "example": {
                "name": "John Doe", 
                "email": "john@example.com", 
                "age": 30,
            }
        }
    )

    name: str = Field(min_length=2, max_length=100, description="User's full name")
    email: EmailStr = Field(description="Valid email address")
    age: int = Field(ge=0, le=150, description="User's age")


@require_http_methods(["POST"])
def handle_json(request):
    """Handle application/json requests"""
    try:
        data = json.loads(request.body)
        validated = JSONDataSchema(**data)

        return JsonResponse(
            {
                "status": "success",
                "received": validated.model_dump(),
                "content_type": request.content_type,
            }
        )
    except json.JSONDecodeError:
        return JsonResponse(
            {
                "status": "error",
                "error": "Invalid JSON",
                "content_type": request.content_type,
            },
            status=400,
        )
    except ValidationError as e:
        return JsonResponse(
            {
                "status": "error",
                "errors": e.errors(),
                "content_type": request.content_type,
            },
            status=400,
        )

Newline-Delimited JSON

application/x-ndjson (or simply NDJSON) is an experimental but handy format where each line is a separate JSON object. It’s useful for bulk imports – logs, analytics, and other large datasets.

async function sendNDJSON() {
    const ndjsonData = (
`{"name":"John","age":30}
{"name":"Jane","age":25}
{"name":"Bob","age":35}`
);
    const response = await fetch('/api/ndjson/', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/x-ndjson',
            'X-CSRFToken': getCSRFToken()
        },
        body: ndjsonData
    });
    const result = await response.json();
    console.log(result);
}

The processing logic is standard: split, parse, validate.

import json
from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_ndjson(request):
    """Handle application/x-ndjson requests"""
    try:
        ndjson_content = request.body.decode("utf-8")
        lines = [
            json.loads(line) for line in ndjson_content.strip().split("\n") if line
        ]

        return JsonResponse(
            {
                "status": "success",
                "lines_count": len(lines),
                "data": lines,
                "content_type": request.content_type,
            }
        )
    except json.JSONDecodeError:
        return JsonResponse({"error": "Invalid NDJSON"}, status=400)

Plain Text

Some systems accept plain text via text/plain, especially object storage services or endpoints meant for raw logs or unstructured content.

async function sendTextPlain() {
    const textData = `This is plain text content.
It can span multiple lines.
Line 3 of the content.`;

    const response = await fetch('/api/text/', {
        method: 'POST',
        headers: {
            'Content-Type': 'text/plain',
            'X-CSRFToken': getCSRFToken()
        },
        body: textData
    });
    const result = await response.json();
    console.log(result);
}

The Django view is minimal and functional:

from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_text_plain(request):
    """Handle text/plain requests"""
    text_content = request.body.decode("utf-8")
    return JsonResponse(
        {
            "status": "success",
            "received": text_content,
            "length": len(text_content),
            "content_type": request.content_type,
        }
    )

HTML Text

You can also POST HTML content using text/html, which can be useful for wiki-style pages or HTML editors that save full documents.

async function sendHTML() {
    const htmlData = `<!DOCTYPE html>
<html>
<head><title>Example</title></head>
<body><h1>Hello World!</h1></body>
</html>`;
    const response = await fetch('/api/html/', {
        method: 'POST',
        headers: {
            'Content-Type': 'text/html',
            'X-CSRFToken': getCSRFToken()
        },
        body: htmlData
    });
    const result = await response.json();
    console.log(result);
}

The Django view would accept the data like this (add your own validation):

from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_html(request):
    """Handle text/html requests"""
    html_content = request.body.decode("utf-8")
    return JsonResponse(
        {
            "status": "success",
            "received": html_content,
            "length": len(html_content),
            "content_type": request.content_type,
        }
    )

XML Data

Older or legacy integrations (especially SOAP-based ones) still rely on application/xml.

Here's a JavaScript example:

async function sendXML() {
    const xmlData = `<?xml version="1.0" encoding="UTF-8"?>
<user>
<name>John Doe</name>
<email>john@example.com</email>
<age>30</age>
</user>`;

    const response = await fetch('/api/xml/', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/xml',
            'X-CSRFToken': getCSRFToken()
        },
        body: xmlData
    });
    const result = await response.json();
    console.log(result);
}

And the Django view can parse the XML data into a dictionary like this:

import xml.etree.ElementTree as ET
from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_xml(request):
    """Handle application/xml requests"""
    try:
        xml_content = request.body.decode("utf-8")
        root = ET.fromstring(xml_content)

        data = {
            child.tag: child.text 
            for child in root
        }

        return JsonResponse(
            {
                "status": "success",
                "root_tag": root.tag,
                "data": data,
                "content_type": request.content_type,
            }
        )
    except ET.ParseError:
        return JsonResponse({"error": "Invalid XML"}, status=400)

SVG Image

SVG graphics can be sent as image/svg+xml, since they are XML-based.

async function sendSVG() {
    const svgData = `<svg xmlns="http://www.w3.org/2000/svg" width="100" height="100">
<circle cx="50" cy="50" r="40" stroke="black" stroke-width="3" fill="red" />
</svg>`;
    const response = await fetch('/api/svg/', {
        method: 'POST',
        headers: {
            'Content-Type': 'image/svg+xml',
            'X-CSRFToken': getCSRFToken()
        },
        body: svgData
    });
    const result = await response.json();
    console.log(result);
}

The Django view (add your own validation):

from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_svg(request):
    """Handle image/svg+xml requests"""
    svg_content = request.body.decode("utf-8")
    return JsonResponse(
        {
            "status": "success",
            "received": svg_content,
            "length": len(svg_content),
            "content_type": request.content_type,
        }
    )

Binary Data

For raw binary streams – images, audio, video, PDF, or any unknown byte sequence – the fallback is application/octet-stream.

Here is a JavaScript example of how to post it:

async function sendBinary() {
    const binaryData = new Uint8Array([72, 101, 108, 108, 111]);
    const response = await fetch('/api/binary/', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/octet-stream',
            'X-CSRFToken': getCSRFToken()
        },
        body: binaryData
    });
    const result = await response.json();
    console.log(result);
}

The Django view would be the most straightforward, because request.body is already coming as bytestring:

from django.http import JsonResponse
from django.views.decorators.http import require_http_methods


@require_http_methods(["POST"])
def handle_binary(request):
    """Handle application/octet-stream requests (raw binary data)"""
    binary_data = request.body
    return JsonResponse(
        {
            "status": "success",
            "size": len(binary_data),
            "content_type": request.content_type,
            "first_bytes": list(binary_data[:10]),
        }
    )

Protocol Buffers

Protocol Buffers use application/x-protobuf, and compared to JSON they’re usually around 50% smaller and faster to parse. I won't cover the code in this article, but do your research if you need speed.

Code to Play Around with

The rough distribution of content-type usage in real-world APIs is this:

JSON: ~85–90%
Multipart: ~10–15%
Everything else: <5%

If you want to experiment with all the content types, and see the code in context, here's a repo as a great playground:
https://github.com/archatas/django-post-content-types

Moving a Domain to Another Registrar

Aidas Bendoraitis — Fri, 17 Oct 2025 21:52:41 +0000

The Situation

The domain for my first SaaS project 1st-things-1st.com was registered with GoDaddy. Even though the whole project was already running under my company’s name, I never really bothered to move the domain to my company’s account at Namecheap.

Last week I noticed that the domain was about to expire, and I thought, alright, time to finally do it.

I had never transferred a domain before, so I wasn’t sure how it would go or whether I could pull it off without any downtime. Here’s how it went.

The Story

Namecheap has this feature called “Transfer to Us.” You just follow a few simple steps: request a transfer for your domain, enter a one-time Auth code (also called as EPP - Extensible Provisioning Protocol - code) from another registrar to confirm you’re the owner, and pay for another year.

At GoDaddy’s side, there was a whole confirmation process to make sure I really wanted to transfer the domain and understood it couldn’t be undone. Once that was done, I got the code I needed for Namecheap.

Then came the waiting part. It took almost a week to get the confirmation that the transfer was successful, which was expected according to their help docs. As soon as I got the notification, I jumped into my Namecheap settings to check if everything looked right.

Unfortunately, the domain records weren’t migrated automatically. Only the ownership was transferred. The nameserver settings in Namecheap were still pointing to GoDaddy’s servers.

And since GoDaddy no longer listed the domain or its DNS records, I had to recover those values myself.

I needed to extract the following records for each subdomain:

A – IPv4 Address
CNAME – Canonical Name
MX – Mail Exchange
TXT – Text

I did it from the command line like this:

$ dig 1st-things-1st.com ANY > 1st-things-1st.txt
$ dig www.1st-things-1st.com ANY >> 1st-things-1st.txt
$ dig my.1st-things-1st.com ANY >> 1st-things-1st.txt
$ dig our.1st-things-1st.com ANY >> 1st-things-1st.txt
$ dig apps.1st-things-1st.com ANY >> 1st-things-1st.txt
$ dig analytics.1st-things-1st.com ANY >> 1st-things-1st.txt

Alternatively, you can look up WHOIS information on sites like who.is.

Once I had all the records, I switched to Namecheap’s nameservers (Namecheap BasicDNS) and added everything manually.

The website was down for about 30 to 60 minutes, depending on where in the world the visitor was. Some downtime was unavoidable, but overall, the transfer went through successfully.

The Learnings

The whole thing could have been smoother if I had known what to expect. Here’s how I’d do it next time:

Save all domain records to a local file before starting.
Get the Auth/EPP code from the old registrar.
Request the transfer at the new registrar.
Wait for the transfer to complete.
Pick the best time for downtime and let users know in advance.
Switch to the new nameservers and add all records manually.

Per-Object Permissions for Elasticsearch Lists in Django Websites

Aidas Bendoraitis — Sat, 27 Sep 2025 15:15:42 +0000

The Challenge

Elasticsearch improves the performance of filterable and searchable list views, reducing load times from several seconds to about half a second. It stores list view details from various relations denormalized in a JSON-like structure.

In the Django-based system I’ve been working on, we use django-guardian to set per-object permissions for users or roles managing various items. This means you not only need to check whether a user has general permission to view, change, or delete a type of object, but also whether they have permission to access specific objects.

A major challenge arises when using Elasticsearch for authorized views based on user permissions – how can we check permissions without slowing down the listing too much?

Things We Considered

Here are a few options I compared:

Check all object UUIDs the user can access via django-guardian, then pass those UUIDs to the Elasticsearch search query. This might work with fewer than 100 items, but it doesn’t scale.
Filter the Elasticsearch list first, and then check each item’s UUID against user permissions. With thousands of search results, permission checks become too slow. If I check permissions only for the first page, pagination data becomes inaccurate.
Create a user-permission Elasticsearch index with all item UUIDs accessible to the user, and filter the list by looking up those UUIDs. This makes updating the index tricky, especially for admins and superusers.
For each item, store the list of user IDs and group IDs that can view it, then check the current user against those IDs in the list view. This is the approach I chose, since typically only a handful of users and groups need access to any given item.

Below is the code snippet that implements the last approach.

Our Chosen Approach

We use django-elasticsearch-dsl for indexing Django models in Elasticsearch. The Elasticsearch index document for an item with user IDs and group IDs can look like this:

# items/documents.py
from django.conf import settings
from django_elasticsearch_dsl.registries import registry
from django_elasticsearch_dsl import Document, fields
from guardian.shortcuts import get_users_with_perms, get_groups_with_perms

from .models import Item


@registry.register_document
class ItemDocument(Document):
    users_can_view = fields.KeywordField(multi=True)
    users_can_change = fields.KeywordField(multi=True)
    users_can_delete = fields.KeywordField(multi=True)
    groups_can_view = fields.KeywordField(multi=True)
    groups_can_change = fields.KeywordField(multi=True)
    groups_can_delete = fields.KeywordField(multi=True)

    class Index:
        name = "items"
        settings = {
            "number_of_shards": 1,
            "number_of_replicas": 0,
        }

    class Django:
        model = Item

        fields = [
            "uuid",
            "title",
            "intro",
            "created_at",
            "updated_at",
        ]

        queryset_pagination = 5000

    def prepare(self, instance):
        data = super().prepare(instance)

        data["users_can_view"] = []
        data["users_can_change"] = []
        data["users_can_delete"] = []
        for user, permissions in get_users_with_perms(
            item,
            attach_perms=True,
            with_superusers=True,
            with_group_users=False,
            only_with_perms_in=["view_item", "change_item", "delete_item"],
        ).items():
            if "view_item" in permissions:
                data["users_can_view"].append(user.pk)
            if "change_item" in permissions:
                data["users_can_change"].append(user.pk)
            if "delete_item" in permissions:
                data["users_can_delete"].append(user.pk)

        data["groups_can_view"] = []
        data["groups_can_change"] = []
        data["groups_can_delete"] = []
        for group, permissions in get_groups_with_perms(
            item, attach_perms=True
        ).items():
            for perm in permissions:
                if perm == "view_item":
                    data["groups_can_view"].append(group.pk)
                elif perm == "change_item":
                    data["groups_can_change"].append(group.pk)
                elif perm == "delete_item":
                    data["groups_can_delete"].append(group.pk)

        return data

Next, we need a utility class for paginating Elasticsearch indexes in a way that’s compatible with Django’s default queryset pagination:

# items/utils.py

class ElasticsearchPage:
    """
    Django Paginator-compatible interface for Elasticsearch search results.
    """

    def __init__(self, results, total_count, page_number, items_per_page):
        self.object_list = results
        self.total_count = total_count
        self.number = page_number
        self.paginator = type(
            "Paginator",
            (),
            {
                "count": total_count,
                "num_pages": (total_count + items_per_page - 1) // items_per_page,
                "per_page": items_per_page,
            },
        )()

    def has_previous(self):
        return self.number > 1

    def has_next(self):
        return self.number < self.paginator.num_pages

    def has_other_pages(self):
        return self.paginator.num_pages > 1

    def previous_page_number(self):
        return self.number - 1 if self.has_previous() else None

    def next_page_number(self):
        return self.number + 1 if self.has_next() else None

Finally, the list view checks user IDs and group IDs in the index against the current user’s ID and group memberships:

# items/views.py
from .utils import ElasticsearchPage
from .documents import ItemDocument

@login_required
def item_list(request):
    user_group_pks = list(request.user.groups.values_list("pk", flat=True))

    search_obj = ItemDocument.search()

    perm_filter = Q(
        "bool",
        should=[
            Q("term", users_can_view=request.user.pk),
            Q("terms", groups_can_view=user_group_pks),
        ],
        minimum_should_match=1,
    )

    search_obj = search_obj.query("bool", must=[perm_filter])

    # more search and filtering go here...

    items_per_page = int(request.GET.get("items_per_page", 24))
    page_number = int(request.GET.get("page", 1))
    offset = (page_number - 1) * items_per_page

    total_count = search_obj.count()

    search_obj = search_obj[offset : offset + items_per_page]
    search_results = search_obj.execute()

    page = ElasticsearchPage(
        results=search_results,
        total_count=total_count,
        page_number=page_number,
        items_per_page=items_per_page,
    )
    context = {
        "page": page,
        "items_per_page": items_per_page,
    }
    return render(request, "items/item_list.html", context)

At this point, it’s important to update the index not only when item details change, but also when permissions change.

This can be done by calling the following in the relevant views or form save methods:

from django_elasticsearch_dsl.registries import registry

registry.update(item)

Final Thoughts

Using django-guardian to pre-process or post-process Elasticsearch-filtered lists is inefficient. Instead, permissions should exist directly in the Elasticsearch index. Storing user IDs and group IDs in the items themselves is more practical. Just make sure Elasticsearch is properly secured with SSL/TLS and authentication (username and password) to protect the data from tampering.

User Timezones in Django

Aidas Bendoraitis — Mon, 28 Jul 2025 14:14:42 +0000

When you create a local website, the local time usually matches your country’s timezone, and all visitors see times in that timezone. That’s not a big issue if your country has only one timezone and your audience is local.

But when building a social platform like pybazaar.com, users are international and need to see times in their timezones. In this article, I’ll show you how to handle that in Django.

Time Zone Database

Since version 4.0, Django has used the zoneinfo library for managing timezones, and it used pytz up to version 3.2. Both rely on the IANA Time Zone Database (tzdata). IANA is the same organization that manages the DNS root zone, IP addresses, and other global internet resources.

Install tzdata in your virtual environment as usual:

(venv)$ pip install --upgrade tzdata

Timezone Changes

Timezone information changes several times a year due to:

Daylight Saving Time (DST) adjustments
Political and border changes
Shifts in standard time offset

Daylight Saving Time (DST) was first introduced in 1914 in Canada and later standardized in the U.S. in 1966. When dealing with historic dates before 1966—or future dates with uncertain timezone rules—precise time calculations can be unreliable.

# Before U.S. DST standardization:
old_date = datetime(1960, 6, 15, 12, 0)  

# DST rules may change in the future:
future_date = datetime(2030, 6, 15, 12, 0)

Some timezone changes are driven by politics:

Country splits or mergers — new countries may adopt different timezones.
Regional preferences — states or provinces may change timezone alignment.
Symbolic actions — e.g., North Korea introduced "Pyongyang Time" in 2015 by shifting 30 minutes back to symbolically break from Japan’s colonial legacy.

And countries sometimes adjust their UTC offsets:

Russia — changed its timezone policy multiple times
Venezuela — changed from UTC-4 to UTC-4:30 in 2007, then back in 2016
Samoa — jumped from UTC-11 to UTC+13 in 2011, skipping Dec 30 entirely

Best Practices for Django

Use named timezones, not fixed UTC offsets.
Update tzdata monthly or quarterly.
Test with historic and future dates.
Handle conversion errors gracefully, falling back to UTC.
Store all times in UTC internally.
Convert to user’s timezone only in the UI.
Include tzdata updates in deployment (Docker, Ansible, etc.).

Timezone Management for a Social Platform

For platforms with global users:

Store all datetimes in UTC.
Store each user's preferred timezone.
Convert times on input/output according to the user’s timezone.

1. Enable Timezone Support in Django Settings

Set the default timezone to UTC:

# settings.py
USE_TZ = True
TIME_ZONE = "UTC"  # Store everything in UTC

2. Add a `timezone` Field to the Custom `User` Model

Use a function for dynamic timezone choices, so you don’t need new migrations when the list changes.

def get_timezone_choices():
    import zoneinfo
    return [(tz, tz) for tz in sorted(zoneinfo.available_timezones())]

class User(AbstractUser):
    # ...
    timezone = models.CharField(
        _("Timezone"), max_length=50, choices=get_timezone_choices, default="UTC"
    )

3. Detect Timezone on the Frontend

Add hidden fields in your Login and Signup forms to capture the user’s timezone from their browser:

document.addEventListener('DOMContentLoaded', function () {
    const userTimezone = Intl.DateTimeFormat().resolvedOptions().timeZone;
    const timezoneInput = document.getElementById('id_timezone');
    if (timezoneInput) {
        timezoneInput.value = userTimezone;
    }
});

You can also let users change their timezone manually in account settings.

4. Use a Custom DateTime Field in Forms

This field will convert datetimes between UTC and the user’s local timezone:

import datetime
from zoneinfo import ZoneInfo
from django import forms
from django.utils import timezone
from django.utils.dateparse import parse_datetime

class TimezoneAwareDateTimeField(forms.DateTimeField):
    widget = forms.DateTimeInput(attrs={"type": "datetime-local"})

    def __init__(self, user_timezone=None, *args, **kwargs):
        self.user_timezone = user_timezone
        super().__init__(*args, **kwargs)

    def prepare_value(self, value):
        if value and self.user_timezone:
            try:
                user_tz = ZoneInfo(self.user_timezone)
                if timezone.is_aware(value):
                    value = value.astimezone(user_tz)
            except Exception:
                pass
        return value

    def to_python(self, value):
        if value in self.empty_values:
            return None
        if isinstance(value, datetime.datetime):
            result = value
        elif isinstance(value, datetime.date):
            result = datetime.datetime(value.year, value.month, value.day)
        else:
            try:
                result = parse_datetime(value.strip())
            except ValueError:
                raise forms.ValidationError(
                    self.error_messages["invalid"], code="invalid"
                )
        if not result:
            result = super(forms.DateTimeField).to_python(value)
        if result and self.user_timezone:
            try:
                user_tz = ZoneInfo(self.user_timezone)
                if timezone.is_naive(result):
                    result = result.replace(tzinfo=user_tz)
                result = result.astimezone(ZoneInfo("UTC"))
            except Exception:
                pass
        return result

The type="datetime-local" widget uses the browser’s native date/time picker.

Use the custom field like this:

from django import forms
from django.utils.translation import gettext_lazy as _
from myproject.apps.core.form_fields import TimezoneAwareDateTimeField
from .models import Post

class PostForm(forms.ModelForm):
    class Meta:
        model = Post
        fields = ["title", "content", "published_from"]

    def __init__(self, request, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.request = request
        self.fields["published_from"] = TimezoneAwareDateTimeField(
            label=_("Published from"),
            help_text=_("Enter date and time in your local timezone."),
            required=False,
            user_timezone=self.request.user.timezone,
        )

5. Output Dates and Times in User's Timezone

{% load tz %}
{% with user_timezone=request.user.timezone|default:"UTC" %}
    {{ post.published_from|timezone:user_timezone|date:"j M, Y H:i" }}
{% endwith %}

Other Options

You can also detect the visitor’s timezone in JavaScript and send it via Ajax to be saved in the Django session. Then you can use it even for anonymous users.

Final Words

Timezones aren’t so scary if you follow Django’s best practices:

Store all times in UTC.
Update tzdata regularly.
Use the user's timezone only at input/output stages.
Detect the user’s timezone via JavaScript—no need to ask them manually.

This keeps your website accurate, user-friendly, and ready for global audiences.

Cover photo by Andrey Grushnikov

Using Google Consent Mode v2 on a Django Website

Aidas Bendoraitis — Sun, 06 Jul 2025 12:00:00 +0000

A decade ago, adding website analytics was simple: you’d just paste a JavaScript snippet from Google Analytics, and that was it.

But things changed. As people became more concerned about their privacy, countries introduced strict privacy laws—like GDPR in the European Union, PIPEDA in Canada, and APPI in Japan. These laws gave users more control over their data and placed new responsibilities on us developers.

One of those responsibilities is showing Cookie Consent banners and respecting users’ choices before loading any tracking scripts.

Today, if you want to add Google Analytics to your website, it’s not just about copying a script. You need to understand Google Tag Manager (GTM)—a tool that lets you manage what scripts run on your site and when, using a web-based dashboard.

When you add Google Analytics as a Google tag through GTM, it doesn’t automatically start collecting data. It waits for a signal called Consent Mode, which tells Google whether the user has accepted or denied tracking. This signal must be sent from your website to GTM as an event.

That’s where your Cookie Consent widget comes in. For Django websites, I created Django GDPR Cookie Consent, which lets you manage user consent and send the proper signals to Google.

In this article, I’ll show you how to make all three—Django GDPR Cookie Consent, Google Tag Manager, and Google Consent Mode v2—work together smoothly.

1. Set up Google Analytics

Go to Google Analytics and set up a property for your website. You'll need to know your Google Analytics 4 Measurement ID, e.g., G-XXXXXXX for Google Tag Manager.

2. Set up Google Tag Manager

You’ll use the Google Tag Manager (GTM) web interface — to add and manage all your tracking scripts (called tags). You won’t have to edit your website code every time you want to change something — GTM handles it all from one place.

In your container settings, check Enable consent overview to make sure that all your tags have cookie consent setting configured.

Check the GTM container id bounded to your website, e.g., GTM-XXXXXXX. You'll need it later for the scripts.

Here’s how to configure GTM to load scripts only after the user gives consent, using the signals from Google Consent Mode.

GA4 Configuration Tag

This tag initializes Google Analytics 4 (GA4) tracking.

Tag Type: Google Tag
Measurement ID: Your GA4 ID (G-XXXXXXX)
Consent Settings: Require additional consent for tag to fire: analytics_storage
Trigger: Consent Initialization – All Pages (this runs very early, before other tags)

This setup allows GA4 to start in a consent-aware way. It reads the default denied state at page load and will automatically adjust when the user accepts cookies later.

GA4 Event Tags

These are additional GA4 tags to track specific actions (like form submissions or button clicks).

Tag Type: Google Analytics: GA4 Event
Consent Settings: Require additional consent for tag to fire: analytics_storage
Trigger: Choose based on the action (e.g., form submit, button click)

You don’t need to check for consent manually here — GA4 automatically tracks or holds data based on the user's consent provided through Google Consent Mode.

Google Ads Tags

If you are using Google Ads, set these for conversion tracking and remarketing (showing ads to users who visited your site):

Tag Type: Google Ads Conversion Tracking or Google Ads Remarketing
Consent Settings: Require additional consent for tag to fire: ad_storage, ad_personalization, ad_user_data
Trigger: Set this to fire after a successful action (like a purchase or sign-up)

These tags respect consent choices, like whether the user allowed ad_storage.

3. Set up Django GDPR Cookie Consent

Download and install the package

Get Django GDPR Cookie Consent from Gumroad.

Create a directory private_wheels/ in your project's repository and add the wheel file of the app there.

Link to this file in your requirements.txt:

Django==5.2
file:./private_wheels/django_gdpr_cookie_consent-4.1.2-py2.py3-none-any.whl

Install the pip requirements from the requirements.txt file into your project's virtual environment:

(venv)$ pip install -r requirements.txt

Add the app to INSTALLED_APPS

INSTALLED_APPS = [
    # …
    "gdpr_cookie_consent",
    # …
]

Add the context processor

Add gdpr_cookie_consent to context processors in your Django settings:

TEMPLATES = [
    {
        # …
        "OPTIONS": {
            "context_processors": [
                # …
                "gdpr_cookie_consent.context_processors.gdpr_cookie_consent",
            ],
        },
    },
]

Add URL path to urlpatterns

from django.urls import path, include

urlpatterns = [
    # …
    path(
        "cookies/",
        include("gdpr_cookie_consent.urls", namespace="cookie_consent"),
    ),
    # …
]

Prepare cookie consent configuration

Create COOKIE_CONSENT_SETTINGS configuration in your Django project settings with these cookie sections:

Essential (strictly necessary) for cookies related to Django sessionid, CSRF token, and cookie consent,
Analytics (optional) for website usage statistics with Google Analytics,
Marketing (optional) for tracking cross-website ad statistics with Google Ads.

from django.utils.translation import gettext_lazy as _

COOKIE_CONSENT_SETTINGS = {
    "base_template_name": "base.html",
    "description_template_name": "gdpr_cookie_consent/descriptions/what_are_cookies.html",
    "dialog_position": "center",
    "consent_cookie_max_age": 60 * 60 * 24 * 30 * 6,
    "sections": [
        {
            "slug": "essential",
            "title": _("Essential Cookies"),
            "required": True,
            "preselected": True,
            "summary": _(
                "These cookies are always on, as they’re essential for making this website work, and making it safe. Without these cookies, services you’ve asked for can’t be provided."),
            "description": _(
                "These cookies are always on, as they’re essential for making this website work, and making it safe. Without these cookies, services you’ve asked for can’t be provided."),
            "providers": [
                {
                    "title": _("This website"),
                    "cookies": [
                        {
                            "cookie_name": "sessionid",
                            "duration": _("2 Weeks"),
                            "description": _(
                                "Session ID used to authenticate you and give permissions to use the site."),
                            "domain": ".example.com",
                        },
                        {
                            "cookie_name": "csrftoken",
                            "duration": _("Session"),
                            "description": _(
                                "Security token used to ensure that no hackers are posting forms on your behalf."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                        {
                            "cookie_name": "cookie_consent",
                            "duration": _("6 Years"),
                            "description": _("Settings of Cookie Consent preferences."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                    ]
                },
            ],
        },
        {
            "slug": "analytics",
            "title": _("Analytics Cookies"),
            "required": False,
            "summary": _(
                "These cookies help us analyse how many people are using this website, where they come from and how they're using it. If you opt out of these cookies, we can’t get feedback to make this website better for you and all our users."),
            "description": _(
                "These cookies help us analyse how many people are using this website, where they come from and how they're using it. If you opt out of these cookies, we can’t get feedback to make this website better for you and all our users."),
            "providers": [
                {
                    "title": _("Google Analytics"),
                    "description": _("Google Analytics is used to track website usage statistics."),
                    "description_template_name": "",
                    "cookies": [
                        {
                            "cookie_name": "_ga",
                            "duration": _("2 Years"),
                            "description": _("Used to distinguish users."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                        {
                            "cookie_name": "_gid",
                            "duration": _("24 Hours"),
                            "description": _("Used to distinguish users."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                        {
                            "cookie_name": "_ga_*",
                            "duration": _("2 Years"),
                            "description": _("Used to persist session state."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                        {
                            "cookie_name": "_gat_gtag_UA_*",
                            "duration": _("1 Minute"),
                            "description": _("Stores unique user ID."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                    ]
                },
            ],
        },
        {
            "slug": "marketing",
            "title": _("Marketing Cookies"),
            "required": False,
            "summary": _(
                "These cookies are set by our advertising partners to track your activity and show you relevant ads on other sites as you browse the internet."),
            "description": _(
                "These cookies are set by our advertising partners to track your activity and show you relevant ads on other sites as you browse the internet."),
            "providers": [
                {
                    "title": _("Google Ads"),
                    "description": _("These cookies are related to Google Ads conversion tracking."),
                    "description_template_name": "",
                    "cookies": [
                        {
                            "cookie_name": "_gac_gb_*",
                            "duration": _("90 Days"),
                            "description": _(
                                "Contains campaign related information. If you have linked your Google Analytics and Google Ads accounts, Google Ads website conversion tags will read this cookie unless you opt-out."),
                            "description_template_name": "",
                            "domain": ".example.com",
                        },
                    ]
                },
            ],
        },
    ]
}

Include the widget to your `base.html` template

Load the CSS somewhere in the <head> section:

<link href="{% static 'gdpr-cookie-consent/css/gdpr-cookie-consent.css' %}" rel="stylesheet" />

Include the widget just before the closing </body> tag:

{% include "gdpr_cookie_consent/includes/cookie_consent.html" %}

Link to the cookie management view, for example, in the website's footer:

{% url "cookie_consent:cookies_management" as cookie_management_url %}
<a href="{{ cookie_management_url }}" rel="nofollow">
    {% trans "Manage Cookies" %}
</a>

Add GTM and Google Consent Mode snippets to your `base.html` template:

Add these two scripts in the <head> section:

<script nonce="{{ request.csp_nonce }}">
   window.dataLayer = window.dataLayer || [];
   function gtag(){dataLayer.push(arguments);}

   gtag('consent', 'default', {
      'analytics_storage': '{% if "analytics" in cookie_consent_controller.checked_sections %}granted{% else %}denied{% endif %}',
      'ad_storage': '{% if "marketing" in cookie_consent_controller.checked_sections %}granted{% else %}denied{% endif %}',
      'ad_user_data': '{% if "marketing" in cookie_consent_controller.checked_sections %}granted{% else %}denied{% endif %}',
      'ad_personalization': '{% if "marketing" in cookie_consent_controller.checked_sections %}granted{% else %}denied{% endif %}',
      'functionality_storage': 'denied',
      'personalization_storage': 'denied',
      'security_storage': 'granted' // usually okay to grant by default
   });
</script>

<!-- Google Tag Manager (head) -->
<script nonce="{{ request.csp_nonce }}">(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-XXXXXXX');</script>

Add this iframe in the beginning of the <body> tag:

<!-- Google Tag Manager (body) -->
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-XXXXXXX"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>

Replace GTM-XXXXXXX with your real GTM container ID.

Create JavaScript custom event handlers for granting and denying consent

Create a custom JavaScript file main.js and include it in the base.html template somewhere before the closing </body> tag:

<script src="{% static 'site/js/main.js' %}"></script>

In the JavaScript file, add these custom event handlers that will grant or deny cookie consent at Google–that's a new feature in Django GDPR Cookie Consent v4:

document.addEventListener('grantGDPRCookieConsent', (e) => {
    console.log(`${e.detail.section} cookies granted`);
    if (e.detail.section === 'analytics') {
        gtag('consent', 'update', {
            'analytics_storage': 'granted'
        });
        dataLayer.push({'event': 'analytics_consent_granted'});
    } else if (e.detail.section === 'marketing') {
        gtag('consent', 'update', {
            'ad_storage': 'granted',
            'ad_user_data': 'granted',
            'ad_personalization': 'granted'
        });
        dataLayer.push({'event': 'marketing_consent_granted'});
    }
});

document.addEventListener('denyGDPRCookieConsent', (e) => {
    console.log(`${e.detail.section} cookies denied`);
    if (e.detail.section === 'analytics') {
        gtag('consent', 'update', {
            'analytics_storage': 'denied'
        });
        dataLayer.push({'event': 'analytics_consent_denied'});
    } else if (e.detail.section === 'marketing') {
        gtag('consent', 'update', {
            'ad_storage': 'denied',
            'ad_user_data': 'denied',
            'ad_personalization': 'denied'
        });
        dataLayer.push({'event': 'marketing_consent_denied'});
    }
});

Check if your setup is correct

Check the correctness of your configuration with the following:

(venv)$ python manage.py check gdpr_cookie_consent

4. Test and debug with GTM preview mode

Deploy your project to production.
Install Tag Assistant Chrome extension.
Enable Preview in Google Tag Manager dashboard.
Visit your website via the preview mode.
Accept cookies.
You should see these events:
- analytics_consent_granted – GA4 tag should fire.
- marketing_consent_granted – Google Ads tag should fire.
Check which tags fired — they should match the user’s choices.
The Consent tab of each event should show the correct preferred consent choices.
Google Analytics realtime report should track you only when the consent was given.

How it works

When a user visits your website, the default cookie consent mode is set based on their previously saved preferences. If it's their first visit, consent will default to “denied.”

When the user sets or updates their cookie preferences—via the modal dialog or the preferences form—your consent widget will fire custom JavaScript events grantGDPRCookieConsent or denyGDPRCookieConsent available since Django GDPR Cookie Consent v4.

Your JavaScript handler will listen for these events, update the Google Consent Mode accordingly, and send the updated values to Google Tag Manager.

Based on those values, Google Tag Manager will decide whether to activate tracking tags such as Google Analytics and Google Ads. These tags can then track usage statistics and, if allowed, ad-related cross-site behavior.

Final words

Now you should be all set. Google Analytics should respect user's privacy based on the choices in Cookie Consent widget, provided by Django GDPR Cookie Consent. Your website will be compliant with Google Consent Mode and will fire analytics and marketing tags only after consent.

Rate Limiting for Django Websites

Aidas Bendoraitis — Fri, 04 Jul 2025 08:18:31 +0000

Sometimes, certain pages of a Django website might receive unwanted traffic from crawlers or malicious bots. These traffic spikes consume server resources and can make the website unusable for legitimate users. In this article, I will explore Nginx’s rate-limiting capabilities to prevent such performance issues.

What is rate limiting, and why use it?

A typical Django website is deployed using a Gunicorn (or Uvicorn for ASGI) application server, with an Nginx web server in front of it. When a request comes to Nginx, it goes through various checks, gets filtered by domain, protocol, and path, and is finally passed to Gunicorn. Gunicorn then runs Django, which parses the URL and returns a response from the appropriate view.

Nginx rate limiting allows you to limit how often certain pages (based on URL paths) or all Django endpoints can be accessed. It prevents quick reloading or scripted attacks that flood your site with requests. This is especially useful for pages with forms, faceted list views, REST APIs, or GraphQL endpoints.

A typical rate-limiting configuration in Nginx defines a zone with a specific memory size (in megabytes), a rate (requests per second or per minute), and one or more locations that apply that zone using options like burst and nodelay. When the rate limit is exceeded, Nginx returns a 429 Too Many Requests response.

You can imagine the burst as a funnel. The requests are like small balls coming in quickly. The first request passes through immediately, and the others get stacked in the funnel. Any requests that don’t fit into the funnel are rejected with a 429 response. The rest are either delayed or processed immediately, depending on the configuration. If more requests come in and there’s space in the funnel, they’re stacked and processed according to the defined rate.

Practical example

Here is an example configuration that limits list views to 1 request per second, allowing up to 2 additional requests in a short burst before rejecting further ones. It also limits POST requests to authentication paths to 10 per minute, allowing bursts of up to 5 requests at a time. And a generous fallback for other URL paths. All excess requests beyond these limits will be rejected. Each zone has 10 MB of storage for metadata such as IP addresses, timestamps, and the current state of the funnel.

map $request_method $limit_key {
    POST $binary_remote_addr;
    default "";
}

limit_req_zone $binary_remote_addr zone=list_pages:10m rate=1r/s;
limit_req_zone $limit_key zone=auth_conditional:10m rate=10r/m;
limit_req_zone $binary_remote_addr zone=global_fallback:10m rate=20r/s;


server {
    listen 443 ssl;
    server_name www.pybazaar.com;
    charset utf-8;

    # ...

    location ~* "^/(|resources|profiles|opportunities|buzz)/?$" {
        limit_req zone=list_pages burst=2 nodelay;
        limit_req_status 429;

        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_buffering off;
        proxy_redirect off;
        proxy_read_timeout 300s;
    }

    location ~* "^/(signup|login)/?$" {
        limit_req zone=auth_conditional burst=5 nodelay;
        limit_req_status 429;
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_buffering off;
        proxy_redirect off;
        proxy_read_timeout 300s;
    }

    location / {
        limit_req zone=global_fallback burst=40 nodelay;
        limit_req_status 429;
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_buffering off;
        proxy_redirect off;
        proxy_read_timeout 300s;
    }

    # ...
}

The $binary_remote_addr setting limits requests per client IP address. Alternatively, you can use $server_name to apply rate limiting globally. Other variables can also be used.

Checking if it worked

Once you have this set up, you can test the rate limiting using the following bash script:

#!/usr/bin/env bash

for i in {1..20}; do
  {
    status=$(curl -w "%{http_code}" -o /dev/null -s \
      https://www.pybazaar.com/opportunities/)
    echo "Request $i: $status"
  } &
done
wait

This script will output the HTTP status codes for 20 requests sent simultaneously, for example:

Request 7: 429
Request 4: 429
Request 8: 429
Request 3: 429
Request 19: 429
Request 15: 429
Request 9: 429
Request 13: 429
Request 6: 429
Request 20: 429
Request 18: 429
Request 10: 429
Request 17: 429
Request 11: 429
Request 16: 429
Request 14: 429
Request 12: 429
Request 5: 200
Request 1: 200
Request 2: 200

Keep in mind that the order of the requests and which ones succeed or fail won’t be in a simple sequence because Internet traffic takes different paths on its way to and from the server.

Next steps

The next step could be setting up the fail2ban service to ban IP addresses that receive the 429 response code too frequently. Should I write an article about that?

Conclusion

Rate limiting with Nginx is an effective way to protect your Django website from traffic spikes caused by crawlers or malicious bots. By setting limits on requests per second or minute—using zones and bursts—you can control how many requests each IP can make to different parts of your site. This helps prevent server overload and keeps the website responsive for real users.

Algorithms: Learning One's Learnings

Aidas Bendoraitis — Thu, 01 May 2025 22:26:48 +0000

When I studied algorithms more than twenty years ago, I saw their general usefulness, but since then, I have never had to use them in the real world because they had already been implemented by database management services or built-in functionalities of most programming languages that I used.

Interviewer: Can you sort this list by size?

Interviewee: Absolutely.
/types confidently.../
l.sort()

There came a time when I had to remind myself of sorting algorithms. At 1st things 1st, I was about to implement a feature that lets users prioritize their options by comparing them two at a time. I thought that the Bubble Sort algorithm would fit perfectly.

So, I built the UI using ReactJS and Django for the backend to let users compare options and sort their priorities.

However, I soon noticed a problem: for longer lists, the process became quite time-consuming for users.

One day, I remembered something about algorithm complexity (also known as Big O Notation) and realized that Bubble Sort isn't the most efficient choice for this task.

In computer science, Big O notation describes how the performance of an algorithm grows relative to the size of the input. It's usually written like O(value), where value is a function of n, and n is the number of items the algorithm processes.

For example:

Finding the first item in a list or a key in a dictionary of n items is O(1).
Summing all items by iterating through a list is O(n).
Using a triple nested loop over the list would result in O(n³).

Here is an overview of most common complexities:

Complexity	Name	Meaning and Example
`O(1)`	Constant	Always the same time, no matter input size, e.g. accessing an array item
`O(log n)`	Logarithmic	Doubles size, just +1 step, e.g. Binary Search
`O(n)`	Linear	Time grows with size, e.g. loop over list
`O(n log n)`	Linearithmic	"Almost linear", but a bit slower, e.g. Merge Sort
`O(n²)`	Quadratic	Time explodes with size, e.g. nested loops as in Bubble sort
`O(2ⁿ)`	Exponential	Be careful — grows crazy fast, e.g. Recursive Fibonacci
`O(n!)`	Factorial	Nightmare mode, e.g. solving Traveling Salesman by brute force

Understanding the Big O practically

Bubble Sort is an algorithm with a time complexity of O(n²). That means: if you have a list of items, the algorithm needs to compare each item against many others, using two nested loops.

To sort a list of 5 items, the user might need up to 25 comparisons.
To sort 10 items, up to 100 comparisons.
To sort 50 items, up to 2,500 comparisons.

Not ideal.

I asked ChatGPT to help evaluate sorting algorithms based on their complexity, and Quick Sort came out as a much better choice. Its complexity is O(n log n), where log n (base 2) roughly means "how many times you can divide n by 2 until you reach 1."

To sort 5 items, Quick Sort would need around 12 comparisons.
To sort 10 items, around 34 comparisons.
To sort 50 items, around 282 comparisons.

That's still some work, but much more manageable.

From a usability perspective, though, it’s even better to let users drag and drop items manually or to offer AI-suggested orderings. So I implemented those options too.

Algorithm complexities in action

Below is a quick cheat sheet with complexities from the fastest to the slowest:

Sorting algorithms

Algorithm	Best Case	Average Case	Worst Case	Notes
Timsort	`O(n)`	`O(n log n)`	`O(n log n)`	Used in Python, Java
Merge Sort	`O(n log n)`	`O(n log n)`	`O(n log n)`	Stable, reliable
Heap Sort	`O(n log n)`	`O(n log n)`	`O(n log n)`	In-place
Quick Sort	`O(n log n)`	`O(n log n)`	`O(n²)`	Fastest on average
Shell Sort	`O(n log n)`	`O(n¹·⁵)`	`O(n²)`	Improved insertion
Insertion Sort	`O(n)`	`O(n²)`	`O(n²)`	Good for small/mostly sorted data
Selection Sort	`O(n²)`	`O(n²)`	`O(n²)`	Simple but slow
Bubble Sort	`O(n)`	`O(n²)`	`O(n²)`	Only good for tiny lists

Search algorithms

Algorithm	Best Case	Average Case	Worst Case	Notes
Hash Table Search	`O(1)`	`O(1)`	`O(n)`	Worst case rare (collisions)
Binary Search	`O(1)`	`O(log n)`	`O(log n)`	Needs sorted data
Jump Search	`O(1)`	`O(√n)`	`O(√n)`	For sorted arrays
Ternary Search	`O(1)`	`O(log n)`	`O(log n)`	Split into three parts
Linear Search	`O(1)`	`O(n)`	`O(n)`	No sorting needed, but slow

Encryption algorithms

Algorithm	Complexity	Notes
AES (Advanced Encryption Standard)	`O(1)` per block	Very fast, symmetric key
ChaCha20	`O(1)` per block	Modern stream cipher, very fast
RSA Encryption	`O(n³)`	Public key, slow for big data
Elliptic Curve Cryptography (ECC)	`O(n²)`	Faster than RSA at similar security
Diffie-Hellman Key Exchange	`O(n³)`	Secure key exchange, slower

Just for the hunger of interest

Python, MySQL, and PostgreSQL use these algorithms behind the scenes:

System	Sorting Algorithm	Notes
Python `sort()` and `sorted()`	Timsort	Hybrid of Merge Sort + Insertion Sort
MySQL `ORDER BY`	Quicksort (small datasets), Merge Sort or Filesort (large datasets)	Depends on storage engine (InnoDB, etc.)
PostgreSQL `ORDER BY`	Quicksort (small data), Heapsort + Mergesort for bigger sets	Intelligent memory-aware switching

Final words

So if you ever have to solve a development issue algorithmically instead of using default libraries — or even when choosing an algorithm from a library — remember the Big O notation. Sometimes the difference can save milliseconds or seconds of time, and sometimes, when users' input is involved, it can save minutes or even hours.

Cover picture by Steve Johnson

How to Use GraphQL in Django with Elasticsearch

Aidas Bendoraitis — Sun, 26 Jan 2025 22:48:35 +0000

When you need to use GraphQL with Django, a common practice is to use Graphene-Django, which allows you to query Django models. However, directly querying Django models might be too slow, especially when you have many complex relations. To speed that up, you can add one more layer of abstraction and index your Django models to an Elasticsearch server. In this article, I will show you how to create a GraphQL interface for querying posts from the Elasticsearch server.

Elasticsearch Server

The easiest way to install the Elasticsearch server locally is using EVM - Elasticsearch Version Manager:

$ sudo curl -o /usr/local/bin/evm https://raw.githubusercontent.com/duydo/evm/master/evm
$ sudo chmod +x /usr/local/bin/evm

Then you can install Elasticsearch 7.17.27 (or other version) with EVM, as follows:

$ evm install 7.17.27

By default, the Elasticsearch server will raise a warning if you don't have HTTPS or basic authentication. You can change one setting to turn off those warnings for localhost. To do so, add a line to ~/.evm/elasticsearch-7.17.27/config/elasticsearch.yml:

xpack.security.enabled: false

Now, you can start the server with:

$ evm start

If you need more Elasticsearch versions on the same computer, you can easily install and switch between them.

Python Packages

Install the latest packages of Django Elasticsearch DSL for connecting to Elasticsearch server, and Graphene-Django for GraphQL implementation:

(venv)$ pip install Django
(venv)$ pip install django-elasticsearch-dsl
(venv)$ pip install graphene-django

Django Settings

Add the installed apps to INSTALLED_APPS in the myproject/settings.py. Also, add a few more settings that we will need in this exercise:

INSTALLED_APPS = [
    # Contributed Django apps
    # ...

    # Third-party apps
    "django_elasticsearch_dsl",
    "graphene_django",

    # Local apps
    "posts",
]

PROJECT_NAME = "myproject"
ENVIRONMENT = "dev"

ELASTICSEARCH_DSL = {
    "default": {
        "hosts": "http://localhost:9200",
    }
}

GRAPHENE = {
    "SCHEMA": "elasticsearch_graphql.schema.schema",
}

Models and Administration for the Posts App

Let's create a posts app with two models: Category and Post:

The posts/models.py file will look like this:

from django.db import models


class Category(models.Model):
    title = models.CharField("Title", max_length=255)
    slug = models.SlugField("Slug", max_length=255)

    class Meta:
        verbose_name = "Category"
        verbose_name_plural = "Categories"
        ordering = ["title"]

    def __str__(self):
        return self.title


class Post(models.Model):
    title = models.CharField("Title", max_length=255)
    slug = models.SlugField("Slug", max_length=255)
    content = models.TextField("Content")
    categories = models.ManyToManyField(Category)

    class Meta:
        verbose_name = "Post"
        verbose_name_plural = "Posts"
        ordering = ["title"]

    def __str__(self):
        return self.title

Also let's create administration posts/admin.py to be able to add some entries easily:

from django.contrib import admin
from .models import Category, Post


@admin.register(Category)
class CategoryAdmin(admin.ModelAdmin):
    list_display = ["title", "slug"]
    prepopulated_fields = {"slug": ["title"]}


@admin.register(Post)
class Post(admin.ModelAdmin):
    list_display = ["title", "slug"]
    list_filter = ["categories"]
    filter_horizontal = ["categories"]
    prepopulated_fields = {"slug": ["title"]}

Create the database schema and the superuser, then run the local development server with the following Django management commands:

(venv)$ python makemigrations
(venv)$ python migrate
(venv)$ python createsuperuser
(venv)$ python runserver

Add some posts and categories in the administration:

http://localhost:8000/admin/

Elasticsearch Documents for the Posts

Let's create PostDocument definition for the Elasticsearch index of the Post model and its categories at posts/documents.py:

import graphene

from django.conf import settings
from django_elasticsearch_dsl import Document
from django_elasticsearch_dsl.registries import registry
from django_elasticsearch_dsl import fields
from .models import Post


@registry.register_document
class PostDocument(Document):
    title = fields.TextField()
    slug = fields.KeywordField()
    content = fields.TextField()
    categories = fields.NestedField(
        properties={
            "title": fields.TextField(),
            "slug": fields.KeywordField(),
        }
    )

    class Index:
        name = f"{settings.PROJECT_NAME}_{settings.ENVIRONMENT}_posts"
        settings = {"number_of_shards": 1, "number_of_replicas": 0}

    class Django:
        model = Post

        ignore_signals = False
        auto_refresh = True
        queryset_pagination = 5000

    def prepare(self, instance):
        data = super().prepare(instance)

        # Prepare categories field
        data["categories"] = [
            {
                "title": category.title,
                "slug": category.slug,
            }
            for category in instance.categories.all()
        ]
        return data

To index the posts on the Elasticsearch server, run the following:

(venv)$ python manage.py search_index --rebuild

Let's see if the entries were indexed and can be retrieved.

At first, run the Django shell:

(venv)$ python manage.py shell

Then try to list out items from the Elasticsearch index:

>>> from posts.documents import PostDocument
>>> for doc in PostDocument.search():
...    print(doc.title)

It will list up to 10 posts (that's the default pagination) from the index:

Creating AI-based Summaries in a Django Website
Creating Open Graph Images in Django for Improved Social Media Sharing
HTTPS for Django Development Environment

GraphQL Types and Queries for the Posts App

Now, create the GraphQL schema posts/schema.py for the categories and posts, as follows:

import operator
from functools import reduce

import graphene
from .documents import PostDocument


class ElasticsearchObjectType(graphene.ObjectType):
    @classmethod
    def from_elasticsearch(cls, elasticsearch_document):
        instance = cls()
        instance.elasticsearch_document = elasticsearch_document
        return instance


class ElasticsearchCategoryType(ElasticsearchObjectType):
    title = graphene.String()
    slug = graphene.String()

    def resolve_title(self, info):
        return self.elasticsearch_document.title

    def resolve_slug(self, info):
        return self.elasticsearch_document.slug


class ElasticsearchPostType(ElasticsearchObjectType):
    title = graphene.String()
    slug = graphene.String()
    content = graphene.String()
    categories = graphene.List(of_type=ElasticsearchCategoryType)

    def resolve_title(self, info):
        return self.elasticsearch_document.title

    def resolve_slug(self, info):
        return self.elasticsearch_document.slug

    def resolve_content(self, info):
        return self.elasticsearch_document.content

    def resolve_categories(self, info):
        return [
            ElasticsearchCategoryType.from_elasticsearch(
                elasticsearch_document=category
            )
            for category in self.elasticsearch_document.categories or []
        ]


class Query(object):
    elasticsearch_posts = graphene.List(
        ElasticsearchPostType,
        limit=graphene.Int(),
        offset=graphene.Int(),
        search=graphene.String(),
        categories=graphene.List(of_type=graphene.String),
    )

    def resolve_elasticsearch_posts(
        self,
        info,
        limit=10,
        offset=0,
        search=None,
        categories=None,
    ):
        from elasticsearch_dsl.search import Q

        search_obj = PostDocument.search()

        queries = []
        if search:
            queries.append(
                Q(
                    "multi_match",
                    query=search,
                    fields=[
                        "title^5",
                        "slug^4",
                        "content^3",
                    ],
                    type="best_fields",
                    tie_breaker=0.3,
                    fuzziness="AUTO",
                    fuzzy_transpositions=True,
                    lenient=True,
                )
            )

        if categories:
            queries.append(
                Q(
                    "nested",
                    path="categories",
                    query=Q("terms", categories__slug=categories),
                )
            )

        if queries:
            # All filters AND-ed
            search_obj = search_obj.query(reduce(operator.iand, queries))

        search_obj = search_obj[offset : offset + limit if limit else None]

        return [
            ElasticsearchPostType.from_elasticsearch(elasticsearch_document=result)
            for result in search_obj.execute()
        ]

Here, we have an ElasticsearchObjectType that provides a custom method from_elasticsearch to initiate GraphQL objects from Elasticsearch instead of Django models.

The query for elasticsearch_posts has two filters: by a search string and by a list of OR-ed category slugs. The search query looks into title, slug, and content fields with different boosting weights and searches for the best match. The fuzziness set to "AUTO" allows mistakes in the search string by two letters.

At the project level, create Graphene Relay - a schema class that uses a Query class inheriting from all the Query classes from all Django apps. The file can be located at myproject/schema.py and have this content:

import graphene
import posts.schema

class Query(posts.schema.Query, graphene.ObjectType):
    pass

schema = graphene.Schema(query=Query)

Finally, enable the GraphQL endpoint in the myproject/urls.py:

from django.contrib import admin
from django.urls import path
from django.views.decorators.csrf import csrf_exempt
from graphene_django.views import GraphQLView

urlpatterns = [
    path("graphql/", csrf_exempt(GraphQLView.as_view(graphiql=True))),
    path("admin/", admin.site.urls),
]

Checking the GraphQL Queries

Navigate to http://localhost:8000/graphql/

Graphene-Django comes with a GraphiQL user interface, which, when accessed from a browser, allows the user to query GraphQL and see results in the same window:

A search query for "AI" with categories "Intermediate" or "Advanced" can look like:

query FilteredPosts {
  elasticsearchPosts(
    search: "AI"
    categories: ["intermediate", "advanced"], 
    limit: 10, 
    offset: 0,
  ) {
    title
    slug
    categories {
      title
      slug
    }
  }
}

It will produce results in JSON just like this:

{
  "data": {
    "elasticsearchPosts": [
      {
        "title": "Creating AI-based Summaries in a Django Website",
        "slug": "creating-ai-based-summaries-in-a-django-website",
        "categories": [
          {
            "title": "Basics",
            "slug": "basics"
          },
          {
            "title": "Django",
            "slug": "django"
          },
          {
            "title": "Gemini",
            "slug": "gemini"
          },
          {
            "title": "Intermediate",
            "slug": "intermediate"
          },
          {
            "title": "Large Language Models",
            "slug": "large-language-models"
          },
          {
            "title": "PyBazaar",
            "slug": "pybazaar"
          },
          {
            "title": "Simplemind",
            "slug": "simplemind"
          }
        ]
      },
      {
        "title": "HTTPS for Django Development Environment",
        "slug": "https-for-django-development-environment",
        "categories": [
          {
            "title": "DevOps",
            "slug": "devops"
          },
          {
            "title": "Development",
            "slug": "development"
          },
          {
            "title": "Django",
            "slug": "django"
          },
          {
            "title": "HTTPS",
            "slug": "https"
          },
          {
            "title": "Intermediate",
            "slug": "intermediate"
          },
          {
            "title": "SSL",
            "slug": "ssl"
          },
          {
            "title": "TLS",
            "slug": "tls"
          }
        ]
      }
    ]
  }
}

Conclusion

Using Elasticsearch with GraphQL is not trivial, but it provides excellent performance, especially for complex data structures. The best way to implement that in Django that I know of is combining Django Elasticsearch DSL and Graphene-Django.

Cover photo Angela Roma

Creating Open Graph Images in Django for Improved Social Media Sharing

Aidas Bendoraitis — Sat, 11 Jan 2025 01:49:29 +0000

Although social media algorithms usually discourage posting links so that users stay as long as possible on the network, people often still post links below an introductory post as a comment or reply. Normal links to websites on social media look pretty dull unless you add open-graph images representing that link.

In this article, I will show you how you can generate open-graph images for a Django website using web rendering from HTML and CSS. I rely on this technique to generate Open Graph previews for links from DjangoTricks, 1st things 1st, and PyBazaar.

What is an Open Graph?

Facebook created the Open Graph protocol to allow websites to provide rich representation of any web page. Although it has specifics for websites, articles, profiles, music, and video, the common use case is to have a preview image with a title for social feeds. Open Graph previews work with most well-known social networks, including Facebook, Threads, LinkedIn, Mastodon, and Blue Sky. Open Graph tags are HTML meta tags that you put in the HEAD section, e.g.:

<meta property="og:type" content="website" />
<meta property="og:url" content="{{ WEBSITE_URL }}{{ request.path }}" />
<meta property="og:title" content="{{ profile.user.get_full_name }}" />
{% if profile.open_graph_image %}
    <meta property="og:image" content="{{ profile.open_graph_image.url }}?t={{ profile.modified|date:'U' }}" />
{% endif %}
<meta property="og:description" content="{{ profile.summary }}" />
<meta property="og:site_name" content="PyBazaar" />
<meta property="og:locale" content="en_US" />

X (formerly known as Twitter) also has basic support for Open Graph, but it is better to add Twitter-Card-specific meta tags to make the preview more prominent. Here's an example:

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:url" content="{{ WEBSITE_URL }}{{ request.path }}">
<meta name="twitter:title" content="{{ profile.user.get_full_name }}">
<meta name="twitter:description" content="{{ profile.summary }}">
{% if profile.open_graph_image %}
    <meta name="twitter:image" content="{{ profile.open_graph_image.url }}?t={{ profile.modified|date:'U' }}">
{% endif %}

Use both Open Graph and Twitter Card meta tags for cross-platform support.

The dimensions of the preview image

The Open Graph Preview image dimensions will vary across different social networks. Many sources online recommend using 1200 x 630 px, but I have also successfully used smaller dimensions.

If the image is too big, the social network might take longer to process it for the preview, which slows down the link publishing.

On the other hand, if it's too small, its text might be too blurry.

Using the browser testing tools for the screenshots

I have used Playwright and Selenium to generate preview images in a background task. Since Playwright has a built-in mechanism to download browser binaries and is much more advanced in testing capabilities, it is also my first choice for screenshots.

You can install Playwright with:

(venv)$ pip install playwright
(venv)$ playwright install

Ensuring to have the necessary settings

To show the proof of concept, you would need WEBSITE_URL and Open Graph Image dimensions in your settings:

WEBSITE_URL = "https://www.pybazaar.com"

OPEN_GRAPH_IMAGE_WIDTH = 800
OPEN_GRAPH_IMAGE_HEIGHT = 418

Passing the `WEBSITE_URL` to the templates

It's easiest to pass the WEBSITE_URL and other settings to all the templates by having a custom context processor:

def website_settings(request):
    from django.conf import settings

    return {
        "WEBSITE_URL": settings.WEBSITE_URL,
    }

Set it in the settings as follows:

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [os.path.join(BASE_DIR, "templates")],
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
                "pybazaar.apps.core.context_processors.website_settings",
            ],
        },
    },
]

Preparing the models

Each model that should have an Open Graph preview image should get an image field open_graph_image:

import os
from django.db import models
from django.utils import timezone


def upload_images_to(instance, filename):
    now = timezone.now()
    filename_base, filename_ext = os.path.splitext(filename)
    return "profiles/{user_id}/{filename}{ext}".format(
        user_id=instance.user.pk,
        filename=now.strftime("%Y%m%d%H%M%S"),
        ext=filename_ext.lower(),
    )

class Profile(models.Model):
    user = models.OneToOneField(
        "accounts.User", verbose_name="User", on_delete=models.CASCADE
    )
    open_graph_image = models.ImageField(
        "Open-graph image", upload_to=upload_images_to, blank=True
    )

    def generate_open_graph_image(self):
        from .tasks import generate_profile_open_graph_image

        if self.open_graph_image:
            self.open_graph_image.delete()
        generate_profile_open_graph_image(profile_id=self.pk)

Creating a Django view for the Open Graph preview

To define the preview layout, we'll create a custom HTML view:

from django.views.decorators.cache import never_cache
from django.shortcuts import render

@never_cache
def profile_open_graph_preview(request, username):
    profile = get_object_or_404(Profile, user__username=username)
    context = {
        "profile": profile,
    }
    return render(request, "profiles/profile_open_graph_preview.html", context)

The Django view used for the Open Graph preview image can use CSS frameworks, such as TailwindCSS, custom fonts, and Javascript enhancements for layout. This allows you to have consistent typography and styling with the website.

Plugging the view into the URLs

The URL rule for the view is pretty straightforward:

from django.urls import path
from . import views

app_name = "profiles"

urlpatterns = [
    #...
    path(
        "<str:username>/_open-graph-preview/",
        views.profile_open_graph_preview,
        name="profile_open_graph_preview",
    ),
]

Creating a background task to make screenshots

I use Huey for background tasks. The background task that generates screenshots looks like this (Celery task would be analogous):

from huey.contrib.djhuey import db_task


@db_task()
def generate_profile_open_graph_image(profile_id):
    import os
    from playwright.sync_api import sync_playwright
    from PIL import Image
    from django.conf import settings
    from django.core.files.base import ContentFile
    from django.core.files.storage import default_storage
    from django.urls import reverse
    from io import BytesIO
    from .models import Profile

    os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true"  # for Playwright
    profile = Profile.objects.get(pk=profile_id)

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        context = browser.new_context(
            viewport={
                "width": settings.OPEN_GRAPH_IMAGE_WIDTH,  # Changed to target dimensions
                "height": settings.OPEN_GRAPH_IMAGE_HEIGHT,
            },
            ignore_https_errors=True,
        )
        page = context.new_page()
        page.goto(
            settings.WEBSITE_URL
            + reverse(
                "profiles:profile_open_graph_preview",
                kwargs={"username": profile.user.username},
            )
        )
        page.wait_for_load_state("networkidle")
        screenshot_bytes = page.screenshot()
        browser.close()

    # Open and resize if needed
    image = Image.open(BytesIO(screenshot_bytes))
    if image.size != (settings.OPEN_GRAPH_IMAGE_WIDTH, settings.OPEN_GRAPH_IMAGE_HEIGHT):
        image.thumbnail(
            (settings.OPEN_GRAPH_IMAGE_WIDTH, settings.OPEN_GRAPH_IMAGE_HEIGHT),
            resample=Image.LANCZOS
        )

    # Save the image
    final_image_io = BytesIO()
    image.save(final_image_io, format="PNG")
    final_image_io.seek(0)

    rel_file_path = Profile._meta.get_field("open_graph_image").upload_to(
        profile, "profile.png"
    )
    default_storage.save(rel_file_path, ContentFile(final_image_io.getvalue()))
    Profile.objects.filter(pk=profile.pk).update(open_graph_image=rel_file_path)

Triggering the generation of the Open Graph preview image

Call the image generation method after saving an entry and its relations in a change view, administration view, or management commands:

profile.generate_open_graph_image()

The method runs the background task that launches Chromium, opens the Open Graph preview page, takes a screenshot, resizes it if necessary, and then saves the image. All that takes up to 15 seconds.

Final words

If you want the pages you share on social feeds to have a branded look and feel, generating Open Graph images with Playwright could be the right approach. You can use Playwright to make a screenshot of a web page in Chromium. The page can be styled with the same quality as your website. JavaScripts can be applied to highlight code syntax or resize the texts to fit into the given container, too.

Cover photo by Zeeshaan Shabbir

DEV Community: Aidas Bendoraitis

Wagtail Routable Pages and Layout Configuration

1. Project Setup

2. File Structure

3. Models

myproject/apps/articles/models.py

4. StreamField Blocks

myproject/apps/articles/blocks.py

5. Templates

articles/article_list.html

articles/article_detail.html

articles/blocks/cover_image_block.html

articles/blocks/description_block.html

articles/blocks/related_articles_block.html

6. Django Admin Registration

articles/admin.py

7. Migrations and Initial Data

8. Wagtail Admin Setup

Final words

How to Show a Waitlist Until Your Wagtail Site Is Ready

The Concept

1. Generate and store two secrets

2. Create the access-control decorator

3. Create the passphrase form

4. Build the cookie toggle view

5. Wrap the Wagtail catch-all with the decorator

6. Build the proxy home page view

7. Wire up the URLs

8. Protect every other app view

Final words

Using tox to Test a Django App Across Multiple Django Versions

The Example App: django-shorturl

Project Layout

Packaging: pyproject.toml

Test Settings: test_settings.py

The Core: tox.ini

envlist — the matrix

isolated_build = true

deps — conditional dependencies

commands

setenv

Writing the Tests

Installing Python Versions with pyenv

Install pyenv

Install each Python version

Make them all reachable at once

Running tox

Final Words

Understanding the Different POST Content Types

Standard Form Data

Form Data with Files

JSON String

Newline-Delimited JSON

Plain Text

HTML Text

XML Data

SVG Image

Binary Data

Protocol Buffers

Code to Play Around with

Moving a Domain to Another Registrar

The Situation

The Story

The Learnings

Per-Object Permissions for Elasticsearch Lists in Django Websites

The Challenge

Things We Considered

Our Chosen Approach

Final Thoughts

User Timezones in Django

Time Zone Database

Timezone Changes

Best Practices for Django

Timezone Management for a Social Platform

1. Enable Timezone Support in Django Settings

2. Add a timezone Field to the Custom User Model

3. Detect Timezone on the Frontend

4. Use a Custom DateTime Field in Forms

5. Output Dates and Times in User's Timezone

Other Options

`myproject/apps/articles/models.py`

`myproject/apps/articles/blocks.py`

`articles/article_list.html`

`articles/article_detail.html`

`articles/blocks/cover_image_block.html`

`articles/blocks/description_block.html`

`articles/blocks/related_articles_block.html`

`articles/admin.py`

2. Add a `timezone` Field to the Custom `User` Model

Include the widget to your `base.html` template

Add GTM and Google Consent Mode snippets to your `base.html` template:

Passing the `WEBSITE_URL` to the templates