Teach Django Tasks to speak Celery: Building a Celery backend for Django Tasks with MatĂ­as Bordese

This video features MatĂ­as Bordese at DjangoCon US 2026 in Chicago, Illinois, USA.

Teach Django Tasks to speak Celery: Building a Celery backend for Django Tasks with MatĂ­as Bordese
0:25:41
Published October 26, 2026
156 views

Django 6.0 introduced Django Tasks, a built-in framework for defining and queuing background jobs to run code outside the HTTP request–response cycle. Django ships by default with backends for development and testing, but production systems often rely on distributed task queues like Celery. How do we bridge that gap?
In this talk, I'll show how Django Tasks works under the hood, and walk through a Celery backend implementation that lets you run Django tasks on Celery workers without changing your application code.

What this talk will be about:

What Django Tasks is and how it works: the new standard API for background jobs in Django 6.0.
How task backends act as adapters between task APIs and execution engines.
How a Celery backend translates Django tasks into Celery jobs.
A practical demo highlighting the pluggable architecture (switch your backend to use Celery without changing code!).

No prior experience with Celery is needed.

This talk was presented by MatĂ­as Bordese at DjangoCon US 2026 in Chicago, Illinois, USA
https://2026.djangocon.us/talks/teach-django-tasks-to-speak-celery-building-a-celery-backend-for-django-tasks/

Transcript

3,450 words · auto-generated · en Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:14

Speaker 1: Hey, I'm Matthias. I'm happy to be here. A little bit nervous or here on stage, but let's see how it goes. The purpose of this talk is to talk about Django tasks and the new framework that is available in Django since last 6. 0 release and particularly focused on how a backend works. So we will go through some internals of how a backend is defined and implemented. Even if celery is mentioned in the title, there is no real previous knowledge needed from celery because it's was mostly an excuse to investigate and review the backend details, but it's not needed for the purpose of following the

1:00

Speaker 1: talk the talk. So yeah, I'm Matthias. I work at Grafana. I'm part of the IRM team where we implement uh plugin for Grafana for dealing with alerts and pages. So We have a plugin implemented a backend using the angle at the angle rest framework for the API. And on the other end we have Some celery services dealing with all the alert ingestion and the triggering of the escalations on Beijing people. So we use a lot of celery to handle all that thing. And that's where my context using Celery comes from. So I have this kind of love -height relationship with celery. I've been using celery for a lot of years, but recently it's been mostly love, so

1:47

Speaker 1: I'm happy to use LR when I can, but this is not the purpose of the code. So what are we going to talk here? And the main goal is to go through why We need Django task as part of the framework. Also go through the task interface. So what's What you as a user will be dealing with to use Django Tasks. And then go through the different backend responsibilities and the implementation details that you will need to implement a backend specifically for some particular engine. And then a live demo because we have learned that they always work. So let's go with another try. And and yeah, and finally some conclusions around the content of the talk.

2:35

Speaker 1: So let's go. The problem we are trying to solve. Almost or a lot of times you need in Adiango app to do work outside of the request response cycle Like sending a welcome email, rebuilding an index, generating a report, or paging a user. So you don't want the user waiting on that. You want to trigger something that will be handled in the background and uh do the work. Usually people go through different solutions for that, like a cron job. in the most basic case, or some table in the database to handle tasks as a queue, or celery or RQ, which are frameworks much more complex

3:20

Speaker 1: that allow you to do that and much more. The problem with that is that uh your business code ends importing and depending on this engine you choose as a background as a back end for dealing with the background task. And that's where the Angle task is important or where the problem it tries to solve. It basically breaks this coupling we have between our business code and the background tasks. So The Angle Task will provide you with an interface, an API for you to write the task and make it possible to send to the worker, to the background worker. And on the other end, the people implementing the backend, which not need to be the same person that using the task

4:08

Speaker 1: framework, uh will have a contract to follow, so you can Choose which we can to use independently of your business goal. So that's the main goal. from DjangoTask. If you haven't used DjangoTas before or you are not familiar with it, this slide summarizes how it works. You import a decorator, you decorate a function that you want to run as a background task, and then later you can call the NQ method with the corresponding parameters, or even uh Customize the parameters of the task itself using the using method before the incube. So it's really really simple API and it's nice to use if you haven't yet tried.

4:57

Speaker 1: So summarizing the model is you have the curator which is back and agnostic You have a setting in the Django project settings that will define a backend that you want to use to run the task, and you have possibly a worker which will be picking up the tasks and running them on other side. Uh we are going to try to focus on the backend implementation details now. So The Yangot Task Framework provides us with a base task class, base task backend class, which is the main interface that we need to implement. It provides us with a few things already figured out for us, like all the validation and some wrappers, as well as importing the config

5:45

Speaker 1: data from the project settings. But it's your responsibility to implement the NQ method mainly, which is the method that will translate the framework task into the engine running the task in the backend. And also you need to implement the uh getResult method. So attach back uh backend can provide different features which are also signaled by those flags that you see there, like the support defer, support async task, support priority, or get result. So You don't really need to implement the whole thing. It will depend on the features you want to support in your backend. But the only method you you must implement for sure is the NQ

6:31

Speaker 1: method. Along all this configuration, you have these checks and validate methods that will perform all the validation for you based on these flags that you set up. You can also extend the checks to force additional settings or some other requirements that your backend may have. For example, we will see later, celery Can work as a fire and forget only task queue. So not necessarily you need to have a result backend, but the task framework allows you to have A result. So more on that later. Then, yes, the framework will define the shape and the interfaces we need to implement, and we

7:19

Speaker 1: need to implement the NQ method. Mainly. The Django Task Framework in the core Django provides two basic backends that you can take a look because they are less than 100 lines. but they are really useful for production. You have the dummy backend and the immediate backend. The dummy backend will just store the metadata of the task. without running anything and the immediate backend is just when you call NQ, you're really executing the task. So there is no async task or async work running anywhere. It's just the same process in the same NQ call running your task. So before proceeding with

8:05

Speaker 1: NQ, we need to know the data we are managing in this process. When the NQ method is called, you will get A task class, a task instance that has this interface. It basically keeps a reference to the function that you were recalling as a task. And the metadata of the parameters of this call, like the priority, the queue, or the backend that will run the task. And as output you need to build a task result. The task result is the uh instance that you will return in NQ and that you can Get later or refresh when you want to track the progress of a task if your backend

8:54

Speaker 1: supports getting results. So the main thing here is The Angotas summarized the status of a task in four statuses. So you need to map whatever your backend supports is. And you need to be able to return the exception following the task error interface that we went going through, but it's keeping the trace back mainly. So, what else? We have the interface for the backend, which is based on the base task class. And the task in and the task out, which are the task instance and the task result. But the other thing that the backend should implement is the signals. These signals are emitted in different points of the task

9:41

Speaker 1: life. Particularly we have three signals provided by the backend. One is task in Q, which is triggered when the uh The angle task sends the task to the backend, and then we have the started and finished signals, which will be emitted. on the worker side when the task is indeed started or finished because of success with a success or fail status at the end Um yeah, good. So if we summarize All this we have been reviewing, we identify five responsibilities a backend

10:27

Speaker 1: should implement or should address, which are declare the capabilities that your backend will support Identify the function that will be run as a task. Make it possible to encube the task that is translate the data we get from the framework into the engine representation. The execute itself, which basically is the main thing running the task, which can be a separate process or anything you want. And uh the report, which is getting the result information back from the engine into the task framework. To go through all these responsibilities, I will iterate on the on three different existing backends.

11:14

Speaker 1: I will take a look at the immediate backend the database backend and a celery-based backend. That's why here I have a short slide summarizing how Celery works. But the main idea is you have uh queue which is a message broker like Rabbit or Redis where you put messages that the worker which is a long running process will consume from the task And run the task, the execute execute the task. And optionally, uh result backend, which can store the information and the progress of the task. So that's Sailor in just a few words. It's a little bit more complex than that, but for the purpose of

12:00

Speaker 1: Integrating or adapting it to the task frameworks, we don't need much more than that. Here we have uh just a summary table comparing the concepts on the end of task and celery. Which we can skip for now. So but the idea is we have a mapping and we need to adapt or make decisions on How we translate one thing to the other to keep the framework uh requirements satisfied. So the backends The idea here is to go through all these responsibilities we define for each of these backends. The backends R this three the immediate backend which is built in into the angle.

12:46

Speaker 1: There isn't a worker running for this backend, it's just The execution of the function in line. Then the database worker, which is a third-party package, so you need to pip install it. This package is from the author of the Django Tass original package Then we have this final package that I work with in the context of this talk, which is a backend based on salary. And that's why it has this name, because it's in the family but it's not celery and also because the Angotas celery was already taken in PyPI, so naming things is hard. You can take a look at the other backend also. Here we are.

13:33

Speaker 1: So what your engine supports. We mentioned that we have these flags that you should enable for each backend depending on their features you want to support. Here we can see that the uh immediate work uh backend doesn't support the defer feature, for example, and the backend the backend. The framework will validate this and will avoid or will raise an exception identifying the issue if you try to do something like the backend doesn't support. Um all the other two will support the whole feature set. Defer as in priority and get result. Then you need to be able to reference the function that you were

14:18

Speaker 1: sending as a task back. That's because The framework requires you to build a task or a task result that still references the original function, so you need to be able to reverse this. In particularly, the immediate backend is not an issue because you are running in line and you have the object in there. The database backend will keep information in the database to allow the rebuild of this. And in Celery we have this task registry that allows us to get the import path of the task and we can build it from there. So And then the most important method that you need to implement is thank you. Thank you will translate

15:05

Speaker 1: the the Angotas fills into whatever your engine wants. So it will hand off the work to the engine. Trigger the signal that the task was in queue and get a task result that will be able you that you will be able to later query for updates or refresh. In NQ and immediate backend, the NQ method is basically called the function. And for the database , Backend, we will add a row into the database. And for the Celery backend, we are using Celery internals to enqueue the task and do it the translation in the backend implementation to map. the right values to the uh

15:51

Speaker 1: salary expectations then we have The execute. The execute is where most engines differ because it's basically making it possible to consume the task wherever it was queue and run the task. For the immediate backend there is no worker, the task is run. For the database worker, we have the particularity that The same package implements the polyum worker which will trigger uh will consume the tasks from the database and update the database with the information there. And uh for salary backend we are using the celery worker itself. So

16:37

Speaker 1: for this final backend we will have On one hand, the backend mapping and translating all the Django task implementation and representation to Celery. And on the other hand, we have the Celery worker consuming, as usual, tasks from the Rabbit MQ or the Freddy's message brokers. And finally, yes, you need to be able to rebuild all the information you have been processing into a task result. In the case of immediate, there is no way to do that because once the task is finished, you don't have any more reference to the metadata or the result of the task. The database will read from the database and for salary we

17:22

Speaker 1: if we set up a backend we can get the information from the result backend. And there is also a few conversions and translations needed to make sure that we have all the information that the task framework respects. So This is all really really high level because the time doesn't allow to get into more details, but the idea that you should get from this is You write recall in Django without importing anything external to the Django core itself. You can define tasks. And you can configure your backend independently of your code and switch the backend in a transparent way. So your backend

18:09

Speaker 1: It's not the most important thing for the purpose of writing the code. So the live demo time. Let's see. The idea here is I have a simple, simple project in which I trigger an escalation after receiving an alert And it will try to notify some people, an on call person, the team or the manager with different delays. Um so it's a task queue that then queues another three tasks using the run after parameter. Let's see if things work. So

18:55

Speaker 1: uh this is the code. I don't know if it's visible enough, but it's Almost the same code you were looking in the slide. And there is my project settings. Right now it's configured to use the uh immediate backend, which will not necessarily work because it doesn't support the defer uh defer feature. Let's see. So here if I trigger an alert, I will see that nothing happens because the backend doesn't support the Run after parameter, right? So we could even if it supports

19:41

Speaker 1: if we trigger a task which supports with the immediate backend supports, we will see that the task is executed immediately without any delay, uh DJ, any background work. So let's now quickly switch you will see that I'm just changing a setting. So I have the Database backend in this case, for which I need to run the worker that will consume from the queue in the database. And in this case, if I trigger a new alert, I should see this progressing here. There you go. And here

20:26

Speaker 1: the task running in the logs of the DB worker. So this is consuming from the DB. And similarly, we can now switch to the Celery button again only changing one setting in the project and I can run the celery worker Here you will see that I'm mapping the task name with a Tjango task prefix. So and that also I'm running the worker with a Django task channel app that I generate from your task configuration in the Django side.

21:11

Speaker 1: So I'm handling all this for you. You just need to run the worker. And if we trigger an alert now, oops, it is progressing and we can see now the logs here also Paging, paging. So as you can see, really nice. You only change one line setting and you can switch your backend depending on your needs, on your needs. I almost some time, so this is what I was talking about. So the important thing is this. You didn't change the task, you didn't change the view or the models or anything. Just one line of settings and you can switch the backend transparently. And your business code doesn't know anything about the backend

21:58

Speaker 1: you are using and you can write your goal without depending inherently on any specific library or package. And that's really nice. Even if the Django Task framework is not super powerful because you have a lot of more features available in Celery or RQ, for example. You can still do a syn work and Don't force people to use any specific backend. You can just distribute your package depending on task framework and the people can use the backend they feel better with. So What do we have here? The Angotask is an abstraction. It's not trying to replace Celery or RQ or anything. It's a simpler and smaller backend

22:44

Speaker 1: framework Right now there isn't a production ready backend in the Angle Core itself, so you still need to Install something external to use it. You have the option to implement your own backend. It's just implementing and doing the translation between your packet and the uh Dyanglock task framework in both directions And yeah, you are not logging. You can switch your backends if you write a call following the task API. So that's it. If you are planning on using background tasks Maybe consider switching to task framework and give it a chance.

23:33

Speaker 1: You can then change the backend as you need or as your possibilities allow. So Here is the repo implementing the salary backend, the task documentation, and the salary documentation. And yeah, we are almost on time, so I don't know if we have time for questions, but you can ask me questions Outside.

24:00

Speaker 2: Thank you. It's time for one question if somebody wants to have one question Otherwise we'll move on. I'll take it. Oops

24:13

Speaker 3: So when you were writing this, how much did you find yourself having to dig into seller internals, specifically with how it interacts with AMQP?

24:22

Speaker 1: Yeah, there are some limitations with the interfaces exposed by Celery and the requirements from the task framework also. Particularly I have to deal with some internals with Celery to keep some additional metadata bit when I enqueue a task so I can review the task later So I have like a some side channel storing some data and then you have yes for priorities not all backends behave the same. So for example Rabbit has some priority but red is is More flexible, let's say. So yeah, it's there are a lot of commitments and decisions that you need to

25:10

Speaker 1: do in the process. Still, I mean Django Task is the idea is to be a simpler framework around task background task, and it I in my opinion, at least, it satisfies a lot of scenarios for people that weren't considered before. So it's really nice to have it part of Tango right now.

Conference

DjangoCon US

August 24 - 28, 2026

Chicago, Illinois, USA

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos from DjangoCon US