Databases, Django APIs, and AI - Haki Benita
Published November 8, 2026
This video features Haki Benita at DjangoCon Europe 2026 in Athens, Greece.
TALK: Reliable Django Signals by Haki Benita
Django signals are extremely useful for decoupling modules and implementing complicated workflows. However, the underlying transport for Django Signals makes them unreliable and subject to unexpected failures. In the talk I demonstrate several strategies for decoupling modules, including Django Signals, and discuss different aspects such as user experience and fault tolerance. Finally, I present an alternative underlying transport implementation for Django Signals using the new tasks framework in Django 6 that addresses the shortcomings of all other approaches. The alternative implementation makes Django Signals reliable for mission critical workflow and for applications that require high reliability and fault tolerance.
More information: https://2026.djangocon.eu/speakers/8QRVC3/
Follow DjangoCon Europe on social media:
*X - https://x.com/DjangoConEurope/
*LinkedIn - https://www.linkedin.com/company/djangocon-europe/
*Mastodon - https://fosstodon.org/@djangoconeurope
#django #djangocon #djangoconeurope #djangoconeurope2026
Automatically transcribed, so expect mistakes in names and technical terms.
Speaker 1: Hello, everybody, please come in. My name is Fabian. I work with Jugo CMS and I have the pleasure of guiding you through the uh session this afternoon. And I hope you had a great lunch. Please find seats. First thing I would like to remember you um signing up for chat. jangocon. eu uh and register your lightning talk. for today, tomorrow or Friday. So that we have a great selection of lightning talks tonight. And you might be amongst them. Please submit them. But now uh it's my honor to introduce Jaki
Speaker 1: Benita. Um he apparently is the most funniest dad on the playground, so very much unlike me. Uh he used to play the piano. He doesn't anymore. Not anymore, he says. One of the reasons might be also he's been a uh recognized individual Django contributor last year, so that takes some work. And uh he's here today to speak about reliable Django signals. Please welcome
Speaker 2: I want to start with an apology for all the people who attended my workshop. You need to stand 30 more minutes with me. So I'm sorry for that. My name is uh Chaki. I've been a Django developer for the past uh ten years. Together with my team, I've built successful products using Django and Python. And Postgres, I talk and I write. And today I want to talk to you about reliable Django signals. So, what I want to talk about is I want to present a pattern for decouple models in complicated systems. And I want
Speaker 2: to show you how you can achieve reliable and hopefully fault-tolerant workflows in your complicated Django apps. So before we start I just I know that there are many different ways of thinking about workflows, but for me a workflow is any business process that is carried out by your application. Usually it will involve multiple steps. Multiple interactions and usually it will involve multiple actors or participants. This can be users, this can be third parties, this can be different models within your system. Okay So, to demonstrate, I want us to actually try and implement a workflow together and try to understand
Speaker 2: the problems that can arise. So we are going to build a checkout workflow for our imaginary e-commerce website. The workflow is very straightforward. I'm sure you experience it almost on a daily basis. You browse around the website, you pick a bunch of stuff, you proceed to checkout, you pay, and magically the order gets filled. This is the workflow. So the first building block that we need to address is uh payment. We need to be able to pay for the order. Payment is notorious for being very difficult, requires lots of integration, compliance. So usually you don't do it by yourself. You use some kind of third party. Maybe it's Stripe, maybe it's uh ADN, maybe you like to keep most of your money so you're using a local provider.
Speaker 2: But normally it works the same way. You use the API, you issue some kind of ID You get a URL, you take this whatever it is, you pass it down to the client. In the client, maybe you show an iframe, maybe you pass it to some client-side SDK And then the user interacts with the third party, some time goes by, and you get notified about the outcome of the payment. Okay, and then you can react to that. This notification usually comes in the form of a web hook, maybe a redirect, maybe through a socket. Okay So, if we want to uh build a very simple version of this payment process in our system, we can use this Django model.
Speaker 2: We have ID, we have amount, we have status, we can create it. At some point in the future, we are going to get a notification, a webhook saying that the payment completed. So we have this very simple view We identify the payment and then we set the status, local status for the payment process. You see payment process set status, succeeded, yes, no. This function to set the status for the payment process starts by identifying the payment process. Then it needs to check the status. This is usually where you handle item potency, stuff like that. And then we set the local state based on the outcome we received from the API. So far so good, right?
Speaker 2: So far so good. Okay. So now that we can process payments, we need to process orders, right? So once again, the user picks a bunch of stuff. Creates an order. At this point, your application would check the price, apply discounts, check shipment, all kinds of stuff, and then produces a number. This is the number that the user needs to pay. So it creates a payment system, a payment process, the user pays, and we get notified about the outcome of the payment. Right? And depending on the outcome, we can either cancel the order or fill the order. So we create a model for the order. Once again, simple stuff, dumbed
Speaker 2: down, nothing complicated. We have ID, we have amount, we have the status. Okay We also have a payment process now because part of the order requires payment. So we have a dependency on the payment process. So when we create a new order, we receive an amount, we create a payment process, we create an instance of order, right? At this point, the order is pending payment and the payment is initiated, waiting for user interaction. Sometime in the future, after the user finishes interacting with the payment, we get notified that the payment was completed. Maybe it failed, maybe it succeeded. And now we can sync the order state we can actually proceed with our workflow and update the order.
Speaker 2: So it starts once again, same thing. We identify the order by the payment process. We obtain a lock and everything. We then check the state, handle item potency, and everything. And then depending on the outcome of the payment , We can update the order. So if the payment was successful, we filled the order, otherwise, we cancel So let's actually do this workflow, okay, and see how this works. So I create an order. You can see the first line I have an order, ID is one, I have a payment process, number two, I have the amount, and the status is pending. payment. I also have the payment process which is now initiated. At this point the user interacts with
Speaker 2: the user interacts with the payment. And at some point in the future, I get a web hug saying the payment was successful, and then we set the status for the payment process. At this point, the payment process is successful So are we done? Something is missing. What's missing? Sorry didn't get that. So the part that we are missing right now is We are missing the order part because at this point, at this point, the payment is successful, but the order is still pending payment.
Speaker 2: So the question now is. Who's triggering this change? What's triggering order to go to payment process and sink the state? Something needs to happen, right? So the most naive approach that you can take here is you just go to payment process and you say, what's the problem? Whenever I Fin whenever I save the status of the payment process, I would just go and update the order, right? What's the problem? Here you go. Payment process save and immediately after order payment completed. Do whatever you need to do. What would happen if I try to run this? Anybody care to take a guess?
Speaker 2: If I try to actually execute this code, what will happen? It's a tough one Yeah. No, this is not even related to that transaction. If I actually try to run this code, I'm going to get a very nasty error. And it's not even a clear error because I get an import error, and the important thing is actually in parenthesis The problem I have here is I have a circular dependency, and that's a problem, because my order creates a payment process. And in order to trigger change in the order, I created
Speaker 2: another link between payment and ODER. So now I have this circle. Now, this is a problem. Circular dependencies are usually symptoms of, let's say, bad design. Okay? Now I know some of you think Dude, that's easy. You can easily overcome this. Just use the reverse relation, right? No problem. No imports. Or maybe import inside the function. What's the problem? It won't yell at you, it's runtime, right? But I say that even if you do that, you still don't solve the actual problem. Because what if I told you That I want to have other top-level models workflows to use the same payment process.
Speaker 2: So right now we have an order, but maybe I have a payout process. Maybe I have a refund process. They also need to use the payment What if I told you that I have a bunch more that needs to feed off changes for payment process? Maybe I want to integrate that into my analytics, into my billing, into the user account. What if I even have more? Hopefully you see the problem right here. This is called in the in the official literature spaghetti. Okay? This is bad. This is something you don't want to have. Ideally You want dependencies to flow in one way. You want your top-level models point to lower-level models that points to even lower level models.
Speaker 2: Dependencies flow one way But this poses a question because now we have a problem. We have a dependency between order and payment. This is a necessary dependency. Order needs to be able to accept payment. But we also need to be able to trigger change from payment back to order. But we just said that we can't create this dependency So how can order a top-level model react to changes happening in lower level models? So before I go into super complicated stuff, there is an easy solution. And the most straightforward thing you can do, you can just pull for changes So in this case I wrote this management command, okay?
Speaker 2: And this management command simply looks for orders which are pending payment That their associated payment is either succeeded or failed. Right? And then I just go and I sync the state. So now I can run this thing every hour, right? And everything will be in sync But there's a big disadvantage to this approach. What is the main disadvantage to this approach? You can talk to me, I'm not a YouTube video. What's the main disadvantage? Real time. Sorry? Real time. Exactly. So if I'm selling I don't know plane tickets six months in advance, if the user needs to wait an hour before he gets his ticket
Speaker 2: Fine. Not the best experience, but fine. But if I'm selling train tickets or bus tickets and the user is on the platform wanting to board the bus, And he needs to wait an hour to get the ticket. This is unacceptable, right? I want this to be fast. So what would be the PM approach to solving this? Yeah dude what's the problem? So don't run it every hour, run it in five minutes. And then five minutes turns into one minute. And then one minute turns into ten seconds. But then we have another problem, right? Because we are all of these processes running in the background doing nothing. Eating at our CPU, our resources, our database, doing absolutely nothing So this is extremely reliable, but it creates an unacceptable delay
Speaker 2: in the process. So This is fine for some situations like the plane ticket, that's fine. But if we actually need to react quickly, and we often do, then we need another approach. So the question now is, is there a way for payment process to signal Like send up a flare or something saying, hey, I changed. And then anyone in the system could just react to this change however they see fit. Right? So I I put up this lighthouse, but I think a flare would have been better, right? You just shoot up a flare and you say I changed.
Speaker 2: So lucky for us, Django has something called a signal dispatcher. Which does exactly that. It's even using all the words I just talked about. Look, it says in a nutshell, signals allow certain senders to notify a set of receivers That some action has taken place. This is exactly what we want. We want the payment process to be able to say, I changed, and we want some other place in the system to react to this change. So let's define a signal. We go to the payment model, we add a signal. We call the signal payment process completed. Okay? Now we go to our payment process
Speaker 2: and whenever we set the status, either failed or successful, we send the signal. Okay, we send the signal. Now other places in the system can Subscribe to this event using the receiver function and simply react to this change. So now when we set the status for a payment process, we send a signal, the order. Now subscribe to this change and it can trigger change in the order. So technically, we didn't create an explicit dependency between payment and order because payment sends out a flare It doesn't know who's going to handle it and how.
Speaker 2: It just says, I'm complete and do whatever you want. And Oda can react any way it wants. I can have a dozen more models here subscribing to this signal and act any way they see fit. If we actually run this thing, we can see that we create the order, we set the status for the payment process, and then magically order gets updated. This is very nice. Just just update it. So far what we've done is we used signals To create to
Speaker 2: prevent circular dependencies, right? To achieve decoupling in our system But did we really? Did we really? Let let me try to challenge this a bit and see if we actually achieve true decoupling. So the third question is, what happens if a receiver fails? What happened if a receiver fails? Okay, so if a receiver fails in my code right now, the sender is also going to fail. So that's bad, right? This is not what I want. I don't want to be dependent and yet when the receiver fail, I fail.
Speaker 2: This is not good. So Django signals, they have two ways. To be sent. The first one is send, which propagates errors, and the other one is send robust, which doesn't propagate errors. This is problematic for a different reason, but For the sake of discussion, this is what we want. We don't want to be affected by errors in receivers. So we go and we switch, send With send robust. So now all the receivers are going to be executed, but if they fail, the exceptions are not going to propagate to the sender. So are we decoupled? You know the answer to that. Are we decoupled? No, no, we're not decoupled. Let me ask you another question.
Speaker 2: We send our signal. Inside the database transaction, right? What happens to the sender if the receiver takes a very long time to execute? Maybe, but that's a problem, right? Because I should be I shouldn't be affected by anything. that the receiver is doing yet I'm inside a transaction and now you have this receiver. It runs for so long and now I'm inside a transaction. And this is bad. This can cause contention and locking. I can end up having to roll back due to timeout. So that's bad. Let me ask you another question. What happens if the receiver
Speaker 2: encounters a database error. In Postgres, when there's a database error, it causes the transaction to abort. So I'm the sender, I have my transaction, and then some receiver that I don't even know about causes my transaction to abort. That's bad. That's bad. What if my receiver has side effects? Huh? What if my receiver, for example, sends an SMS to the user? or sends an email to the user or increments some counter in the database or in some analytics service. If I end up rolling back ,
Speaker 2: These side effects are not going to roll back. So it's very likely that I would send a message to the user saying your order is filled. But the order is not filled because I rolled back. So this is all very bad, and a lot of it is happening because I'm sending the signal inside the transaction So the solution must be very simple, right? What is it? Move it out of the transaction, right? So I can just simply make this change and move My signal sending the signal out of the transaction. Okay? If you notice this is the diff, so just moving it left So now we're good, right?
Speaker 2: You know we're not, okay. So we know we're not good. So let's talk about fault tolerance. Okay, fault tolerance, what does it mean? This is a very charged term because different companies interpret it differently. But the way I like to think about fault tolerance is very simple If I look at my code and I imagine the system crashing at every point in my code, I need to know exactly what's going to happen. It needs to produce the desired or expected result Okay, so let's look at our code and see if our code at this point is fault tolerant. Okay? So if we fail right at the beginning of the function before we even created a transaction, is that okay?
Speaker 2: Yeah, that's fine, no problem. Nothing happened at this point. User can retry, no problem. What if we opened a transaction, did a bunch of changes, and then we failed before we had a chance to commit? Is there a problem here? No, right? Because the database would just roll back, the user can try again, no problem. But What will happen if I committed the transaction and before I had a chance to send out the signal, the system crashed. What will happen in this situation? This is the worst possible outcome.
Speaker 2: Because at this point As far as I'm concerned, the user was charged. But because the signal was not sent, I am never going to fill this order. So that's that's bad. This is exactly the kind of places where you lose all trust from users. Okay, so this is very very bad. So a quick recap before we actually get to the topic of this stock. Okay So we saw at the beginning that when receivers fail, they affect, they can cause the sender to fail. So we switched from send to send robust. We then said that things like side effects, long-running uh receivers
Speaker 2: can also affect The uh sender, so we move them out of the transaction, but then we saw that there's no guarantee that the signal is going to be sent, right? So that's also bad So let's try to actually articulate what we actually want from this process. So the first thing that we want is we want to keep models decoupled. We do not want to create these circular dependencies in our code. Second, we want receivers to have minimal impact on the sender. Ideally, the sender should not be affected by anything the receivers are doing. Okay? We also want to be able to guarantee at least once delivery.
Speaker 2: This means if the sender committed We want to make sure that the receiver is going to be executed at least once. If the sender rolled back, we want to make sure that the receiver is not going to execute. And also to top it off, because we're spoiled now, we also want to have some kind of mechanism. To retry failed receivers. If for example I failed the order, I try to send the SMS, something failed, I want to retry. Okay So here's the idea. Dramatic pause No way.
Speaker 2: Okay. Okay. So here's the idea. What if we just use A database queue. Okay? Whenever a signal is sent, instead of executing it immediately, We enqueue a task to execute this receiver later. Let's break it down even more. When a signal is sent inside the database, inside the sender's database, the receivers are not processed yet. Okay, instead of executing the receivers, we enqueue a task in a database queue to execute this later. Now at this point, we're not committed yet.
Speaker 2: So the receivers are not going to be executed. And if we roll back, they are also not going to be executed. Once we commit , Okay, a worker process can start processing these receivers. So in the last version of Django And we talked about this last year. Django added a tasks framework. The tasks framework allows you to run essentially background tasks. Now the interesting thing about the tasks framework is that it's essentially an interface. There is no implementation. For the database queue is just an interface saying this is what a task framework should look like.
Speaker 2: Now you can plug in your own implementation. So one implementation that was developed by where's Jake? Where's Jake? Hands out to Jack. Okay. So Jake here. He developed the database backend compatible with a tasks framework that we can use as a backend for our tasks. So let's put it all together, implement this together right here on the slides. It's going to take more than three minutes. So we start by installing the dependencies. We need uh Django 6 and up because of the tasks framework. We need the backend. We configure the app. We set up the settings.
Speaker 2: We apply the migrations, run them. This is a database queue, it needs tables. And some people sometimes misunderstand this. You actually need to have another worker running in the background executing these tasks So we also need to run the db worker. So now that we have the infrastructure in place, we can move on to implement the actual signal. So the first challenge that we need to tackle is in order to be able to execute some function, any function essentially, in our code, we need to be able to reference it. Right? Because once we write a task to the database, it's all strings. So the first thing we need to do is we need to be able to reference a function.
Speaker 2: So we write this very, very simple function that receives a callable function and returns some qualified name that we can use. So if I provide my on payment completed, I get this string order models or the on payment uh completed. We then also implement the opposite. Given a qualified name, we want to get the callable. Right? So now I can pass in the qualified name and I can get the callable. Right? So now I can write a task to the database saying I want you to execute this and then the other side can get the callable. With this function, we can now move on to the next step.
Speaker 2: We can now build our task. And our generic task receives the qualified name for the for the receiver and the arguments that I need to use for this function. And it simply executes it. Okay? So now we are ready to implement our actual reliable signal. So we subclass signal and we add a new function called send reliable. This is essentially the same as send and sendrobust. The exact same thing with one difference. Instead of executing the receivers We enqueue
Speaker 2: a task. The highlighted part is where we iterate the receivers and we add a task for each one. So now we go and we change our signal to a reliable signal. This is a drop-in replacement. Okay. And we go back to our payment. And now instead of using send robust, we use send reliable and we do it inside the transaction. Okay? So if you want to put all of this together and see what it looks like, I create an order, I set the status for the payment process, and shortly after the order is Updated. If you look at the worker logs, you can see
Speaker 2: that while I was doing this, the worker picked up the task and synced the state for the order. So what did we achieve? First of all, we keep models decoupled. Payment process doesn't know anything about ODER. It just sends out a flare, and I can handle it any way I want. Receivers don't have any impact on the sender because they are executed by a separate process. Right? We also achieved at least once delivery Because we enqueue the tasks inside the transaction. So now executing the signal or enqueuing the tasks to execute the receivers are now part of the sender's transaction. And as an added bonus, because we're using the tasks
Speaker 2: framework, we also get this nice built-in mechanism for retrying failed receivers. So we got to eat the cake and have it full, right? That's the term? Okay. But there are a few restrictions or limitations that we need to be aware of. First of all, because we're using a database queue, receiver arguments need to be serializable. So we can't Use receivers that accept Python objects. We need to keep things serializable, meaning primitive types that can be persisted to the database represented as JSON or whatever. We also only maintain partial compatibility with
Speaker 2: the existing Django signals because signals currently receive uh signal and sender arguments which In this type of framework is a bit more difficult to provide. You would have to manage registries for these. And also, it's not a good fit for all types of signals because there are signals Which are meant to be executed inside the same process. Okay? So it needs to live alongside the existing mechanism for signals So before I'm finished, promise, before I'm finished , some of the things that we can think about. Uh so first of all we did not support async tasks because we didn't need to, but should be fairly easy to add.
Speaker 2: There's a lot of Features from the tasks framework that can be very very helpful. For example, we can have a dedicated queue. We can give priority to some signals because maybe they're more important than others We can also do some work to actually increase compatibility with the existing uh signals framework And since we we're just stitching together two frameworks within Django, maybe there's a place for it in Django. If you think about it. The existing signal is basically using the tasks framework with the immediate backend. Does the same thing. So if you're interested, you can check out the blog. It has a lot more information about this pattern
Speaker 2: along with full uh code examples and if you want the formal definition for what we just talked about make sure to read about the outbox pattern And uh that's it.
Speaker 1: Thank you. You're here for the conference, right?
Speaker 2: Yeah, yeah.
Speaker 1: Yeah. So um you can talk to him. Uh I think we can do two quick questions now because we're a bit late in time. So Uh questions from the audience. I can't see anything here now. Uh any questions?
Speaker 2: They're shocked. Um
Speaker 3: the first obvious parallel that comes to me watching this is uh salary tasks. So if I as engineer have to choose between salary tasks and And this sort of uh jango tasks what will be the pros and cons
Speaker 2: Well, uh technically you don't have to use the tasks framework for it. The concept here is to use a database queue to achieve uh decoupling and achieve reliable execution It's just uh you know Django 6 came out with the task framework and it was a good match. So I used it. Uh but you can use whatever you want as long as it's not salary.
Speaker 1: Okay, thank you. Uh I've been told um we'll need to really be strict on time. I would like to inform you that workshop downstairs are starting. So if you want to attend the workshop Um it's maybe now time to leave. Um let's give Haki another big thank you.
April 15 - 17, 2026
Athens, Greece
Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026