Auto-prefetching with model field fetch modes in Django 6.1

This video features Jacob Walls at DjangoCon Europe 2026 in Athens, Greece.

Auto-prefetching with model field fetch modes in Django 6.1
0:29:21
Published October 26, 2026
80 views

TALK: Auto-prefetching with model field fetch modes in Django 6.1 by Jacob Walls

New in Django 6.1, model field fetch modes provide a way to avoid the deluge of queries from accessing un-fetched objects during a loop (also known as the "N+1 queries" problem). You may be familiar with how Django's prefetch_related() and select_related() methods can prevent these queries, but unlike those tools, the "fetch peers" mode does not require maintaining a list of fields to fetch. This can significantly improve project maintenance, especially when QuerySets are constructed at some distance from where they are used. In this talk, we'll cover what's new about fetch modes, what tradeoffs are entailed, how to configure fetch modes per-model via managers, and how to use the "raise" mode to surface issues in development or put guardrails around performance-intensive sections of code (like the django-seal package).

More information: https://2026.djangocon.eu/speakers/7QDHKK/

Follow DjangoCon Europe on social media:
*X - https://x.com/DjangoConEurope/
*LinkedIn - https://www.linkedin.com/company/djangocon-europe/
*Mastodon - https://fosstodon.org/@djangoconeurope

#django #djangocon #djangoconeurope #djangoconeurope2026

Transcript

5,172 words · auto-generated · en Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:02

Speaker 1: And now for our next session this morning we have someone who started as a Django fellow last summer, Jacob Walls. Giving a talk on auto prefetching with model field fetch modes in Django six point one.

0:23

Speaker 2: Hi, thanks for that great introduction and thank you all for being here. It's great to be here. This talk, auto prefetching with model field fetch modes in Django 6. 1. I want to bring to life or bring the documentation to life for this new feature that I want to hype up for Django 6. 1. But more than that, I want to share my mental model for what this is, when you would use it, how am I thinking about it, what to keep in mind when considering using it it. So hello. That's my handle on GitHub. I'm a Django user, builder of full-stack web applications. Some of the niches those applications have been in are music education apps, civic water tech apps, and cultural heritage applications. I've been a Django contributor since 2020, which is an integral part of becoming a better Django user, and had the privilege of starting as a Django fellow last summer, which means that I can devote paid working time to the maintenance of the Core Django framework, working alongside my colleagues Natalia and Sarah.

1:16

Speaker 2: In this talk, I'm going to go over the background for the use case. I'm going to discuss specifically what failure modes this addresses, what makes this a missing feature from the from the Django ORM. I'll specifically cover what we're shipping. We're shipping three fetch modes. I won't hold you in much suspense. I'll give you the briefest version right now. Fetch one is the behavior you're used to for the last 20 years of Django. You would only specify it explicitly if you needed to undo some transformation. Fetch Peers is what's really new that we're shipping and that you would consider using. Fetch peers is like a just-in-time pre-fetch related call. So at the it you can tell your query set, don't let me get into an N plus one query situation.

2:01

Speaker 2: Just fetch the peers that I'm likely to need And finally the raise mode, which is a tool that will instead of fetching just in time, it'll raise exceptions just in time, alerting you to the specific part of your code that's problematic. Useful for development. I'm going to talk about how we got here. I'm going to spend a couple minutes talking about laziness because by deciding to use fetch peers, you're deciding to do lazy data fetching. I'm going to share some insights that I I kind of owe to Simon Charette who mentioned this in the In his DjangoCon 2022 keynote to DjangoCon US, where this missing feature in the ORM was discussed at length. I'm also gonna show you how to use it with model managers to avoid the boilerplate of repeating yourself all over the place, and finally show you something that's not in the docs, which is if you want to write your own fetch mode, you might be looking at that third mode that raises an exception and wonder, can't I please have a warning instead?

2:50

Speaker 2: So I've got you covered there with an exception. I've also teased you with a link at the bottom here. I'm leaving this footer up on all of my slides, noticing that the color contrast is not great, but uh it's hosted on Dry ORM. It's worth checking out Dry ORM if you haven't seen it before. It's a web-based um playground uh that allows you to um run examples against the Django ORM uh without installing anything. Um thanks to Amar, you may know him as Xterm on Discord for this excellent tool. tool, uh dryorm. exterm. info slash fetch mode simple. Dry or m fetch mode simple. My goals in this talk, I want to inspire usability feedback. Inspire pre-release testing. I'll tell you when the pre-releases are going to come out. The first pre-release is next month. So really what I'm doing in this talk is inspiring pre-pre-release testing.

3:38

Speaker 2: So you're going to need a GitHub URL if you want to test it today. But the the alpha will come out next month. And I want to inspire further development. So what I'm sharing today probably won't be the last word on this subject. Okay, so quickly, in case you haven't seen the N plus one queries problem or are not familiar with the term, I can cover it briefly. It's a problem that has to be confronted by pretty much every ORM out there. At the top of the snippet, I'm fetching some books. I'm going to iterate over the books and I'm going to dot into the dot author relation. So if I rely on Django's default behavior. Django defers relations by default. So on every loop iteration, when I dot into dot author, Django is going to fetch the single author for the single book.

4:24

Speaker 2: There are n books, that's n queries for each book's author. Throw in the first query at the top of the snippet and I have n plus one queries. If n is large, that's something I would never do if I were writing the SQL by hand. This contributes to pretty much the most fundamental performance bottleneck in any web application is trips to the database. So, the traditional tools for dealing with this that the Django ORM provides have some specific characteristics that we can cover briefly. Select related and prefetch related are the two tools that ship with the ORM to mitigate this. If you're fetching a single valued relationship, you may reach for select related, which grabs the data from the origin table and the tables that are related, performs uh fetches all the data with one query by performing an inner join You have to provide the relation names though.

5:11

Speaker 2: We'll return to that in a minute. Uh you actually can omit the relation names and tell Django to fetch every relation it can possibly find at any depth. But that behavior is so problematic that it's being proposed to be deprecated and removed in a future version of Django because we don't want to make it terribly easy to bring down your database. If you're going in the too many direction, you have a multi-valued relationship, perhaps from book back to many authors, from author back to many books. You may reach for prefetch related, which also requires passing a list of relation names. This performs two queries, uh still much better than n plus one when n is large. The first query is on the origin table, though a second query adds a where clause and stuffs in the result of the first query.

5:58

Speaker 2: You can also use prefetch related when you're going to fetch a single value relationship. It'll do two queries. It won't do an interjoin, but two queries is still more efficient than n plus one. And perhaps you prefer two queries. if the amount of data coming over the wire is massive or if the joint is expensive for some reason, you have a wide table. This is essentially the strategy that fetch modes is going to do. Sorry, the fetch peers fetch mode, which is one. Um, it takes a sort of lazy pre-fetch related strategy for single-valued relationships. So you can think of it as replacing select related. with a just-in-time prefetch related call that will do that second query just in time. Okay, so what failure modes is this really addressing, right? Why is this why did we need new options in Django

6:45

Speaker 2: 's RM for the N plus one queries problem? It's the issue of providing the relation names. What if you haven't your code is going to evolve? What if you missed some? What if your code evolves and now you don't need to fetch as many relations as you specified the first time? When your when your query set and the use of your query set are co-located, it's very easy to spot. When they're very far away, it's much more difficult. To spot? What if your use of the query set is several templates away? What if it goes through several layers of serialization? What if someone else is in charge of that? What if your query set is being provided to you from somewhere else or from a third-party model? The larger the distance, the harder it is to read in about, reason about, and the drift gets worse. Okay, so what do people do to discover the problem?

7:31

Speaker 2: You rely on instrumentation. Maybe you have sentry wired up and it's complaining about N plus one queries. Maybe you have a maybe you have a query logger um turned on. Maybe you have a certain number of queries in your tests. So we're giving you more strategies. Other ORMs have other strategies. Um SQL Alchemy has relationship fetching strategies. So Django now is dipping its toes into providing more strategies. We're calling them fetch modes. Strategies for fetching a model field or not. For instance, raising an exception is refusing to fetch a model field. So as I said, the fetch one uh behavior is the status quo. Um

8:16

Speaker 2: what we're shipping is uh a query set method that you can chain on a query set after dot all or dot filter, etc. dot fetch mode where you can supply a constant that will apply a fetch mode to a query set. It's an enum so you're looking at um values of that enum that are stored on the Django DB models module. So, as I mentioned at the beginning of this talk, you probably wouldn't specify fetch one unless you were trying to undo a transformation. That's the syntax for invoking what's new in Django 6. 1, the fetch peers mode. Uh it will fetch a deferred field for all instances that came from the same query set. The basic idea is that if you're iterating in a loop, you saw in my earlier slide, I'm iterating over books, I want to dot into book dot author. The principle is I probably want to process all of the books the same way.

9:05

Speaker 2: I probably want to be Yes, able to dot into dot author on the next iterations through my loop. So what that means is this is not a good choice for you if you You've fetched too many books and you only care about the first book. If you only care about the first book, you certainly don't care about all the authors for the other books that you don't care about So I don't want to pop your bubble too much, but it may not be a straight find and replace for select related. You may need to look at your code and decide, do you have any places that are already overfetching anyway? Just slamming fetch peers in everywhere is going to amplify that problem. It's going to be fetching more data where you already had overfetching. And there's the raise mode that we're also shipping. So during development, if you want to try this out, this is another way to get some feedback.

9:51

Speaker 2: Uh at the moment where the fetch peers mode would have gone off and fetched all the other authors when you've first tried to dot into book dot author, it'll just raise field fetch blocked and say you were missing either a select related or a fetch mode. Or what have you? Okay, so to reiterate, the fetch peers mode is like prefetch related where the follow-up queries are on demand. So if you're familiar with prefetch related Query sets are lazy, so the prefetch related won't happen when you type it. But when the query set is evaluated, the prefetch related will happen immediately. So the two queries that happen Uh the first query and then the second query with the WERE clause, they happen um consecutively. Um There's the possibility that some other

10:37

Speaker 2: something could have happened to the database perhaps in that extremely small interval between those two queries. Um but in practice That would happen less often than the situation well let me let me phrase that another way. If you decide to start using fetch peers What you could what you could have called a race condition about prefetch related is going to become less of a race and more just something you have to deal with. That window between the first query and that just in time second query is going to become much larger. And so that if you have It's something you have to take into account. I'm going to discuss this a bit more in a second. The principal advantage is that there's no list of relations to fetch. Um and as I said, it's uh it's used for single-valued relationships, potentially replacing select related.

11:27

Speaker 2: How we got here? Um prefetch related landed in 2012 in Django 1. 4. The next year, uh Gordon Wrigley kicked off a discussion on Django developers suggesting that we needed something like this. He opened a ticket, it was accepted in 2017 to start on you know start some motion on this. Um at Simon Charette's keynotes at DjangoCon US in 2022 on the state of the ORM. This was described as one of the Major missing features in the ORM and discussing a few possible paths forward. I'm sure it was discussed at many other Django Con uh Django Cons of which I was not present, but this is the one I have evidence of on the internet of it being discussed. And a PR for it that was authored by Adam Johnson and others. Adam Johnson is the successor maintainer to Gordon Wrigley, who authored Django Auto Prefetch that inspired this work.

12:15

Speaker 2: So after iteration and third-party packages, it was merged. To Django last fall and Django 6. 1 is out next summer. I promised you I would talk about the timeline. The feature freeze is next month, May 20th. Uh there'll be an alpha you can test then, and Django 6. 1 is expected In August. But as I said, you can play with it today either on dry ORM or by installing from the main branch. Okay, so single-valued relationships. In more detail, what kinds of relationships are those? I'm talking about foreign key fields like book. author. one -to-one fields and their reverse accessors, which you know are special flavor of foreign key. Also it would affect model fields as opposed to relations, right?

13:01

Speaker 2: Django will fetch fields like the title of a book, that's a char field, Django will fetch that eagerly. Uh but if you've opted into some laziness, if you've opted if you've opted to defer it with dot defer or dot only, you could also end up in an N plus one query situation if you kind of change your mind about accessing that data. You can also use fetch modes to confront that problem if you've been if you're working with a query set where that dis decision to defer has already been made. And generic relations if you're using generic for anything key. What is not affected? I spent all this time talking about single-valued relationships. What's not affected are multi-valued relationships and where you encounter where you would encounter that mostly is in related managers. So let's look at author and book again. I'm going to try to use the fetch peers mode with a query on authors, but on the second line after the comment.

13:53

Speaker 2: I'm dot I'm calling for book in author. books. all. Now there's some private state on the model instances that will record which fetch mode was used to fetch these objects. Whoops, resume presentation. Uh if I print that private state, I'll see that the fetch peers information is persisted all the way down to the child objects. However, this will still produce n queries for n uh n authors because on that second line of Python, I'm c I'm invoking a new query set. I'm doing author. books. all. We're not telling the model manager for book what to do. The model manager for book is in charge of making its own queries on books. So I just wanted to reiterate, this isn't something that you can find and replace all of your prefetch-related calls for multivalued relationships.

14:43

Speaker 2: You still need to be explicit when you want to prefetch multi-valued relationships. Okay, let's look at some potential pitfalls trying to um you know apply this in your code. Um I alluded to this earlier. On line two here, I'm converting a query set to a list, which means I'm getting all of the books, but then I'm immediately throwing away everything but the first. This is not a situation where I would want to fetch all the peers for all the all the other books. Luckily there's a fix and it has less to do with fetch modes and more with something that was already a problem in this code, which is that it was overfetching in the first Now this code is safer to use with fetch peers because I've corrected a problem with overfetching. Maybe that should be.

15:34

Speaker 2: Um another possible thing to keep in mind, uh this changes when your data is fetched. Um so let's say I want to use fetch peers here uh to fetch some books. And let's say I want to do something that affects the authors. Well, on the first iteration through this loop, remember the fetch-peer strategy is just in time. So it's going to fetch the author down on line six after I've done something that potentially affects authors. But on the second iteration through the loop. Django will have already fetched the authors for all the other peers. Now on line five, I'm going to do something affecting authors where the authors may have already been. um fetched. I may have already had to deal with does not exist or things like this. Again, something to be conscious of when you want to start putting

16:19

Speaker 2: fetch peers everywhere. You may need to look at how your code deals with related objects and think about the fact that your data fetching is now more lazy and may happen before or after um yeah, just stuff you're doing during a loop. Generalizing from that a little bit, to use the fetch peers strategy means that you're doubling down on lazy data fetching. And this is the insight I mentioned that I owed to Simon Charette's keynote. Up at the top, I'm grabbing a book, but let's say that you know there's a long function here, and perhaps during that time elsewhere authors and books are deleted out from under my feet. Now I'm accessing book. author. I need to be prepared for a does not exist right to escape. This is a problem you wouldn't have if you'd use

17:04

Speaker 2: select related up at the top, because remember that select related doesn't interjoin in one query. So there's no possible race condition where something else can happen before you have the opportunity to fetch the authors. So with great with great power comes great responsibility, I suppose, to reason about the reason about your fetch strategies Okay, I promised you um an example about reducing boilerplate, right? What if you don't want to uh uh uh copy this call to dot fetch mode all over the place? Django has a documented pattern for specifying default query sets for your models You implement them on managers by overriding the get query set method. Um because dot fetch mode is just a method you can chain on any query set.

17:50

Speaker 2: Um that's all you need to do. There's your manager and you can attach it to your book model there on the penultimate line. Notice I haven't called it the book manager. I've called it something generic that I can reuse with other models. I've left commented out another pattern. Perhaps your project uses not model managers but query sets that you like to use use to generate automatically a model manager. If you find yourself in a situation where you need to have your cake and eat it too, both have a custom query set and a custom model manager, that's the syntax for Uh a project that may be uh exposing reusable filters on a custom query set, that's the syntax for also being able to leverage a custom manager to get default query sets. That's what the exception looks like in the raise mode.

18:37

Speaker 2: This is a great tool for uh debugging or experimenting with this feature if you just want to get more feedback about where your application may already be issuing n plus one queries. Okay, and I promised you an extension. So what if instead of an exception I want a warning? Let's just walk through writing your own fetch mode. I could have made this example a little shorter, but I wanted to show you all the bells and whistles. So a fetch mode has a fetch method. It takes a fetcher and an instance. Really, what I'd like to just do is use the nice error message that the raise mode gives me. I could write my own message, but I want to delegate to the raise mode because it has a nice exception message. So I just call that. I expect it to raise. I catch it and I log it out as a warning.

19:24

Speaker 2: Finally, I also tell the warnings module to not include stack frames from Django because I'd rather see stack frames from my app Uh and then finally I delegate to the fetch one fetch mode to get Django's default behavior. Um Obviously, this I hope this inspires some imagination that you can compose your own fetch strategies in a similar way by you know delegating in some cases to the fetch strategies that we're shipping with Django. All right. I'd like to hear more about this at the Sprints. I also want to make a plug for the Sprints. I'll be at the Sprints. I can help you get set up with testing against the main branch of Django in your projects. uh testing out this feature or um other aspects of contributing to Django uh like working on tickets or uh contributing

20:11

Speaker 2: pull request reviews, so make sure to grab me there. Thanks.

20:24

Speaker 3: Thank you for the great talk. I think it's really important to go into details and to make all of us more aware of new goodies that are coming to Django six point one. Any questions from the audience?

20:42

Speaker 4: Thank you. Um I didn't quite understand yet if this fetch uh PS mode is Recursive or transitive going down multiple layers. So from book to all like if I say book dot objects fetch b uh fetch mode peers and then go book, author, city, country, blah blah whatever. Is this all the way down?

21:07

Speaker 2: Yeah, I think it's one query per table. Yeah. You mean how do the queries behave or does the behavior cascade? In the single-valued direction, it continues to cascade. As I said, it's only for single-valued relations. Um yeah. Uh

21:31

Speaker 5: this looks awesome. So first off, thanks for showing it. Um I'm just wondering if a book has like authors and library and other things. uh that you then have additional joins on, uh is it possible to have like a single query to fetch all of those or like does it does it fetch whenever it encounters a new item or

22:02

Speaker 2: Excellent question. Yeah. It's lazy. So it's going to fetch when you encounter when you dot into something new, right? So if you're dotting into author, it'll only fetch authors. If on the next line of code you dot into editor, then it'll fetch the editors. And that prevents overfetching. So unlike prefetch related, where you may declare, you know, that you want to or yeah, you you Uh you you want to prefetch some some related instance, but what if the first query executes and provides no results? You know, uh well I guess the ORM still prevents you from getting into that situation, yeah, because it won't quit So is your is your question about can I still do a join to get everything efficiently in one query? I think that's a case where you would rely on select related, right? So this doesn't replace everything.

22:47

Speaker 2: You know, if you want select related to do a join. use select related. Hi. Hi.

22:57

Speaker 6: Thank you for this w work and talk. Um Uh i if I want to test my project and find out the the the cases where I'm doing things inefficiently, is there a way for me to just say make it all um like raise or or things like Something like that. At the level of like a setting so that I could also catch not just my code, but also like third party libraries that I use.

23:24

Speaker 2: Uh yeah, so catch could mean two things. Catch can mean I want to get some exceptions or warnings, or ca uh because if it's kind of transparently just fetching all peers, then you won't gather any information about whether your third-party dependencies are fetch peers ready. Uh but is your question how can you So we have shipped something that lets you just raise, you know. I see globally, you want to set set it globally. Um yeah, Django's not shipping anything for that in 6. 1. I think your best bet would be to uh so I guess so the premise of the question is there are query sets I can't control. I don't want to use the model manager strategy because there's third-party apps I can't control. Yeah, my advice to you would be, because it's Python, do something at runtime, maybe in the ready

24:10

Speaker 2: method, swap out a constant on the Django DB models. If you want to be debugging third-party apps automagically, that would be my advice to you. But it's as you might imagine, we're not shipping anything for that.

24:29

Speaker 3: One question from the top. Thank you.

24:33

Speaker 7: Thank you. I think it's fantastic. I think my question just got asked, but uh Uh I I will reframe it because uh maybe it was something different. But first I would like to start with an anecdote, uh which I find personally funny Where I work, I was suggesting to use Django Auto Prefetch because as uh in many uh companies uh there are a lot of performance issues that are caused by N plus one And it should be uh like our role as developer to take care of it. But if we are just unaware and if uh like our DevOps people are not really looking also at what happens Most of the time

25:19

Speaker 7: it just like the the issue is is is undetected. Anyway, I asked for uh Django Auto Prefetch So from Adam Johnson, at least maintained by him, as you mentioned. Maybe I forgot some titles, but used to be I think in the sterile console for Django uh core developer for years is also in the PSF uh whatever The answer was uh we have some liabilities to our our customers who might ask who is this Mr. Adams Chains. So you know at some point we will have to migrate to Django 61 and then the problem is solved. Um anyway, the question was

26:06

Speaker 7: Is it uh like will it be possible to have a setting so that the default mode without having to define a custom uh manager would be raise Do you think this is a good idea? Because for a new project it would maybe make sense.

26:25

Speaker 2: I I think you you got to the crux of the issue. For a new project it makes sense. What do you do with a community that's says depending on mature software that's been behaving in a certain way for 20 years? So I I think when this uh you know this was part of the original discussion. I mentioned that there was a Django developers thread about this in twenty thirteen. You heard you heard objections to this feature, like no, we shouldn't do this, and then you heard um you know voices in favor saying that eventually the peers strategy should become the default. I think it's gonna need some time. I think we need to let third-party apps become fetch peers friendly. We need time for apps to address their potential overfetching, right, that could be amplified if we just let people slam this on by default. Um I think we need to see how this evolves, and then we can revisit the question of of global settings.

27:12

Speaker 2: Oh okay. You're right. Right. So uh but it this is a similar principle b which is because you're you're sort of saying I know what I'm doing, including for third-party apps out of my control. That for the apps that I control and the ones that I don't, I'd like this to be the default. And again, this is kind of at your own risk, isn't my answer to Shy. You could implement something in a you know appconfig. ready method that you know swapped out something to produce that behavior if you really want it. But I think it's just it's too in In my opinion, it's a little too early for Django to make it too easy for a user to make inferences about what's safe or not safe in the third party dependency. Like so if you go through this exercise, you'll find like if you turn on, if you try to hack the hit the raise mode on everywhere, you'll find errors emitted all over the place.

27:58

Speaker 2: Maybe from Django. contrib. auth or maybe from Guardian or just all over the place. I think we need a little more time before we sort of create a setting that gives that much uh gives the user that much control over the behavior of third-party apps.

28:16

Speaker 3: Any other questions? Ah

28:24

Speaker 8: thank you Jacob uh great talk. Uh can't we have a fetch mode setting in class meta of each model so that we don't have to write the manager for every model?

28:34

Speaker 2: Um it depends on what it looks like.

28:39

Speaker 8: Okay, cool. So we could do it right?

28:48

Speaker 2: But you know the the documented pattern for generating a default query set is you know we have two ways to do things. Sorry?

28:55

Speaker 8: Through the manager.

28:56

Speaker 2: Yeah. Which again you can reuse. I showed an example that wasn't specific to book.

29:01

Speaker 8: Cool, thank you

29:08

Speaker 3: Okay, I think it's time for another round of applause.

Presenters

Conference

DjangoCon Europe

April 15 - 17, 2026

Athens, Greece

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos by Jacob Walls

More videos from DjangoCon Europe