Django Through the Years with Andrew Godwin

Speaker 1: Hello everyone, uh welcome. So yeah, as mentioned, um I'm Andrew Godwin. Uh I currently as a day job am I a principal engineer at Astronomer What that means is I work on Airflow, Apache Airflow, the open source project, as a good part of my day. I have done Django for quite a while. I've been working with Django as a user since 2006. I've been writing stuff for it as a developer in somewhere or another since 2008. And in general, I do a lot of Django. One of the other things I really love is I do love a lovely, relaxing conference. Where I don't submit a talk and I just turn up, chat to people, watch some good stuff, and then tomorrow afternoon I'm on a panel. I was like, great, that was my plan for this particular conference. And then about ooh, I think 7 p. m. on Monday night, I get an email

Speaker 1: where, unfortunately, someone has pulled out for various reasons. And they're like, well, can you fill in? I was like, of course, that's one of the things I do. And uh there's a very important thing here, the word free choice of topics. This is a very dangerous phrase to give me because boy are there lots of it. So one of my choice I could have done a CNC machining talk. There is some Python in this at least, right? Like I considered doing just a recap of my camper van builds on YouTube. Always fun. There's also of course I could have a one thing about my cats. 45 minutes of photos will go down super well, I think. All of these probably would have been better talks than what I've got for you. I apologize. But at the end of the day,

Speaker 1: one of the things I sat down was like, well, I've been using Django since 2006. This is approximately half of my life. It's a terrifying number. Not super happy about this. But as a result, like I've kind of been doing Django for a while. I've been coming to Django Con since it first happened in 2008. I somehow have not missed a US DjangoCon ever, or a European one, in fact Because I'm a weird person. And that first DjangoCon was kind of a transformative experience for me. Back at that time, I was like 19 or 20, and I was at university in the UK And I had just released a new piece of software called South. For those of you who weren't around back then, which is probably most of you, at that point Django didn't have any kind of migrations or schema change

Speaker 1: system. And uh Russell Keith McGee and Simon Willison had both released different ways of doing it. And I, being the young, naive, full-of-self-importance developer I was, was like, oh, I can do better than them. And so I released a thing called South, about I think it was a few months before Django Kong. And then approximately, I think it's like maybe Seven weeks before DjangoCon, um, Simon Willison emails me and says, oh, we're having a panel at this thing in America. Remember, I live in the UK at this point. We'd love to have you come over and talk about it. Yes, 14 years, two months ago is again a worrying thing on this. And so I'm a poor student university, I can't afford this. And I won't show the rest at email theory, but like Jacob Captain Martin and the DSS

Speaker 1: step in and say We can probably fund your trip over to be on this panel. And so with about three weeks notice, I buy a plane ticket to America where I've never been before. I fly to Silicon Valley, where I've never even seen properly in movies and television. And I end up at Google Campus. with nobody I know apart from Simon who I'd known for like a year at that point. Like just a whole load of people I'd never met before. And it was some of the most welcoming, lovely people I've ever met. I mean, one of the reasons I'm still here, I don't do Django as a day job right now, I'm still here A lot of it's because of the people and the community. If you want a good idea of what that panel looked like, I have grabbed this this particular photo. Russell, myself, and Simon all looking considerably younger in this picture.

Speaker 1: There is a wonderful full uh 480p, the highest resolution you can imagine. Uh full video of this still available on the internet should you wish to go through it. I watched about three or four minutes and I I just couldn't watch anymore It is nice though seeing all three of us have both changed and haven't changed in equal fashion. But one of the things was interesting about Django 's O days, like Django was very different Back then, right? Like it's kind of sometimes hard to appreciate from where we stand now in the present just how much Django has changed. So what I've done is I've gone through some of the old release notes and things from that particular timeline. To remind you, so for reference, Django 1. 0 was released just before that first Django Con. It was kind of deliberately timed to coincide with it.

Speaker 1: So the release announcement starts with this, where we're very excited to announce we've had a thing called documentation, which is a brand new thing, very exciting. And that's like, you know, the bit uh that's the thing these days, you're like, well, Django is so deeply associated with documentation, you couldn't conceive of it not being. But that was the new doc site was the one of the big new things. We have the the new admin application, the one you now all know and love. My favorite thing here is this no more class admin in models, which if you've never seen it, congratulations, it was awful Uh you put all of it in the same place and it was really weird. Um and also uh new forms, which we now just call forms, of course, but back then there was old forms and new forms, and that was very fun.

Speaker 1: One of the things I found uh I I'd forgotten this arrived during my tenure and and using Django. We didn't use to auto escape variables and templates HTML objection used to just be a thing and this this came along so that was great. And then one of the things I had totally forgotten about, in 1. 0 we refactored that well-known package you all know and love, Django. contrib. comments which is of course no longer with Django. It was it was removed seven years back. But like I'd forgotten this even existed. I used this. I I was a user of this particular package. I'd totally forgotten it existed. And that first ShanghaiCon was fascinating in many ways, right? It was the first time we'd done one of these. We as a community was exploring what it meant for us to be a community. Some of the talks are really interesting. I have a strong memory of Malcolm giving an amazing talk on the RM that I went to, like a

Speaker 1: room somewhere upstairs in Google HQ. But I think the most notable talk I saw, and a lot of other people saw, was the somewhat infamous Why I Hate Django talk by Cal Henderson Now, uh, I apologize to Cal who ever sees this. There are no flattering pictures from that particular video stream. This is the best I could do. Um But Cal did a fantastic talk, of course, uh on this whole list of reasons why he hated Django 1. 0. It was a great, sarcastic, honestly like loving talk, well done. One I think we've always tried to have some version of it. Many Django events since then. But it's always worth looking at the list of things he didn't like about Django 1. 0. This is a preceded list, a shortened list

Speaker 1: of those things No multiple databases, dumb SQL generation, cookie-based sessions were not in there, no good debugging tools, no migration system, and it wasn't smug enough. I'm pleased to report we've fixed all but one of these things We are still unfortunately not very smug. We're working on that. We'll get there eventually. One of us needs to buy a race car at some point, I'm sure. Sorry. You get the hot takes when it's a last minute talk. That's how it works. Um got that one. Um but the thing I want to talk about that's my personal history with Django is the migration system. So The thing I kind of came into Django for, and the thing that made me a core developer back in the day, was developing migrations

Speaker 1: Back then my initial package was third party. It was an add-on. It was called South. I should say this in a talk at this point. South is a pun. Because birds migrate south for the winter. You'll be surprised how many people got this like 10 years later. Someone got it literally yesterday outside. And was like, wait, I I didn't know. So it it happens. I'm just sort of put that out there. It's a pun. It's deliberate. That's why it's called South. Yeah, there we go. Good good feedback there from the audience. But one of the things there is that like this panel you saw me on, that was my second ever talk. So on my website I have this list of talks. It contains every talk I've ever given. along with country flags. When I designed this, you can see I did not plan on having quite as many countries as I do now.

Speaker 1: So the CSS has broken a little in the years since. But this is a full list of every talk I've ever given with slides and video if they're there. Right at the bottom are the first two talks I ever gave. There was a lightning talk at my local meetup back in Oxford. That's my first ever appearance. And I will point out if you want to start talking and doing public speaking, lightning talks are where I started. They're really, really good good, right? It's one of the things to remember here or other conferences, they're fantastic. The second ever appearance was that panel. And you know, Oxford to uh Mountain View. Quite a geographic distance and cultural divide. Um, I was a lot more out of my depth. But even then, like that kind of kick-started all the things I've worked on since, and also a lot of the open source work, even beyond Django, I've done there. Like I've always said that.

Speaker 1: Conferences are the positive feedback that keeps my open source like engine going. It stops my burnout. Otherwise, all you ever see are the bugs forever. So like there's a good feedback loop there I really valued. So what I've done is I've taken some slides from various South presentations over the years just to show you some of how it developed as well. So you see my graphic design here in Just stellar graphic design. I'm top of my league at this point. So this is the at that panel we had short presentations on what each of us were doing. um Django evolution and demigrations being Russell and Simon's solutions respectively. They both don't die like life. South kind of overtook them. I believe Demigrations was MySQL only, if I remember correctly, which was kind of its

Speaker 1: death now, I think. Jangue Evolution was more full, but eventually I think just South won out for one reason or another. And so, you know, I was there presenting like, oh, like if you remember SyncDB, congratulations. Uh you're probably old at this point, so well done. Um but I was there talking about like how well, you know, we have this thing called SyncB, it only makes tables. What if we didn't just make tables and maybe deleted them and changed them occasionally And then one of the things I I did call out specifically was, and it's a thing that still a lot of migration systems don't do. So like these days I'm using Alembic because Airflow is built on Flask and SQL Alchemy, and that's Alembic is part of it merge migrations understanding like merged branch histories I still think is one of the innovations that

Speaker 1: I probably stole it from somewhere I imagine But it's still one of the big innovations I think South bought into the Django world, these of like we can detect when two branches are merged together and then tell you it's wrong. And obviously Django migrations inherited this particular So like this is one of I think the innovations I was really proud of at the time of like yeah like it it understands nonlinear histories and branching histories and stuff. So that was state of the art in 2008, let me tell you. And then about six years later, um the time I hear like South kind of becomes popular, becomes dominant, becomes kind of everyone installs it with Django basically as a assumed thing. Excuse me. And then we kind of like, well, we should probably put it in Django Core.

Speaker 1: And so a I started Kickstarter to fund myself as a freelance time to put this code in Django Core. And that Kickstarter was funded to a ludicrous degree. It was like 14 or 15,000 British pounds. It was a lot of money at the time for me. I spent a good, you know, like three, four months full-time on this. And then at the end of that particular project, I gave some presentations already done. So like the whole plan was to build South into Django. And then the initial plan was to just copy it directly. And I was like, ah, but There's so many problems I've seen over the last six years. What have I got a chance to fix them? And so I came in with this like, well, I have a chance to rewrite. and keep the same logical flow but have a different sort of technical flow.

Speaker 1: And so I have all the things of like, and this is still true in Django today, I separated out migrations into two different portions. There was the schema editor. This is the sort of SQL abstraction of here is how you change a table on Postgres or SQLite or MySQL. That was kind of the low level. And then above that was in the migrations framework, the high level of like this understands models and how to like merge migrations and gives you the the commands and so like the idea was always you have these two separate pieces and this has persisted today. Um my initial vision was that I was trying to be very nice and be like, well, I shouldn't just have one migration solution. We should leave the schema editor in place so that if someone wants to write a different migrations framework Half of the work's done for them already. They can already have the abstraction layer for database changes. They just need to write the sort of the new front end.

Speaker 1: In reality, what we've seen is people use this occasionally for like dynamic table changes at runtime, please don't do that. It's not great. But in general, it's not been used quite as much as I thought it might be. The other big change from South Some of you might remember this in South, you had migrations, this very top of the file, and then down here was what we call the frozen ORM, which was every single model in your entire database was written down as a set of tuples This is so we could detect changes, right? We'd load this into memory, we compare it with your current models and go, oh, you've added a field, and then we'd do that particular edition. This was stolen from a different migrations framework. And I forget which one. I definitely did steal the idea though, so I want to make that very clear. All good art is in some way borrowed, right? But what we did in Django migrations, I was like, well, we don't need this.

Speaker 1: Because in fact, if you run through all of the actions in order, we can actually work out what this is. So we can talk a fake migration set on an internal in-memory database. And so one of the big things with the change to Jang migrations was migrations are now like this as they are today. They're nice and short because what we do is If we want to work out what changes have happened, we load all the migrations into memory, run through them in memory to build that frozen RRM state, and then diff that to your current models. So it still works like that. And I think this I think this generally might have been actually a thing I invented. maybe um one of the few innovations actually authored um and that's I think this this still work looks good today. And then finally um later that year I think at one of the I'm guessing it was at Django under the hood given the name of the talk, which is migrations under the hood.

Speaker 1: I did a bit more of a deep dive. And my favorite thing from these slides is my initial plan I announced when I first thought to do this was We'll put that schema editor stuff in Django and then we'll have a South 2 that is just the user interface. And I was like, well, but that's not really batteries included. So like, well, what if we put all of it in Django? And then I backport it to Django 1. 4 to 1. 6. And then I was like, well, what if I didn't do that and just didn't do the work and just put it in Django? And that's what we ended up with. This also started my particular hatred of swappable models, which Uh was a new fe yes, thank you, Russ. Uh a n a relatively new feature at the time. Uh this slide is maybe i illustrative of my feeling on top of all muzzles. Um they they now kind of work.

Speaker 1: They're still not perfect with migrations. We made a lot of improvements over the years, but back then swappable models and MySQL were one of my two particular things I would always make fun of. And so that was that was the thing. But the thing to consider is, you know, I've taken you through a good portion of the history here. And now in many ways, about migrations, at least in particular. It kind of just works, right? Like one of the things I noticed I sit down and use Django having come back from Flask and Alembic and my C and SQL Alchemy or coming back from even a different language. I'm like Oh, this is and obviously it's familiar and warm because I wrote it, that does help. But it also has that user and developer experience that's I think honestly

Speaker 1: hard to match in most other frameworks. And one of the things I consider on a daily basis is like, well, how did we get here? Like obviously there was some part of Django's mission I understood in those early years. years and I sort of built towards it, we built Django around it. I think, you know, the the illustrative phrase from Django has always been in the header of Django Project. com for a very long time, the web framework for perfectionists with deadlines. And there's a lot in that phrase, right? I'm sure it was I don't know the history of it, I'm sure people here can expand on it, but it's very powerful in its sort of precision of selection of words right like perfectionists with deadlines think is a very excellent description of many of us. We want to write good software.

Speaker 1: We're also pushed to write software in a very limited amount of time. I have had to come to terms as I become an increasingly more senior engineer, both with having to do lots more, you know, like making decisions, which it turns out is a big thing you have to end up doing. But also with just having to be okay with imperfection, right? Be okay with like well it turns out if you ship a thing that's 90% complete people won't notice I mean actually Generally works. And the thing that is shipped and out there is better than an imperfect thing that never ships. And in some ways, Django kind of comes to represent this. A few years ago we started saying Django is now boring software. Many of us are very proud of this. We mean it in a very nice way. Django is not surprising. Nothing about it is gonna like

Speaker 1: come out of a left turn and shock you. You'll always be like, oh no, like I understand how it works. It's predictable, I can build with it, it's just a good piece of software that I can work with that never goes against me. And I think that's also a very interesting facet of how Django exists in the modern world. As I'm sure you all may have noticed over the past five or ten years. The web went all in on single-page applications, right? We entered the world that don't no no hollering, come on. The world went all in on this particular thing of like Hypertext and CSS are old hats. We can do everything in React or View or whatever, right? Um I am maybe an old man before my time.

Speaker 1: A little bit. I was never particularly a big fan of this, but of course I went along with it. That was the whole point. And in some ways Django was relegated to this like, well We're just gonna take a back seat. We can do good APIs. The ORM is still good, right? You can still use the pieces that make sense there, but templating, Django forms, all that stuff maybe has fallen behind the wayside a little bit, right? Right? Like who needs forms when you've got React components? Who needs templating when you've got, you know, React components, I guess? Everything, everything comes down to that. And and like one of the things I had, especially over the pandemic, was a slight crisis of conscience or crisis of existentialism in a way. Of like, well, what is Django? Right? Like one of the things we've struggled with, I think

Speaker 1: it's kind of an open secret for the past five or so years is we've done a lot of the big features. Like Django kind of got to a good place. It became boring software in that sense. sense. And it's a very interesting of like, Paul, is Django finished? It was one of the questions we started asking, I think, like four or five years ago. We were like, well, what's left? What what do we have to do? I of course came up with channels and And that was a big project for a while. We've made good progress on that in the last five years, I'd say. Like we have ASDI now. It's a standard people use, which is always surprising to me, is the person who's like, we should have a standard. I will write one and see what happens. And it turns out. It turns out if you turn up and do the work, people follow you. It's very weird. And like we have, you know, asynchronous handling in Django now. The asynchronous RM is in the most recent release.

Speaker 1: Like it's not all the way down, but it's a great start. You can start using it And so I really struggled with this, especially in the pandemic when I wasn't writing a lot of Django then either. I was disconnected from what I'm going to say is kind of my emotional support network. of conferences, right? Like having people come up and talk to you about like, oh I loved using your software, I love your thing. Like you may find it awkward. If you ever want to go and thank an open source maintainer for the thing they've written, please go do it. It makes us all feel very happy. We really like it. It may we may sort of go, oh, it's fine, but we're like, oh, it's so nice on the inside. So please thank everyone you see for the work they do, as I do regularly with everybody else here. Like, thank you for taking over the thing I left behind. Thank you for fixing all my bugs. Right? Like that's the thing I do as well. And for some time I thought, well

Speaker 1: Is Django destined to be the API layer, right? Like is this one of the things where what we end up as is just, well, the ORM is good, and for reference, the ORM is about half of Django's code or more by volume? And so Django is in some ways an ORM with stuff on top of it, right? And so maybe we just take some of the other stuff off top and put some different stuff on top and then it's an API layer. And this, you know, for a long time I think looked like the way. Like we saw mobile apps, we saw rich web applications, they had sim similar APIs. GraphQL became a big thing. And then these days, GraphQL is kind of becoming less of a big thing. Companies is going, oh, it turns out having a query planner is part of your API is maybe not the best approach. Or I think people do arbitrary joins. It turns out

Speaker 1: giving everyone control of arbitrary joins is maybe not the best way to write a performant web application Um and so like this kind of came away. And then like, you know, in the Django world, we have Django Rose framework, which I've used uh at two or three different places these days, and it is good, but we've also seen progress in other parts. One of my particular favourites is Fast API, which of course is based on all the new uh the new breed of asynchronous by default web frameworks. Um built on ASCII and in many ways they drive its development much more than I do. Fast API is both that and typing. Like you do use Python typing to declare all the types. Like a thing I never thought I'd be saying 10 years ago is

Speaker 1: Python has typing and boy do I love it. But I really do. It's a really fascinating, strong thing that I'm a really big fan of. And so you look at Drang of Red Framework and it's very good for the era it was designed in. And I think Fast API and Pydantic as a big part of Fast API and kind of its serialization and somewhat of its parsing layer are also really interesting to have good ideas. And I look at those and go, well, here's a thing where like we could embrace the API future and really make that a strong part of maybe Django itself as well. It's always been a bit weird to me we left Django Rest framework as a third-party thing. It makes sense, obviously. But it's always like, well, everyone's using Django for APIs. Why do we just maintain the Forms library and not the API library?

Speaker 1: That was always a very weird thing to me. The other thing to I've come across in the last you know year or so even is HTMX. Now if you've not seen HTMX, it's a different way of doing well, I'm gonna say is hypermedia, but like it's like sorry, a full confession. I was using jQuery until last year. I love jQuery. It has a mental model that fits the way I think perfectly. Big fan of it. And I tried React, I tried writing Vue, I have a whole Vue application I written for my personal like automatic bag scanner. Different talk. You put a bag on a thing, it reads all the tags inside, it tells you what's packed in it. It's great. That's all written in view. I cannot maintain it. It is impossible for me to understand. I just hope it mostly works and never

Speaker 1: have to touch it again. But like HTMX is one of those things where like, ah, like this is significantly different. It is a way of writing web pages as web pages with progressive enhancement. Progressive enhancement was a big thing when I sort of grew up in the web world in the 2000s, right? Like as an emerging developer and young person then, we kind of lost it in the decade since, I think. But like, you know, you write code like this. You have a normal form that you put a few extra tags on saying, oh, by the way, you can post this via Ajax or WebSockets with Ajax. enough um and do stuff and like this code in a form like it's what three extra tags there I think maybe um and I'm using crispy forms too I 'm still a big fan of crispy forms

Speaker 1: And then a little extra code in your view to work out if it's a HTMX request. Because the way it works is you send back a partial bit of HTML and it swaps that in the page. And so what this does is there's a small sub-template that renders just the form with all the validation errors and all the error messages and they're all formatted nicely with CSS. And then it just works. As you type, if you look back here, there is a trigger on change. As you change the values in the form, it Ajax submits it back to Django forms, it runs the validation, it works out the new piece of HTML, swaps it into the page. happens in like 100 milliseconds. This is not the first time I've done this. I wrote a thing to do this. I think several, I think maybe Lanyard had this, if I remember correctly, Simon mind though. I think we had this back at Lanyard.

Speaker 1: It's not a new concept. But it's so well written and formalized here that I'm like, well, maybe this is the way again. Because like in recent years we have seen people falling out a bit of writing really, really wit rich single-page applications and going It turns out the web is built around ordering pages off of a server and they come to you and displaying them, and doing things that way can be more efficient. There's a very good talk at DjangoCon Europe, if you're interested in HMX, about converting a whole application to it. The application was particularly suitable for conversion. It was mostly showing rich media, but they like halved their load time. and everything feels more responsive and there's a massive cut in the amount of code. It's a big success story. So if you're curious about this in particular, I encourage you to go look at that particular talk.

Speaker 1: But you look at these particular things and you kind of start thinking like, well, you know, I said earlier, is Django done, right? Like that's one of the things you always talked about, like is Django finished? Are we good? Do we done you know Close the door, lock it, make sure the lights stay on, make sure the power's on, just keep maintaining it. And like I have always been, I think, a incessant optimist. I'm sure several other people who know me for a long time will agree I'm always pushing Django to go forwards. And I think Django does still need something else, right? It's not done. It feels to me like I sit down and write websites with it, and I'm like This is great, but it still feels like I'm doing too much work. There's more shortcuts. There is more developer experience that I could have improved here In

Speaker 1: many ways, I'm kind of the wrong person for this, right? Like you look at me like I'm a principal engineer now. What that mostly means is I sit in meetings and defend my team from all the other meetings they don't have to go to And then sit there making all the hard decisions all day and going, well, these are two bad choices. We'll pick option B because it's slightly better than the option A. Like that's that's what the job is, right? Like I write code One or two days a week, maybe less, depending on the week. And so in some ways, like my biggest touch point with modern software development is my hobby projects. Right now I am writing a Django website for the first time in two years for a personal hobby project. That is the closest I got to it in that time And so I think in many ways I, and many of the other long-term Django developers, are in some ways

Speaker 1: not the right people. Because as we progress in our careers, we end up in roles that are more detached from where the code is. And so one of the questions is who does know best, right? We don't know. A lot of the drive in the past has come from people actively using Jan. I will say my time at an agency writing websites like once a month was the most productive time. That's where South came from. South came from my time at Torchbox. These days they make Wagtail. But back then we were sort of experimenting with Django as a sort of change from our um gosh what was it i i forget what language it was it was an awful uh XML based language Cold Fusion there we go it was from Cold Fusion yeah there you go wonderful language no um And like I was embedded back then, right?

Speaker 1: Like I was making a new website every month and I felt the pain of migrations on such a regular basis as we changed them and clients changed their mind. and then someone gave us data that was wrong and had extra fields in it. These days it's harder. And so I think there's a big thing there of like it's important for us as a community to reach out to newer developers and those doing like this job still making websites and doing this stuff full time as well as people like me Who are at these big single product companies where like what we care about is scalability and that it responds quickly and we can maintain it as a big team. That's not what everything is. That said, I have got some suggestions. So I do think that it will be sensible for Django to integrate a first-party API frame. This is my current particular thing.

Speaker 1: I'm like, it's probably time we did this. Um, and I have a history of going, we should do this thing in Django, and then kind of pushing it far enough. And occasionally I finish pushing it, occasionally I don't It does feel like this is it's time we did something about this in particular though. And I think we can pull from Django, REST Roman, and Fast API and others. But I think there's something there. What it is, I don't know. I'd rather not be the one to fully define it, honestly. I just feel like there's an obvious need there. Like as I write a lot of Django apps, this is the thing I'm always going of going to third-party solutions for. I also think there is room for a more high-level HTMX templating form handling class. Like let me write my form -based, navigation-based websites more easily. easily with like

Speaker 1: give me pagination done automatically, give me all the form handling validation automatically. Doesn't have to be first party. I think one of the models over the years we found works well is develop the thing as a third-party package first because then you're not you're not tied to Django's super long release cycle. You can make changes more than once every nine months or whatever. And then when it's reached a good point point bring it in. And there is a Django HTMX. That's what you saw in the previous code examples me using. It's a good start. I like it for what it is. It feels like if we are to take Django as a presentation layer helper as well, there is also more there. And then finally, uh one of our oldest things, along with composite keys. Wouldn't it be nice if the ORM was separable? Obviously, everyone said this.

Speaker 1: I literally first heard this in Silicon Valley in 2008. Where people like, oh, we could make the RN separate. People would use Django without the views. We've never done it. There's a long list of reasons we haven't done it. It's very difficult. But as someone who has to use SQL Alphony right now, boy, would I love it if I could do this. Because I want to use Django for its RM. I don't necessarily want to use it for its view or routing layer. And of course you can just use one piece. is difficult, but like I still, it's an old problem, it's still a bad problem. We should at some point work out what to do here. Of course, this is just what I want, right? Like this is a very personal, uh, very opinionated idea of what's what's needed. And of course, again, I am not Particularly wide-ranging in my views, right?

Speaker 1: I think we all have to come and work together and work out what we need. But one of the things I've noticed over the years, and this is true both in open source and also inside companies , Everyone is a team, but one of the very important things that you come to realize is that behind every big feature push, behind every big change, there's one or two people driving it. They're actually doing the work that it has to be like writing all the code commits, but someone has to, as I say, turn the handle Someone has to sit there and always be like pushing forward and checking statuses and like this is what my role is in some ways as a principal engineer is to sit there and go, I have a plan for this project. I have a lot of developers I trust who are amazing to write it And my job is to make sure that they're all doing the right pieces and it's moving along.

Speaker 1: This is also true in open source. All the big features I have been involved with in Django. Have had me or somebody else pushing them along. And in many ways, I think one of the reasons Django hasn't had a big feature push in a while is that there isn't anyone doing this. Like a lot of us burnt out and don't do it anymore And so I think if you are thinking about, I wish Django had a big thing, I'm not going to tell you you have to do this, right? Like if you want to, please come talk to us about it. But like this is the reason maybe hasn't happened, right? Like it takes, it's not just that we have an amorphous team that you send ideas into and pull requests come out. Right? Like we have a collection of people who have interests and they also get excited, right? Like one of the best reasons, I mean Simon Willison

Speaker 1: is one of my role models in this. Simon is the most enthusiastic person I know I learned Django from him because just because he was enthusiastic about it, to be perfectly honest. I wouldn't be here if he hadn't gone, I've got this thing, it's very exciting. You should learn it, as if you've met Simon, I'm sure you'll be familiar with. And that was in two that was in 2005 and got me on this path. So it's important to have that enthusiasm and have that drive. And so my call to action here at the end is not is not so much like, you know You can come and help in any way possible, but like it's one of the things where like it takes inspiration and vision to drive a project like Django, right? Like one of the things it really takes for a big feature is having a vision of like I know what my end result is. I want to drive it and I'm willing to help.

Speaker 1: And there's plenty of other calls in this conference for help in all the different parts of Jang. As open source developers and a community, we are perpetually short of volunteers and people. That's the way things are. People burn out. It's totally understandable. But if you want to help us inspire and think of things and being enthusiastic, that is a skill as well. It is important sometimes to just turn up and go, no, no, no, no, I want the thing. And I've got a plan for the thing, and I've got a rough idea of how we do the steps in order to get to the plan for the thing. And that can be a big contribution in itself. So if you're interested in a big feature in Django, if you want to just like talk about it with me and get inspired or just chat about what it means in the future and maybe figure out what a path could be. I am always around to do it. You can do it here in person, you can do it over email or whatever.

Speaker 1: My email is on my website, which we'll see on the last slide here. Yeah, like my ask to you is both in Django and elsewhere, don't just be the person who turns up turns up to maintain, maintenance is important, be the person who turns up to inspire as well. Thank you very much.

Speaker 2: All right. We have uh five minutes for questions. If anyone has any questions for Andrew, I'm sure we can do that right now

Speaker 3: Uh this might be too lofty of a question for the context of this talk, but since you've been working with Django for so long, I'm just curious. Um Django was written long before typing was available and standardized in Python, and because it relies so heavily on metaclassing. There's this really big gap between sort of the source code both inside of a project and at the framework and like the types that are occurring at runtime. And the MyPi plugin tries to like arbitrarily bind this gap.

Speaker 1: Yeah.

Speaker 3: But it's not really a sustainable solution. And I'm just curious whether you think it's possible for Django to one day become like very well-typed. Um Uh without without a significant rewrite. Um, d is that possible within the context of what's available now or is it is it too a fridge too far?

Speaker 1: So this is interesting, right? Because like like it's not just typing that metaclasses are a problem with. So one of the things I've said to every engineering team I've led for the last decade is Metaclasses are great, never use them right because they're they're incredibly powerful and I think in Django they are very carefully crafted to give you a good developer experience and I think that is very valuable. I do think I I would love to see a way of making the type stable absolutely and that's that 's really important goal. I'm not the weird MyPi plugin still gives me the creeps. It's kind of weird. It works well though. It's credit credit to people who write it. But yeah, like it's hard to disentangle like, yes, I want good typing, but I do care about developer experience more. Right? Like it's a bit like settings, right? We always said like we should remove settings as a global variable. Well of course we were should.

Speaker 1: How do we keep it working as well for like average user? So like this is where the perfectionist with deadlines comes in, right? Like yes, I'd love a perfect type system I did part of my degree in type theory, but I know that I don't want that to actually interfere with the practicality. So I think the thing to look at there is like Can we maybe disentangle the metaclasses a bit with some of the new Python 3 high-level metaclass stuff potentially to make them more easy to understand? Is there something like, you know, like Param spec turned up recently? in typing to let you do decorators. Is there a extension to typing we need as well? Yes, I'd love to see it, but I also don't think I don't want to compromise Django's experience to do it basically. Yeah.

Speaker 4: So given your kind of day job as well as just the ecosystem, I feel like I go to PyCon and now like more than fifty percent of the people are doing data stuff.

Speaker 1: Oh yeah.

Speaker 4: How does Django is is there a a data integration is there like how does that kind of whole revolution change how what Django should be or is there anything?

Speaker 1: That's a that's a really interesting question. I don't think there's directly a a thing that we should become come from it, right? Like I think what what I see in the in the data science community is a set of people who are so happy they have a decent language now to do stuff in that isn't, you know, one of the old ones Like you know, if you used R, you know what I mean. Um But also like a lot of them aren't developers, right? You talk to them like they they they identify scientists first and developers second. And like, you know, like things like source control are not necessarily in their vocabulary. And like part of it, I think, is how do we reach out to them and bring them into the Python community first and foremost as like You don't have to be a developer to be in the community, but we'd love your input as to how to push the language forward. And I think from that then into the web Python community.

Speaker 1: So like us and Flask and Fast API and everybody else and be like, well, can we do something all needs too? Because a lot of these projects do have some kind of display component at the end of them I just don't think I don't want to be self-serving, right? I want to have I want to help them and their problems first and really establish Python as the data science language where like We have a foothold for decades to come and then work across that boundary and be like, okay, like you've got loads of data, it's very exciting. We have a way of displaying it in browsers, which are also very complex, but we've done a lot of the work for you how we bridge that gap and make it much more accessible. Things like dataset, for example, are a good way of thinking about that too. Like it's not a particularly difficult concept. But it's such a sort of amazing like you can just give me a database that I can show it on with filtering online.

Speaker 1: This is incredible. Like it seems small to us, it's big to them, and that's the kind of thing I want to work on of like what are the things we have that we take for granted that I think data science could do.

Speaker 5: Andrew, uh thank you for your talk. Uh small question. In your example you use um uh function by based view. Uh can you can you answer why not generic class based view? Uh I think this is uh the mostly underrated thing in Django.

Speaker 1: There is some knowing laughter from the second row here. Um There were two competing proposals for class-based views in Django, mine and the current one I, as everyone who's worked with me, I'm not a huge fan of mix-ins. The current one is very mix-in-heavy. So like I I still find I do use them in big projects. Here I uh for small views I prefer for functions because functions are shorter. I do think class-based views that currently exist You have to have a whole separate documentation site to work out what the mix-ins do, and I find that a little bit much. Um but yeah, I I'm a simple man I have simple pleasures like simple views. I generally am not like you know the the example I showed you, right, like I'll put it back up here for reference actually.

Speaker 1: So what I'm taking here is what would be a create form view in class-based views, but I wanted to override the logic of the way the rendering flow works and the way the is valid flow works. Now I can just write this from memory in my sleep because I've been doing this for like, you know, 14 years. I don't know what the functions are on ClassX use that overwrite. for this and it would be longer. So this is purely a case of me just being lazy and going, I know how to do this, I'll just do it this way because I've done it forever, basically. But also I'm not a huge fan of class play views. So that's that's just me though. They they are fine. They they they serve a good purpose, they're not my thing. There is some cringing going on over here, it's great

Speaker 2: All right, thank you very much, Andrew. We are done for this session. We will go join back up here in five minutes. Let's give a big round of applause for Andrew. Thank you.