Lightning Talks Day 3

Speaker 1: Okay, so hey guys. Um just have to

Speaker 2: Wait.

Speaker 1: Okay. Okay, so introduction speech, right? So just just uh just wh while I was coming uh here to here to the stage, uh I thought that you know Coming here after the closing ceremony speech is like coming to a bar after it's working hours just to get some beer. But uh but hopefully but hopefully it will went fine. So yeah, uh good day guys. Sally I have only five minutes, but I will try to make it as quick as possible and uh as detailed as possible So database query counts uh checks and tests. Um definitely it's not some kind of uh new, I don't know, reinvent or something like that, but um let's try just just hear it out. Firstly, very shortly about me. I'm Justin. I'm a software developer and team lead of 15 great uh Django and Python developers back in Lithuanian corn case technologies.

Speaker 1: Uh we're development agency. Um I'm Python and EWS enthusiast and uh yeah, I'm first time standing here in the DjangoCon. Even if it's for five minutes, probably it still counts, right? Great. Uh motivation. Uh we all know how to optimize queries, right? There were tons of really great uh topics about testing uh Optimization and so on and so forth for select related, perfection related course and other cool things. But the problem is sometimes it's very hard to spot where to optimize it, right? Um and uh how I spotted that uh when doing audits, when joining, when augmenting uh other teams, joining other projects on the Python and Jingo, uh just like

Speaker 1: very simple example Um right here we've got like you know in serializers we're just trying to get some foreign key relationship, but we forget to perfetch it, to select related interviews. And immediately we get additional queries to the database and uh well you know what happens when we got uh some list uh endpoint with 20 or more um items. So what are the possible solutions? First possible solution you can let your application be tested by the clients, by the users. Um yeah, react fast, try to optimize everything That's totally a solution, right? This this can be counted. We can write tons of great integration load uh end to end tests, uh but They are pretty hot to maintain to write and eventually you spot that

Speaker 1: you got the entire team taking care of that thing. Or third thing, just maybe Maybe try it in unitests. Try to catch at least a little part of those of the um performance issues in unitests. How to do that Um firstly you can already try checking that in your unit test without any any um um hard thing. Um For example, in case you've got PyTest, it's uh Django Assertnum queries fixture. In case it's unitest library, you've got uh self dot num queries But it's not very convenient because you need to still use context managers. Um what is the solution which we're using in our company? It really paid us uh I would say well, is make a wrapper around the API client to assert the B query accounts implicitly.

Speaker 1: So this means without doing any any additional thing so that your query accounts would be checked uh under the hood. Um there will be some code so I'll just try to make very very quickly through through them. So uh what we did um Uh yeah, basically we just wrapped API client. Uh this can be REST frameworks, this can be Django 's. Um it's actually not not a big difference. Uh what we said we just um allowed to pass default query limit And uh later on we just overrides all the possible um and uh um all the possible calls, all the possible methods post, get, and so on and so forth But basically we just wrap them in context manager so that we would do this implicit uh checking of queries. And uh the the helper the helper

Speaker 1: the helper just returns um that context manager 's uh context And in the next slide, uh what that assert maximum queries context is. Basically, there isn't much magic. Django already helps us to do that within the utilities because it provides us capture queries context Uh what it does underhood under the hood basically just captures all the queries within your tests, right? All the queries to the database. So here if we don't need to write anything uh anything for that. Uh then we pass maximum query limit, which is uh which is sufficient. And uh what we do, we just go through uh captured queries. Um there is uh some com very small conditional just to um uh get rid of of uh transactional begin and commit lines.

Speaker 1: And uh what we do we just Get the count of them, right? Very simply uh the the length uh function. And we check whether it's less or equal uh to the maximum query limit which was passed. Right? And lastly, if not we just um raise an error. So basically in PyTest we just raise raise value error. In uh in unit test library we can just uh do self dot fail Uh so how do tests look now? We can provide query limits when making a call. Uh we can also provide the default value so that you know you can just override your fixture, for example, for the client And all your tests already will be doing this check implicitly. So without you know any any more uh test uh updates. And uh

Speaker 1: in case you really want to change that for your existing test, you can just pass query limit one. So where where is that coming from?

Speaker 2: Yeah yeah.

Speaker 1: Yeah so this

Speaker 2: god.

Speaker 1: Yeah, so uh they are some to do, so in case you're interested, just give me a ping. We might release the library at some point So thank you for listening. Here is my LinkedIn. Let's connect. Thank you for listening.

Speaker 2: Thank you. Thank you so and

Speaker 3: me again. I'm uh Seb. I won't be presenting myself, it's too much time, and I have a boat to catch So uh my lightning talks about sleepers. You may have heard that on uh Wednesday by uh Thibaut Colas, and I'll uh justify wha why I uh choose this title. So I'm here to uh talk about Django templates templates, sorry. And um you can do basic things with Django templates. Like uh it's very simple you register oh sorry Doesn't work? Yeah. Better? Yeah. Okay.

Speaker 2: I pause it.

Speaker 3: So inclusion tag. Basically you f you uh pass a template name and uh some variables and yeah you just you can uh use it uh as a jump link. For example. But what about if you try to put some HTML in that on the on the bottom? There's no simple way to inject HTML in uh in answer variable in uh this example. So What if you could do that? Like let's say I have a an FAQ tag which I gave which I give uh some parameters and then I have a body that uh can uh can be used in the template that uh that I include.

Speaker 3: So To do that you can use sleepers, which is a great library. So basically you declare a template. And then you declare it in the YAML file. Well, okay. And then you um you put it in your template uh why uh by loading slippers and use it But I don't like uh YAML, so I uh I have thought about uh doing my own and let's hack inside the jungle template uh code So let's let's try to do a custom template node. You know the the decorator uh register inclusion tag re relies on inclusion node And it just can't handle uh parameters, so we just need to handle parameters and uh a body.

Speaker 3: So we are going to declare our own. Uh let's say we have an inclusion body with body note. And we'll take a node list, which is a body of our template tag, and a context object name, which will be answer in our in our case. So yeah, node list and context object name. Now we need another registration decorator. So let's call it register inclusion with body. And it has the classic parameters to inclusion tag. So that may be complex for you, but basically I take the normal parameters from a register inclusion tag And then I just add context subject name. Now

Speaker 3: the goal is to uh use that decorator like that. So a register uh a function and then I name in I name it FAQ, I give it a question parameter, an active keyword parameter, I do some things inside of that. like putting an FAQ counter which will be useful. And uh yeah so let's see how do we get from how where can we go from from that So we will need to know which parameters are given to the FAQ decorated question. For this we use getFull Arg spec. Uh this takes the name of the decorated function and uh

Speaker 3: oh yeah this takes the name of the decorated function in case in case we didn't give it a name We have a parsing compilation business, I'll get back on that later. And this function is finally registered in the template library. So again, the goal is to do that So we have the parameters, we have the body, the node list, and we parse it until the the end tag Now about the parsing part. You take basically all the bits without the FAQ uh tag name And then uh this code analyzes the parameters after the FAQ tag name and maps them to the one we have in the decorated

Speaker 3: function. So Yeah. That passes uh the node list until the end tag and uh it excludes the the first token Which is the starting tag and finally we give all of that to our new node and yeah believe it or not that's it we you just declared a new kind of jungle template node You declare a new decorateur and instant shade. Another use case you can do is uh some kind of panel with uh a middle tag, so you can have two bodies and yeah

Speaker 2: Thank you.

Speaker 4: It's the first time on any conference stage for me, so I'm a little bit nervous. Um I work at Ambient, which is luckily here on the sponsor list as well, which is a really cool thing. And um I had the other day a problem with which I thought couldn't be so hard to to tackle and was picking a random or at least more or less random record from a table. And um there's this uh wonderful order by question mark thing. Here in documentation, which works, has this annotation that it might be expensive and slow, which is true, at least in our setup, with a couple of thousand records. I got a friendly email from my ops guy telling me that What am I doing there? So I had to find another solution.

Speaker 4: And um then I Googled it because I thought I can't be the first person, it can't be that hard, right? And there's like uh Lots of suggestions which are all not so uh which might not be very performant depending on the distribution of your data. So I was like thinking okay there must be a really easy solution and in the end I came up with just adding a new column to my table and adding something semi-random like UUID or hashing some values together And just adding a new column and pre-filling this in with the great Django pre-save signal. And um Yeah, just ordering then by this new column and this is I mean it's not really random, right? But it feels random and it was completely sufficient for my case

Speaker 4: And um unless I don't know you have like huge data with millions of records or something and then you will create lots of extra data. I think this is a very easy solution um to you know get something very efficiently back which seems to be random. You only have to take care if you use the pre-safe as always if you do the creation not via the um the Django RM meaning you do a bulk insert or something, then it won't work, you have to do a find another way, but in my case it worked and uh yeah it was really easy and solved the problem.

Speaker 5: Good afternoon everybody. Um my name is Afonso. I live thirty minutes from here, so it was kind of a missed opportunity if I didn't talk today. Um and I work remotely for a British company called Colabra uh and that's my GitHub. So um make files are pretty powerful, but they have a kind of arcane and weird syntax and lots of uh strange rules in my opinion. Um and so there was one day when I had like a a stupid crazy idea of okay I'm going to read the GNU Make manual Um I don't know why I did that to myself, but uh I so do not try this at home. It's these slides are really hacky uh and ugly stuff that we can do in makefiles. Uh but I find it interesting that it was able to do uh

Speaker 5: so so much different things. So Uh what I was not understanding at first when learning about make was that it's actually just building a dependency graph between requirements and prerequisites. So I I thought uh instead of trying to use make for for C, I'm going to try to apply it to a Python project and see what happens. Um so basically uh I can define the ru the the targets for for files that I need make to build. So for example in this in this example it is a really simple example uh Here the the Python target uh will create a Python virtual environment, um and then the the install target uh will just read the requirements uh file and and uh and install it and i it has it it uses one of the one

Speaker 5: uh one of make features which is it only runs the the targets if the pre-require requisites files have been updated. So if I try to to m to do an install right after it has been run, it will not run because it knows that the dependencies have not been updated, which is Kind of nice. And the basic feature of make probably uh everybody knows. Then I started hacking stuff and things got a bit weird. Uh so what what's happening here in this example is that uh okay so I can have placeholders to target. So basically with this rule what I'm saying here is that For every target uh that uh that is the whose whose file name is the virtual environment slash bin directory, I'm going to install that package.

Speaker 5: And this allows us to do some crazy stuff like automatically install uh dependencies just based on the name for the for that target. For example, for LinkedIn formatting we we probably uh use black i sort uh DJ HTML which was shown up yesterday I think uh and with this rule we can automatically uh install all of that stuff. I don't know why this is useful, it's just curiosity. Don't try, don't really use this as it is not quite clear. Also Mike is quite nice to to automate some small steps or comments that we usually do during our development lifetime. Usually we would use talks or nox or py invoke for this kind of stuff, but make is native, so uh it's nice to be able to

Speaker 5: quickly program this these different lines. Um another cool thing uh also a bit weird that I discovered is that you can define default rules uh that make will try to match if it doesn't match any of the other rules. So for example um with this with this example that I'm showing here, basically I'm alliing all of the Django manage. py commands directly to make. So it's one less word for me to type when I'm working in terminal. Um and this one this example here is also really weird. I discovered that you can actually embed uh scripting directly in a make in a makefile. So make files allows you to select your own shell So uh

Speaker 5: in this case I'm I I'm I'm saying that the shell is the Python virtual environment uh executable. Um and in this in this example uh I I thought it would be useful to have a command to automatically generate a secret key for Django using the uh the Django function for that. Uh so I discovered that it was actually possible to do that with make. So I did not need to create a separate script, Python script So I could just I like use three two three lines of Python directly in the make file. Um and this feature is also useful uh nowadays that we are starting to work more and more with Docker workflows So you can actually uh specify the shell to be a Docker command uh to run a container uh

Speaker 5: to run a container and then you can add any code you want to execute inside that container directly in make. A bit crazy but it it works and it's something that's actually it is possible. Um so yeah basically that's it. So um if you want to if you have nothing to do you can read the GNU make manual I was just curious to try it out in the Django setup. Thank you very much.

Speaker 6: Thank you everyone for staying until the very end. It's very, very exciting to be here. I mean like please give a big. applause to yourselves for being here. It's so good to have you here because all of you inspire me a lot. on my journey journey uh not only in programming but like just spreading the joy of programming. That's why I also like Jungle Girls a lot and I'll talk a bit about it today. So as you know, Jung Girls is a free programming workshop for women. It's started by Ola

Speaker 6: and Ola. It is all over the world, everywhere. This is just uh some of the upcoming events. We are at the top, middle, porto, and This is really difficult. This is us, right? No. Yeah. Okay. So tomorrow we will be with you next in next room with the sprint. So uh If you are a woman and if you're joining the Sprints, please do come to the next door and meet with our participants. We are going to be running the workshop uh from 10 to 5. just next door to the sprint. So

Speaker 6: uh actually now I'm done. This next is just uh we will see some photos and just celebrate because I think Jungle Girls is also a lot about celebrating. There is a like huge contribution from uh coaches, mentors Yeah, that's me, that's me. And that's my son, uh, also Um so it's not just a um event but uh there is a like big tutorial behind it. It's translated into many languages and like I know it's a good

Speaker 6: source of documentation and like a a good offline uh tutorial. Uh a lot of people, even though they're not women or joining Django Girls, follows and learns Django. So this is a very very old map of uh whereabouts. It's just like you can also start your event. I think like they did a great job. We did a great job. Like there's everything you need to run a workshop and I think like uh this is actually a good read for anyone. to be more aware about uh inclusion. Like I go back to this tutorial just to remember

Speaker 6: and remind myself like what's the best way to um like communicate with the people in my team so that I can be more inclusive. It's like it's a great, great, great document, especially I think the um organizer's manual has a big big section about like how to be more inclusive and all and um yeah something uh a professor in my university prepared for one of our first workshops. And the rest is just like we are going to look at some photos from a very, very old young girls and uh in the last one minute I'm going to talk you into it. So this Those are uh photos from uh nine jungle girls workshops happened in Istanbul

Speaker 6: between 2016 and 2019 uh and um it's very fun. It's Guido who brought Django girls to Turkey. Uh that professor who pre prep uh prepared that beautiful uh jungle girls thing. Uh she used to be uh neighbors with Guido in Netherlands And he was visiting Istanbul, somewhere after the first Europa Python Jenga Girls uh introduced. And he was like, when are you going to run the first workshop? I think uh it was 2016 sum or something like that like that December the first young girls workshop happened in Istanbul uh I think uh it was a big game changer for a lot of people

Speaker 6: And yes. And uh just want to finish with a last note. Like uh It is impactful to help others. It's impactful to inspire. And on the way, you'll get inspired and empowered and see you next year.

Speaker 7: Uh I'm sorry again it's my f first time on the stage. Um Uh I'm coming here from Finland from my company Kprou. We are both uh very old into into Django. Uh thanks uh everybody uh for uh grey dogs since zero ninety six and um use it uh with a lot of code you see we have l lots of code uh written already. Uh Kipro is uh uh one of uh first uh to uh employ Django like on a b on a big scale in

Speaker 7: in SAS and we still uh do it But let's go go into the problem. You know that we uh have lots of APIs everywhere, and uh APIs are sometimes used for integrations. And uh most APIs are rest and they are uh create retrieve update delete list and um So at at at some point we thought uh that uh I I I like I thought that. It's okay to use it and um

Speaker 7: what uh come from it Uh what do you actually need? Um first of all, uh you need some hacks and it's uh making fake model and fake query set Uh module can be data just a data class or something like it. And uh query set can be just a list uh with some custom functions for la la like uh f filter or something for uh uh if you need it. And uh then you need to uh make some uh subclasses from paginator and change list um to um

Speaker 7: Actually use your API for API client functions. And then you do need to do lots of overrides in mod admin Um as you see lots of cats um particularly those functions need to be uh overridden but then you get Something like this, you can probably like see an admin and suddenly you're using rest in admin. And uh you can of course use custom actions like uh but uh even

Speaker 7: using normal actions You can retrieve data. In this particular app uh we don't do a update uh or create, but at least lead works and uh you see it's totally the same experience. So that's all. Um that's how we we solved it and um just another so solution. Uh if you have questions Please ask.

Speaker 8: Hello. Hello. I'd like to talk to you about the most productive Django developers in the whole world. Maybe. And we're gonna use the incredibly flawed and biased metric of the lines of code these developers are affecting per day. Here they are. These are the most uh productive Django developers. And yes, some on the list, but uh Bruno Aller there, the number two, laid most of the ground work. Um So who are we? Why are we here? And are we the most productive? Or are we building something to help you become the most productive? Yes, we're building something called Django Upgrade. Django Upgrade is a tool that

Speaker 8: automatically upgrades your Django project codes. Go from one version of Django to the next, avoid some of the grunt work. So you can install it with pre-commit, so then you can have it just running on every time you commit any Django code. If it's old stuff you copied, it will get upgraded. Or you can just install it the classic way with pip and then you can just run it on files whenever you want. You pass it the target version of Django for your code base. So 4. 1 is the latest. And what it will do is it'll run all the fixes it knows about from Django 1. 7 now and up to rewrite your code into the latest versions and not only take care of deprecations, but also And take advantage of some new features.

Speaker 8: So let's take a look at some. Who remembers Django 1. 9? And going around your code base and adding on delete to every single foreign key and one-to-one field. It does it. There you go. There's a default. Of course it's picked Cascade, which I think was the original default, but if you needed to change it, you can It rewrites your URLs into the lovely new syntax from Django 2. 0. And if you just did a find and replace when you upgraded a 2. 0 There's a new feature pending that it finds the repathic calls and if the regex you're using matches the ones Django uses, it also rewrites it

Speaker 8: Um Django two point two introduce the HTTP request. headers interface It's much better than writing request. meta. getHDP server and remembering all what that actually means. So it rewrites it. And uh pending release, the new Django, uh the new admin decorators, admin. action and admin. display, it finds all those quirky attributes you're attaching to functions that sometimes you know people insert code in the middle and they're halfway down the file and just finds them and puts them into the decorator form. A lot easier to read I've got a lovely review here on Twitter from Justin Mayer. He said thank you for the tool. Yeah, I can't take full quote uh Full responsibility for this.

Speaker 8: Loads of people have contributed at this point. Yeah, so please go check out Django Upgrades. If you can test the unreleased main version, I've got a lot of stuff pending. Some someone called Thibaut Dacon has uh contributed a lot of the Django admin stuff recently. And please join me in Sprints. We've got ideas to uh improve it to even rewrite your templates for deprecations as well. So thank you.

Speaker 2: Thank you.