Clean up your code with quality tools - Marijke Luttekes

Speaker 1: And uh everything hopefully working. Um let's first of all make sure we give Mareg a big warm welcome. Yes.

Speaker 2: I think do I have sound now? Uh first of all, I have just recovered my voice after illness. I will be coughing a lot. I will try not to upset the microphone. Um so if I'm making a lot of coughing sounds. It's uh it's me, uh sadly. Well fun to be here again. I was here last year as well, but that was Uncle Danny's plots, I cannot pronounce any Danish whatsoever. Um and we have lost um our What's it called again? Our display. I actually can improvise most of this presentation, but I do have some example code in the end which um This might be a problem. We'll figure it out. We'll figure it out.

Speaker 2: I can actually do this by hand, which is cool because I have actually not prepared this presentation. As I wanted to due to illness. Um well these people figured out. I'd like to know who is in the house, and for that I want to have a little example and then kinda we are going to raise hands after a bit So be prepared to engage. Imagine this story. Imagine that you are working on a project with someone else. And uh you that someone else they gave you a pull request or a merge request if you're a GitLab user. And um it's actually quite okay, like the the the unit tests they pass, which is good. Um it it it works.

Speaker 2: But there's some small things that aren't really great like um documentation is missing or it's doing some things that could have been simpler, better, more efficient. Maybe your logging, um well I don't know if anyone used uh knows this, but you are not supposed to use direct string interpol interpolation in logging functions. I didn't know that until a while ago, but maybe you knew. Well maybe they're doing that kind of things. And actually the the string interpolation thing will mess up all your error patterns in like sentry and whatnot. So it's good, but it's not really there yet. So you kind of want them to improve it a bit, but it's their day off tomorrow. We're going to pretend it's a Friday and that we're shipping on Friday.

Speaker 2: There are reasons why you shouldn't, but for the sake of this example, we are shipping This on Friday. And but Friday is the developer's day off. So we have some options. We can mess with the uh pull request ourselves, which Who would do that? Raise your hand if you would do that. Mess with the other person's pull request to fix it. Good, you shouldn't. Oh well, I say that that people are like, yay, it's me. Um I have at times actually done this. I try to avoid it, but if I know that I'll be merging it right after, it's okay, and I will probably tell that person. But then there's two scenarios. I want to know which one of the two you would pick. We have scenario one, which is

Speaker 2: ship it now and fix it later. Who would do that? Yes, I would also I also call it f ship it now regret it later. So the other scenario is I I saw two hands. The other scenario is Regret it now, but don't ship it and wait for a couple of days and upset sales. We would do that. Somewhat more hands. Yeah. Um salespeople hate this one simple trick. Although it's uh for the example it's Friday you probably shouldn't ship it unless you have a good reason. to do it. So um but my example it it's kind of focuses on like very like I said the code works but it's not really that

Speaker 2: there yet. And um I hate those fixes. I hate giving people requests back on such simple side things. But you need it for like like keeping the quality up. And I am going to tell you today how to automate some of those things because we are developers and automation is Making us happy? Yeah, can we get some okay? I see very hard nods. Yeah, automation is good. Uh we are very much mourning people here. I can see it. Um so first something about me. Um Well I still have my voice. This is me. This is uh not my office. It is my backyard. Uh very

Speaker 2: Danish teapot. I I freaking love that thing. I have three of them now. They are amazing. Like please people who produce this pay me to advertise this thing for you because they're freaking expensive. Um this is my backyard. Uh we actually have Wi-Fi in the backyard. really cool. Um well my name is Mereike and a lot of people have no clue how to pronounce that outside of the Germanic languages. So I had always to pr tell them now you sort of pronounce it like Ma Rike and it escalated to this now being my company name I'm a freelancer. I started a few months ago. I actually still need work. So I have calling cards if you need it. Come and pick them up later These are my debugging tools and for the people who cannot see the screen.

Speaker 2: It's uh two what they're called rubber duckies. There's one he's green, he kinda looks like a dragon. Officially it's apparently green A dinosaur, everyone calls it a dragon, and the other one is a mix between a snail, it has like a snail house on its back, and well, it's cool. Uh Ewis, I know you're watching on the live stream, you love these things, so enjoy Um yeah, what's that? And uh we all had that excellent talk by uh Yo Joe a bit ago. I actually know that I'll have to remove some well statements from a couple of tests. Thank you, Joe. Now I have to fix my tests. This is just a bunch of like

Speaker 2: Python Tools that we use. It's not even the Django tools. I I mean Willem is here with Colo. If you haven't checked it out yet, it's awesome. But this is just Python stuff, and we are probably uh going to ignore most of that and we are going to focus on on some quality tools, rough, black, and iSort. And we are going to run it on pre-commit, not because I like pre-commit, I do like it, but there's reasons why I don't like to promote it. But it's the quickest. um thing that we can use right now to to run it. So um we're just going to ignore most of this, but I kinda I wanted to make a graph because graphs are cool and it Looks like I prepared this talk really well. You know, that's the real reason. So uh a bit more about the why.

Speaker 2: I love consistency. Is absolutely my favorite word in programming because it makes code easier to ship and share. And I actually have a slide for that. And also the automated part is, don't make me think, a lot of tools like Black and ice. or you can use them to fix the code for you, which means that you can just write the weirdest uh we get demonetized if I Yeah, the weirdest code ever. Um, I have to really try not to get us demonetized because I love cussing. Um Fingers crossed. Plus, my my beloved, my students, some of my students, I work in the school one day a week, they are watching, and I have to set

Speaker 2: an example which is really hard by the way. Um but yeah automating it's kind of don't make me think like you can write weird things that are not really up to code and then the tool is like, yeah, we're not doing that. And then either it fixes it for you or it gives you this hint of how to fix it. um rough is more in the latter zone. We love automation, which is why there's a ta-da here. But for me the very best thing is I could actually focus on just the functional part of merge requests. Like when I got a um a merge request from a junior for example, like I love training juniors for those who've seen my Last year's talk, I knew that all the basic things like documentation and just some simple things

Speaker 2: they had been taken care of. So I just knew that I had to look at how they did whatever They did. And the fun thing is actually, spoiler alert, when I put this these tools that I'm showing you in my project with one of my juniors, he was mad at me for two weeks. Especially since it was quite a young project but we had to fix like three hundred bits of code documentation. I can tell you that that sucks. I love documentation. That's why you usually write it at the start when you write code, but we had to fix like over 300 strings anyway. God, he was mad. But after two weeks he was like Hey Mike, how can I put this in my project? I was like, yeah. Two weeks it took me. And actually I sold it to like my previous employer. We actually had a two-day

Speaker 2: where we put all the stuff that I'm showing you, or in slightly older. configuration into their existing projects and that in the end I think they even started to like it which is you know one of my prouder moments. I have no clue what my next slide is because I don't have presenter mode on here now. Ah, preparation. That's actually important. I'm going to focus a bit more on how to get this whole thing done Because everyone can read the documentation of all these tools and they're pretty excellent, so I'm just not going to explain that in very much depth. I do have some example code. I made the repository public a while ago. If anyone needs the link, I will show you or just find my name on GitLab.

Speaker 2: Um the actual username is a bit will also get us demonetized when pronounced in English uh on GitLab, so just ask me later. Um well it's an M and an H and an L and U and a T, don't uh pronounce it uh on stream. So I'm just going to do some findings on how to get this done. The entire stack that I'm going to show you, which you don't actually have To do all at once. I've had results within four hours, I've had them in two days with bigger projects. This is a bit of a guesstimate, as we all know, estimations are They're guesses. That's why guesstimate, not estimate. And if you do, if you add black, you're going to touch the entire code base.

Speaker 2: Which is also exactly why most projects don't want to do this. But if you do it, and I can highly recommend it because it's like one or two days of drama and after that years of bliss. It's worth it in my opinion, but um you know also for the reasons I previously mentioned. The quickest way is to get rid of everything that is not strictly necessary in terms of branches and merge requests in this project. So what I did is I would just ask the the sprint planning to kind of reduce the number of tickets on the project. Of course this is a bit harder when it's your core project. Okay, there's my voice going But just try.

Speaker 2: Just try to remove as many stale branches as possible, merge as many things as possible. Actually, this whole process might start two one or two sprints in advance but just work towards it. It's not strictly necessary, but it saves you a lot of rebasing and misery if you close everything first. and everything that's not strictly critical for that sprint just either plan at the end of the sprint like after the whole ref factor or just you know try to avoid it. Again, it's not necessary, but you will thank me later if you clear the field first. I missed my clicker, but last year I started using it as a feature toy, so I'm just walking around now

Speaker 2: The tools actually there are some things about that I don't like about this tool, especially the line length settings. I've had to I think define line length in like four places in the same file, which if anyone has a solution for that later, I'll happily take it. But the most important thing with the tools is if you run them, make sure that everything runs on the same Python major version. So use it in in the same uh VN for TOX is great. I love TOX like uh that that's like its own presentation in itself. So I'm purposely not using it in this presentation, but TOX is awesome. Like wow. And then you plunk it into your CI and it's even more awesome. But anyway, um just make sure that each tool

Speaker 2: You used on the same directories, used in the same Python version, you know, like you would always do. And they all need to use the same line length. Like I said, I had to define it. I think like three or four places. There must be an easier way. I haven't found it yet. And there's two types of line length. There is the uh uh code line length and the documentation line length. The only tool that actually has this support for the doc length is um Rough or like one Py doc style um branch of rough. This number, like 79, is the standard library rules for lime length. Uh PEP 8 is like they kind of want to want you to stick around 79.

Speaker 2: Black by default takes 88. Um and PEP 8 again the are like code standard. It's like well you can stretch it to 80 if or to to a hundred if you want to. I used to use 120 with a Jacksley. the PHP standard, but I recently switched back to 100 and it's actually pretty nice. So or 99 actually because my ruler is at 100 and I like to stay one character to the left of it. You know. So actually I have my rulers on on 73 and and uh 100 characters and then go to a hundred and seventy two. Well, you'll see it in the documentation uh in the code later. And for the love of God, just run the standard version

Speaker 2: first, or like maybe black just with the line length override because if you are starting to configure the whole thing without actually running the vanilla application first you're going to get overwhelmed. My voice is still working, but like just run, just first use black on source and or like with only the the the line length and Just keep it simple. And for black and iSort, it's doable. Rough if you start adding things to rough right in the beginning, you're going to end up with. I think my record was like two two thousand two hundred issues that Rough wanted me to fix and that was after I disabled the documentation rules Yeah, we're not doing that. Um actually

Speaker 2: rough in legacy projects, I would actually disable it most of the time or just try and get something done. You get varying results depending on how how old a project is, but it's very interesting to just try it anyway So how are we doing this? Like I said, we run the basic configuration first. And how I do it, I just sort of run it first, and then usually I just get run reset the whole thing and just tweak with some settings. And once I'm happy I commit the configuration separately. For just the simple reason that cherry picking is really easy. Like if you have a couple feature branches that really could not have been fixed earlier, you can cherry pick those config

Speaker 2: configurations and run them on the new code. It's a bit more of a hassle, but it works. It's sort of a workaround too like getting multiple branches up to date. Like if you can avoid it, again, avoid it. But you know, cherry picking, yay. After that, I immediately commit the code itself. And this is where other people can start rebasing their code onto yours. Usually it goes without a hassle if their like change logs are really small. But yeah. It also depends on how good people are with rebases and merches. Some people really make a bit of a mess of it. It kind of helps if you ask the person who write the who made like the the the configuration is sort of

Speaker 2: sitting next to the other person who's trying to make your changes into their branch. And after that you can sort of expand the configuration if needed. And then of course commit that as well. For black, that's not really necessary. I sort you can tweak it. Actually, i sort might even not be necessary if you just like the vanilla settings because you can run because black also does a little bit of of import sorting. But I like I like extra settings. And once it's done, that's when you start copy-pasting it to other projects. Because once you have a good configuration, don't let it go. Some people will actually put this in a cookie cutter.

Speaker 2: Cookie cutters are pretty awesome. I have noticed though that every company That starts out with a fancy cookie cutter eventually abandons it. Yeah, we all I I hear some giggles in the in the front. Um But if you use a cookie cutter and you actually keep it up to date, just put the config in there and then people can take it from there. Um this is a process, like I said. I have done it in four hours, I've done it in two days. I can do it in four hours, but people who do it for the first time need a bit more time. But it's absolutely worth it in the coming years to like always have this uniform styling. and just not really having to bother anymore. Ah, well. The good thing is, I actually wanted to do like these fancy slides

Speaker 2: in between the live demo we are just going to show you some code. Oh one disclaimer by the way I'm talking about rough here which is not flake eight but it is a sort of reinvention of Flake Eight in Rust, which means it's smaller and which means it has more maintainers, which I think is one of the best things. things like both things are really good. And I like I said, I made some example code, which Is this? I'm actually maybe not even going to run it because I made a before and after. I don't know which one of you have ever used black, but black has a couple really cool

Speaker 2: cool things like they will enforce single quotes which I have no clue where where they are. I thought they had them at the top. I actually made this and like I said this is now public on my GitLab thing and every change in here I have added a comment to see what it does. But just to be really quick about it. I by the way also have a makefile to run this. By the way, I have no clue how make files work. I had a a a colleague who said make file is like syntax is like an assault to humanity. I agree. But if anyone can ever make a presentation that makes me understand make files, please. Please make it.

Speaker 2: But anyway, it has a run example one, run example two, even has a little README. So you can run these yourself. But like if you have this very very long list like black, I think I have this is first line is my documentation line, second line is the AT line line and I think I have it set to this particular line. I also have a lot of like extensions for black and rough in here. So actually rough here is already complaining. Oh yeah, line too long, whatever Rough doesn't actually need to have an opinion on this because Black will fix this for me. And the only thing you have to do is I'm here. I'm in my There's a source directory here. I just have to run from my VN

Speaker 2: black and then the source directory where I put it. So this one an example No wait this one and it goes like yay one file reformatted and it goes well you can see in the in the sidebar that it's changed a lot of things Like without effort. And of course there are some other issues, but that's rough. This is awesome. Like single effort. changes and I also have one for i sort. By the way everything here lives in pyproject. conf dot config. I sort by default just sort stuff in a particular way. I have made some changes here too incorporated with jang

Speaker 2: with black read this like seriously read this this is too much effort for me to explain right now but you can it's really cool if you tell it like oh yeah I have a source directory then it will make sure that Every item in that source directory is set to be a first package, first party package. And I shall run iSort from here because That is also an option. Sort inputs. What it does is I have told it actually to I have told it in the configuration to treat Django separately. So it What you're looking at here is normally it has like sections and I've added Django and Django Extra because I kind of like having these

Speaker 2: as separate sections. Which is what you're seeing here. Django is separate. This is separate. I have a Hello World in as a first package that it put at the bottom. It's pretty cool. You don't need this black does some of this for you, but if you want to have extra sections and shit. Sorry, demonetization. Then um look at it again look at the example and then one more I have some rough thing Ruff I has a lot of things like literally if you look at the rules that Ruff had it's it's these are all packages. It's so much You can enable them all like this is I've actually scrolled through every one of this. There is so much to do here.

Speaker 2: It's it's ridiculous. Look for this. Um fun fact is in the Pipe project I've actually added links to all this so you can see it. I have enabled a couple and I did some like ex conclusions because Django does some things that Rough does not like. Again, just look for it later. But I will show you a couple really quick and then we're done. Yes, rough. Rough, for example, I told you about the F string thing in formatting. And logging functions. So now it's complaining to me, oh yeah, you're not allowed to use this this

Speaker 2: string interpolation thing in here. It actually wants me to do, I think. world and then this has to be I think it's like this. I think it will stop complaining if I do this but oh it's like oh yeah Yeah, you have an F string without placeholders. Ha! Now it's happy. And of course this doesn't exist. Well, this is really fun. Um this is actually really handy if you have this this rough checker in your IDE. Well, this one is complaining about a lack of documentation. Simplify this loop. This one is actually really fun. It tells you like this thing does return true in the loop, and it just tells you, oh, you can use any. And for this one one this is really cool it has a fix so if I do rough blah blah blah with dash dash fix as a flag

Speaker 2: one fixed Yay, it did it for me. Uh of course never use magic numbers. It's actually also complaining about Raf is complaining about everything. It's amazing. So uh you don't have to think anywhere. You know how cool this is when you're training juniors And you just this just saves like half my time reviewing people. Well, um of course this it only I think it only does like E and W warnings by default, but I've enabled so many and even more like you can see I enabled this is the entire list. I actually recommend just adding a comment to remember what everything is because otherwise you're You will have no clue what everything is. Again, documentation after. If you can't find the link, I'll send it to you or I'll put it on

Speaker 2: FosterDon Um I think it's actually if you just go to Of course I cannot s it's this one. If you just find my name on GitLab, um I don't have a lot of things public. So you it's called Quality Tools Demo, you you'll find it. A quick Google away. Well, um, I hope my voice did not upset upset you too much because it does upset me. Um so sorry about that. I've been eating paracetamol and coffee drops for the last two weeks but I made it I'm very happy and I hope you learned something today.

Speaker 2: I hope we still have room for some questions. I don't know In my defense I started a bit later. God, I need a lot of water right now.

Speaker 1: I think we can take those questions.

Speaker 3: Hello, thank you for your presentation.

Speaker 2: Yes.

Speaker 3: So for me this morning uh rough is a discovery. So if you're If I follow you correctly, uh if you if we move for Ruf, we can bypass Flake. So we can install Blake plus Ruf instead of Blake plus

Speaker 2: Well I used to use a lot of flake eight. Rough is just newer, which also means it's a bit more experimental, so some people like it already, some don't. It's just faster than Flake 8. Um plus it has, I think, a bit more maintainers. It's actually interchangeable. There's also a few things in the Flake 8 configuration that doesn't yet network in Rough. They made a little tool where you can just put in your like Flick8 configuration and rolls out like the rough version and I'm lost like two or three settings I can't remember which Just see if it works for you. Like if it doesn't work for you yet, try again in a few months and just use flake eight. But this is Rust, it's faster, and it's just one package except like for Flake

Speaker 2: 8 for every check you have to add another Python package so you end up with with a lot of dependencies and I kind of just like that rough is just pip install rough and it works you know So but they are kind of interchangeable right now, but Rough is getting better than Flake 8. It's just, it's continuously improving at the moment.

Speaker 1: There was I saw another question. Okay, okay.

Speaker 2: Oh, another one.

Speaker 3: It's uh going to be a follow-up uh question. Uh in my daily life what I do is I apply formatting and uh lint linter uh right uh when I saved each time. So I never uh process as you do, uh have all the code processed and then apply them through command line, etc. or uh uh with precommit what would be your recommendation?

Speaker 2: I kinda I don't like formatting on save because I have a thing where I press command save like every few seconds which means that my ID at some point it's like yeah can you please stop formatting now um but just enable it whenever and your usually your tools they just take the configuration files like This is a pie project. tommel that I used. Your IDE tools will also read that configuration. So like like I said, I have it in in active mode in pre-commit. When I commit it just Sort of I just realized I completely skipped pre-commit, which is awesome. Um but I have an active mode in pre-commit that sort of disables my commits if they're not adhering to the standards. I have a check mode that's just like a truck True false check in TOX, which also runs in

Speaker 2: my CI. I have the IDE that does all the little like the real lines that tells me that things are wrong. And you'll use the same configuration, just like whatever works for you. But I like the pre-commit one because it actively fixes stuff. And for all the people who disable pre-commit because you can do that. I just have the CI running as well that's sort of for all the smart asses who think who they can offer avoid my checks. I have it in CI as well and just sort of disable it just tells you, oh yeah no this this merge request is faulty now. Um but actually it never the CI should never fail if you use pre-commit. and the local tools. But yeah. Use whatever you like.

Speaker 4: I just realized I did have did have another question. Um near the beginning you mentioned make sure you're running the same major version of Python with everything across I mean we've got multiple projects. They're running on different versions of Python. How do you deal with that?

Speaker 2: Yeah, I uh in in the same project. I should have clarified that. You always use the same pi the same like version of the tools as you use in that project so it could be different VMs per project but more like some rules are Python specific so it kind of helps to have have like if your entire project is Python 3. 10 just use free 10 for the tools as well or just pick the latest version if you support multiple Python versions because some versions of Python have a different needs than others so and and sometimes the rules can be different. I have no example for you yet but like I said just use the same Python version for that project and for the tools and all that

Speaker 1: Big big hand for Mareike.