Turning test writing into a consistently brief and pleasant experience

That's right, yeah. Uh no, great to be here. Great to see you all. Um Very much looking forward to giving this talk. Yeah, you might have read a slightly different title, but I think this one rolls off the tongue a little bit easier. How to spend less time writing tests. Um this might sound a little bit heretical to some of you. Uh maybe I will be saying some heretical things during this um presentation We'll see. But uh so to get things started, this is me. Um I'll try and keep this pretty beep pretty brief. Um I've been using Django for about nine years, mostly to build Simplepol, which is a Slack app. Um so Django app uh has grown to be quite large. People people sometimes tell me, did you just take the Django polls tutorial and add it to Slack and now make a bunch of money from it?

That seems unfair. I'm like, yeah, that's exactly how it happened. Um Yeah. Um so simple poll is is has grown to be quite popular. It's been around for a number of years. You have millions of users. There's a um a team of six of us building it And yeah, it's a Django app. At some point we um started hacking on this internal tool called Colo. And we're also sponsoring the conference as Colo. Um woohoo. And um uh it was yeah, it was just a tool for us internally to build simple poll more quickly. And we'll talk a little bit more about Colo kind of at the end. But yeah, Colo is is a you know always on debugging understanding and also now something else for Django. Uh I also spent a few years working at GitHub on their massive Rails monolith.

So and no one has ever taken me up on this conversation topic. I really hope someone does now. I have some opinions about what we can learn from Rails and what Rails can learn from us, but no one So um that that was kind of fun spending two years doing that. But yeah, mostly uh I've kind of grown up with with Python and Django for uh past nine years or so. And yeah, at GitHub I worked on the on the API and on on the first version of GitHub Actions, which is not really around anymore. All right, um quick disclaimer up front. So this is a talk about testing Django applications. It is not a talk about writing software that controls airplanes. or about sending rovers to Mars. Um I think a bunch of you will probably have this little achievement thing on GitHub that says

you contributed to some code base that now is on Mars. I think like Flask and a bunch of um bunch of really popular Python open source packages are are now currently on the latest Mars rover. Um The tests for those types of software might look different for some of the tests we're talking about now. So this is a talk about uh testing Django applications. It's actually also not a talk about testing Python packages either. Uh so by Django applications I mean Our websites, our Django web apps. Um and obviously I will have a massive bias to the simple port code base, which is the main big Django code base I've worked in uh the past several years. But yes, uh long story short, I have a confession to make. I don't particularly enjoy writing tests

Um and I never really have. But I do like the tests that that already exist. I can see the value they have. I'm glad that Someone, probably past me, has written tests in the past. So as I add a new feature and I go to write a new test, I kind of sigh and I go through with it. I add the new test because I know future me will be will be happy about me adding it. Um but I I kind of realized this that I don't enjoy doing this. Uh and I asked myself a few questions like how can I spend less time writing tests? How can my team spend less time writing tests? And I was a little bit curious if um if anyone else felt felt this way. Sometimes I get the sense that uh Everyone but me religiously practices test-driven development

and only writes tests first in this perfect way that uh all software is meant to be built. But I think in reality it's not it's not quite like that. Um So we have three different parts of this talk. The first one is you know setting a little bit of context. Um you know how much time do we actually spend writing tests? Uh do we actually mostly enjoy it or do we not? Uh then what can we do to spend less time writing tests? And then part three, mysterious part three, can we do more than we might currently be doing? Um So for this part one, which is kind of setting the context, you know, how really answering these two questions, um, in preparation for this talk, I went and interviewed about 15 uh professional Django developers.

who um very graciously gave me some of their valuable time to have me ask them a bunch of questions about writing tests. Some of them are here, so if you're here, thank you very much. I really appreciate it. And I was actually gonna do uh a poll with all of you. So get get your phones out in a sec for some live questions. Um but the first the first thing I was curious about, this first question is is how much time do we spend writing tests? Uh And the act there's actually a bunch of academic research on this, and it suggests between 17 to 50 percent. Uh so this is essentially to tell you, hey, You know, we it's not insignificant the amount of time we spend writing tests. Uh it's not something that we just do, you know, five percent of the time. And in my interviews I heard very similar figures to this you know published academic research

Uh the lowest I heard was 10 to 20 percent, as in someone said I spent 10 to 20 percent writing tests. Um but the median was probably fifty percent. Uh so out of all the people, most people said 50% or the the middle. You ha you know how median works. Uh And the highest I heard was probably uh 80. Uh and it depends a little bit, you know. Some say when I'm updating tests it's quicker, some say when I'm updating tests actually it's it's more time. Um but really the The main point I want to make here is that we do spend a lot of time on um on tests. So that's question one covered. The second question is uh do we enjoy writing tests? And obviously I asked everyone I interviewed this question as well. As you know, I have my own feelings.

But putting all the bias aside, I would love for you to Share your thoughts. So if you go to slido. com and put in those five um five digits or I guess uh yeah scan the QR codes, we can get a little live sample of everyone in this room And obviously it's been great talking to some of you about this already at the at the Colo booth. Um there seems to be a bit of a uh divide. Oh, wow, this is so cool. Wow, so much movement around. Okay, seventy-six of you have cast cast there at ninety-one.

Oh, this is awesome. We'll give it another like thirty seconds or so. And obviously don't look at what the person next to you or the colleagues um who who you are here with are voting for. Because um You know, if I was here with a team and the team culture is we all write tests all the time, obviously for your colleagues you love tests. Cool. All right. I think we have a pretty clear picture here. So the the winner is enjoyed a little bit. Uh big a third is, you know, don't love it, but it's part of the job.

And then we have kind of on the polar opposites, you know, some some love it, some hate it. I would actually love to know which one is the winner. out of those bottom two. But maybe we'll never know. Maybe we'll never know. Okay, so these were some of the quotes I heard from uh from the people I interviewed. So You know, do you enjoy writing tests? I like shipping. I like building features for customer uh for customers. Some people seem to think that actually no one enjoys writing tests, uh, which doesn't seem quite true, but still uh clearly some strong opinions Um I think the one on the left in the middle I found kind of most insightful perhaps. That um It depends on the test, right?

So sometimes it's quite fun, but if you have to mock a lot of things, then it gets annoying. And we'll explore that in a little bit more depth. But yeah, you can you can kind of see a bit of a theme. I think yeah, there seems to be a you know, it's part of what we do. We're you know professional software developers. We want to make sure that the code we write actually works. How do we do that? Well, we write tests. And how do we make sure it keeps working? We write tests. Okay. So um Yeah, this is kind of the the the end of this first part. Like what what are the the things um Yeah, what w what is the answer to those two questions? Number one, how much time do we spend on it? You know, not an insignificant amount of time, spend a little bit of time on it.

And then do we enjoy it? Um Yeah, I guess um some of you enjoyed a little bit. Uh there seems to be a portion that says, okay, yeah, it's it's something we have to do for uh for the job. Don't love it. And then you know thirteen percent hate it or or love it. So um now that we've established this that maybe writing tests isn't maybe the most fun part of what we do, and we do spend some time doing uh it, uh we're gonna cover what we can do about it. And um yeah the answer is actually there's a lot of different things and I have only 30 minutes. So there's um Yeah, I I kind of grouped both like based on my own experience, based on the interviews I did with um 15 of you, and um

yeah, kind of some some of the published writing from uh some of the folks who write blog posts about Django, uh and actually outside of the Python community as well. Kind of group these into like into like three different categories. And um Yeah, I think the third one especially will be is kind of the the the most significant one. All right. So the first first um first thing we can do is kind of on the behavior side. So we can ask ourselves, what actually is the the value of the test that that we write? Uh and I actually asked everyone this as well. I interviewed. So, you know, think about worst-case scenarios is is one that is kind of different.

But the one thing I heard uh overwhelmingly from folks I talked to is essentially this point around if I'm making a change, I want to make sure that nothing else breaks. Everything that I've already written still works. Which is essentially how I guess we talk about regression testing. And actually going into like doing some of this research and thinking about it, I thought, you know, some tests are regression tests. Some tests are tests that make sure that you're not breaking things as you make a different change. But in talking to people it seems like once the tests exist, maybe that is their main purpose. Um You know, you can argue maybe tests are a bit of documentation for the rest of your code, or there's some, you know, sharing what someone who wrote this code was thinking, and you can kind of see their

their thought process. But really the main value, it seems, is um Is that we make sure that everything in our application still works. That's the that's the core value of them. So Okay. Uh some this might be familiar to some of you. Um yeah, let me um Yeah, so so I guess like if that is the main point, right? Um at least after the tests exist, what does that mean for the for the tests we want to write? And uh there's this quote about food and eating food. Um so eat food, not too much, mostly plants. And then uh actually the I it sounds like it was maybe originated with the C of Vercel. Uh Who said

who put out this tweet in 2016? Uh write tests, not too many, mostly integration. And this kind of like maps up with my experience that integration tests give you kind of the most bang for your buck in terms of achieving this The thing that works now still works in the f you know as I make any other change, uh and it will break if it if it doesn't work. Uh So I kind of like this like this perspective. I should clarify also what I mean when I say uh integration test. Because I think people have different definitions, but But you know, long story short, this is what I mean by integration test. In testing our Django applications, it means using the Django test

client to make an actual request to our app. And then um you know making some assertions. I mean this is a very simple example. Uh you you would probably have to do some setup before typically maybe you have a user that needs to be authenticated to do this. Um Maybe uh yeah, you need to do some mocking or some some some different things. But just to say that this is clearly not a unit test, right? It doesn't import a specific function and call it with a specific thing. We're actually sending this through Django. To a specific URL. But also uh I don't mean an end-to-end browser test, right? So Chrome isn't getting fired up in this test Uh we're not clicking on anything in a browser. Uh so when I when I say integration test, mostly this is what I mean. Uh we're kind of firing a request and then doing some assertions afterwards.

Uh uh I am yeah, I'm not gonna talk too much more about um yeah, some of the reasoning for why I think this like Write tests, not too many, most of the integration is a good approach to take. But there is a fantastic blog post by Ken C. Dodds from the JavaScript community that actually really takes this apart. So I would encourage you to check this out if this is interesting to you. Or if you ever f Feel challenged by a colleague on this perspective, uh, you can send them both this talk but also uh this blog post. So to come back to this idea of like behavior and challenging some of the workflows that might exist in our teams. I want to use this opportunity to, I guess, yeah, use this talk to tell you that It's okay. You know? It's okay to have few tests that do a lot.

It's okay to never write unit tests. Um yeah, I know heretical stuff. Uh it's also okay to not have a hundred percent coverage. I think you know very few of us will have a hundred percent coverage. for our code base, but like if you make a PR and it has um some changes in it, I think Sometimes there's a bit of an expectation that the diff coverage should be a hundred percent. So any new code that you add should be a hundred percent coverage. I would go as far as saying it's okay not to have that. Um again, not if you are like controlling an airplane or sending software to Mars, but you know, we're building Django apps, right? We want to kind of We want to have the tests work for us. We don't want to serve some arbitrary goal. If there's a bug, well, usually we can like fix it. You know, we have a website. We should be able to deploy that like pretty quickly.

It's also okay to delete tests that get in the way. I think this is like a big faux pas that you can't delete things that already exist. If there's a test that constantly breaking for unrelated changes and it doesn't actually you know it's not a real breakage like it doesn't you didn't break anything it's just flaky or it's just annoying it's okay to like delete the test. And also crucially, and I think um I'm actually not sure how frowned upon this last point is, but I think it's more than fine to copy-paste Freely uh when you're when you're uh adding new tests. You know, I will often copy an existing test entirely and uh just make a few tweaks and then it's working. Maybe eventually you come back and uh deduplicate some of it. But you can go pretty far with just copy-pasting. Um

and yeah, I think like this is just uh useful reminder that tests are here to serve us, not the other way around. I think software development is not easy, right? So I think it's um Sometimes w I think what we do is we we figure out these kinds of like uh mnemonics or goals or best practices, like you should have 100% coverage, and that's what we should all do. Um and I think this can be helpful, but there's there's a there's a limit. So I think a good reminder that you know it's worth sometimes taking a step back and maybe even with your team say, hey, what is our approach to like writing tests? Like Do we care more about shipping more functionality to our customers than having the occasional bug? Like depending on where your company's at, that might actually be the right solution.

Uh some a lot of buggers can be fixed quite easily and if they affect very small number of people, maybe that's actually the right way right way to go. Um but I will take this opportunity as well to say That um unit tests can be a very powerful tool. If you are building very complex logic, I think unit tests can be an amazing way to clarify your own thinking. If you're building a um a Python package or a library, I think they're a great way to see the API that your users will use. I think they 100% have their place. Um but for us in in SimplePoll and I think for a lot of Django applications, what's most important is That the functionality works and the integration tests and and it doesn't break by accident. And integration tests are kind of the best way to go. So that's the quick shout-out

for unit tests in this talk. All right, moving on. So speed is another one. So that so we have behavior speed scaffolding. I think how do you spend less time writing tests? Well if your tests are really fast, then you know you need to s you can spend like less time on them. If your CI is really quick, that means a quicker feedback loop. If something's broken, you can fix it more easily. So this is really not to be um underestimated. I think one of the best ways to do it. I um yeah I don't want to spend too much time talking about this because our very own Adam Johnson sitting in the front row. uh has wrote a fantastic book which you should all go and buy um to speed up your Django tests. For us, the single biggest thing that made a uh an impact is Uh running the tests in parallel. So simple call, we have about a thousand tests in the code base.

Most of them are integration tests. It would take about two to three minutes to run them locally. Uh we did a bunch of different things. I followed a bunch of different steps in the book. Uh but the single biggest improvement was actually running um running the tests in parallel. You know, m maybe this is not a surprise to like some of you who do this for who have always been doing this, but uh I actually could never get it to work on Mac. I think there was some change that fit got fixed in 4. 2 made it much easier to run the tests in parallel uh locally. But yeah, it used to take about two to three minutes to run them locally. Now it takes about 15 seconds. And the the Feedback loop is just much uh much nicer. Uh and I'll use just another slide to say you should be doing this in CI as well, obviously.

Um And the one thing that might be less obvious here is that the default like GitHub Actions CI runner, uh which is what we use, I think it gives you like two cores. So you know you can run in parallel, but You can also get larger runners with eight cores or sixteen cores, and GitHub will charge you a bit more for that, but uh it's worth it for the overall productivity. And bonus tip, there's actually something pretty cool, which is a fairly new tool called BuildJet. Um they It's basically like a drop-in replacement. You need to like install an app and change this one line which they're highlighting on their website. And then You you can get access to these larger runners without needing to do the GitHub beta thing.

Uh and it costs about half as much as GitHub Actions. So we just use this now. The other cool thing they have is um they have ARM runners. Which GitHub doesn't have yet. Uh and it's kind of cool actually because for sample all of our laptops are like M M1, M2, so it's all ARM. We run our code on on AWS on ARM as well. And now CI is also ARM. So it's all Uh all arm which is kind of cool. So I would recommend checking out buildjet. Uh at best you might cut your like GitHub Actions build in half. Maybe it's even faster and uh ARM as well. So that is um That is speed. Okay. Now scaffolding. I mentioned this was gonna be a big one. Um So in these interviews I did, the single biggest thing that came up around like

why might writing tests sometimes be painful, or when is it the most painful? The thing that always came up was um the test setup. So not actually writing the assertions themselves in a test, that I think can actually be quite fun, but getting to a point first where you can actually just send a request using the Django test client and you can get it to not like 500 or to not error. So the the the main point of this section is you want to get to a place where it's as easy as possible uh for yourself to write an integration test. And you can achieve that by investing some time in setting up fixtures, factories, some mocks. And yeah, I mean the more painful it is to get the test data right, the more painful it'll be to write an integration test. And then me you know either you will really dread doing that and spend way longer than you need to, or you'll write a test that tests less and is more brittle, um, or doesn't give you this assurance that

Uh yeah, it'll break if some if if some functionality breaks. Um so let's let's talk about how we can do this. Uh Alright, the the first one is is factory boy and essentially custom yeah, these custom factories I think this will be familiar to some of you, but as an example here, um we have the the factory boy set up on the left. Essentially what this lets you do is set up a bunch of defaults for the all of the models really that you make use of in in your tests. And then on the right, um can I use my cursor here? Yeah, look at that. So on the right, you can then write a test that essentially just adjusts this um record you want to create. With whatever you want to test. So in this case we care about amount 200, status paid, and customer is VIP.

And all of the defaults are taken care of. Whereas if you don't have these factories set up, you need to do a lot oops You need to do a lot more work um to set up all of the the the models in the right way, right? So an an order always needs a customer uh and that's taken care of by Factory Boy for you. So if you don't yet use Factory Boy or something like it, I think this will yeah. This will go really really far in just making it very very straightforward to write your next integration test. Um all right, there is also um a new approach to doing these factory functions or doing yeah, factory setup. uh which is this uh blog post from Luc Plant. Um I will let you go and check that out uh on your own time.

Uh we still use Factory Boy at Simple Paul, but this blog post makes a couple of interesting points around, hey Factory Boy, you can write a version that is better in some ways, maybe yourself with not that much code. So it talks about some of the drawbacks of Factory Boy. Um yeah, I have an example here. Essentially you know it it looks a bit more like this if you uh follow this pattern that's described in that blog post. So you still have very much the same idea that you can create these records with great defaults much more quickly. But it's a yeah, it's a it's a different way of doing it, and you don't have any kind of third-party package, it's just your own code. But yeah, my main point here is really you should have uh some way of doing these uh factories because that just makes it much easier for yourself to um to write uh your next integration test.

The next one I have here is have a custom test case. And again, this is something that came up a lot in the interviews I was doing. A lot of folks had this, and this is one of the um actually I think the question I asked is: have you done anything to make writing tests uh more straightforward for yourself for your own team. The first thing people mentioned was the factories and then having a custom test case was was the other one. So Um we use unit test at simple poll and we inherit not from test case, but we inherit from custom test case. And then on um custom test case we have all of this custom setup logic, assertion logic, which again just makes it easier to write that next integration test. And I have a couple of examples here. So sending a Stripe event, you know, this is again uh you know you if you want to test some kind of Stripe logic, um that, you know, some Stripe

event processing logic that we have. uh is a little bit more involved, you need to generate like a signature and all of this stuff. You really don't want to be doing that every time. You want to add some kind of payments test. Uh so having some abstractions is really nice. And in this way we can just in our test code do uh self. send stripe event and pass in the data and then everything else is taken care of. So that's I guess an example of sending just like a different type of uh request with the test client. And then the the bottom example, a cert analytics event. Uh we like almost every action a user takes in in simple poll, we log some kind of analytics event like poll created or voted or Pre-fill options changed. Um and these analytics events help us understand how people actually use the app, but uh we were like

it was one of those things where it was quite brittle. Whenever we added like a um Whenever we added like a new property to one of these events, then uh so many tests would fail. So making the assertion logic a little bit less strict and a little bit more flexible is what removed a lot of pain and and now we just use this instead of uh using the mocking uh directly. Uh all right, uh a couple more. So mocking HTTP requests. I think it's more and more common that as part of your Django apps you will be doing outbound API calls. Uh that can be a little bit annoying to mock sometimes. We use HD pretty to make this easier. Looks kind of like this. Um we have freeze gun and time

machine. Dealing with time is always a bit of a pain. Uh again, our very own Adam Johnson created Time Machine. Second shout-out. More talk. Wow. I'm gonna rush through these a little bit because I want to show you uh something else. So you've done all of this But is there more we can do? And yes, there is. So let's talk about generating tests with AI and also without AI. So um A couple of the folks I talked to use GitHub Copilot and ChatGPT uh to write tests. Uh and yeah, I I was gonna give you a demo, but I think this is the kind of thing that's best played around with your your yourself. So the strategies you know you can use with GitHub Copilot, you can write one test, maybe it will generate some other ones, then have it fill in kind of the rest.

And also you can just write good test method names and then maybe it can take it from there and and and write the rest of the test for you. Uh So, yeah, and then this is a prompt from I believe uh Darien who gave a great talk at last JungeCon Europe about Nix Um he has been doing a lot of experimenting with ChatGPT uh with uh getting ChatGPT to write Django tests. So he will write a prompt like this. And um yeah, I think sometimes you can get really great results from that. Um I was gonna try and demo this to you, but I don't think there is time for that. So I will just talk about some of the drawbacks with this. I think I would really recommend you everyone use GitHub Copilot. I think it can save a ton of time, not just for generating tests, but for other things as well. Um but yeah, it doesn't come out perfectly. Like expect some trial error.

Uh obviously you need to be okay with sending at least parts of your code to OpenAI Um these LLMs like to hallucinate, so sometimes they tell you that things are there when really they aren't. Um and if you have a lot of code, the context window, which is what these things, you know, what you can give the um large language models like ChatGPT, uh it can be a little bit uh const constraining, but I would still really recommend that you you make use of it. Uh all right. So um who here is already familiar with colo? Can I get a quick show of hands? Okay, that's I think about 40% of you. So come by the booth to get like the full demo. But long story short, you know, we help you understand what your own code is doing. Um Can show you this beautiful visualization and all of the local variables. But yeah, I think what you need to know for the purposes of this

is that in here we show you recent requests that your locally running Django app served essentially. And then we you know as I was doing some of this I realized wait a second so we have these requests that we saved. Can we actually generate a test? From the request that we saved. So I want to um I want to dem demo that for you. Quick warning though, it's very alpha, and I'm gonna mysteriously walk to the other laptop for this. All right. Um cool. So I um I'm connected to the Wi Fi. Lovely. So I have a I have a demo app here. Um it's a to-do app you can add to 's

It works nice. You can obviously refresh and it's still see the to-dos. But also it has a cool feature that I think all to-do apps should have, which is break down a to-do into c small consumable chunks to make it more approachable. So you can click the breakdown button. And it will use chat GB3, chat GPT. And look at that. Practice and refine presentation to ensure smooth delivery. Definitely did not do that. You can also break down takeover the world with kindness. Anyway, I think this is just like something that all um all to do have should have. Uh and obviously it would be nice to order these so that you know what's going on. Create a plan for how to incorporate acts of kindness in daily life. Anyway, this is kind of besides the point.

Um We now have all of these requests recorded in um uh in uh in VS Code here. Uh and it this is a very simple uh Django app. It literally has these three views. Not that much going on. But um if we now want to generate some tests, we can maybe we we try listing the to-dos first. Uh actually I'm gonna use this one because it has a little bit less in it. So I can click generate test. And then we get this created. So automatically we kind of Oh thank you. So let me actually run this just to show you that it it does work, if it does work.

It doesn't work. Oh wait. Oh sorry. This is the test from ChatGPT, which actually doesn't work. Uh let's try it again. Boom, it works. So I should clarify that like you know this is very alpha. We're just getting started with exploring how can we generate tests for you. And really the Focus is not on doing all of the work for you. Like this is not something you just plug in and turn on and you never have to write tests again. The idea is that this is a starting point, right? So if mocking and getting the fixture data into the right place is actually the really hard part of um um uh of creating a test, then that's the part we want to help with. So creating all of the right objects in the in the database so that you actually have something to test. That's kind of what it's about. It's very easy to edit something that

that's already that already exists. It's much harder to kind of start from a blank slate Um I'll show you one other test which is this um break request which actually uh generates This HTTP fixture, right? So in this case we actually make an a a call to the open AI API. So there's more going on here, but hopefully this will still pass. Boom, passes, right? So we have a full like HTT mock setup here. There's no HTTP request that happened within that test. It's all kind of uh mocked out and generated for us. Now you might think, okay, that's a cute demo, but that's a basic silly demo app. Does it work on anything real? And well, good thing we have the simple Paul code base. Um This is a a pretty fat trace here. This is someone voting on a poll, loads of things going on.

Um let's try and generate a test from this. So yes, let's do that. Generate test. So here there's a load of setup, right? And this is not great. Like this should this should be factories, this shouldn't just be dumping and all this raw stuff. Again, very alpha, we'll work on improving this. But If you're starting kind of from very little or if you just want to have something that you can now edit, then yes, you can um you you now you have something, right? Now you can edit it. And I believe this also should pass Oh, that's running all of the simple pull tests. Maybe not the best idea. Let's do it like this.

Drum roll. Hey, and that passed. So there's a yeah, there 's a ton going on in this request, right? There's a really like no shortage of of of things, but this actually did go through the whole um the whole simple pause stack through all of this code and we even have a couple of assertions, right? So the main point is that we want to help you with the fixture setup. But um we actually know from the database, from looking at the trace that this channel um was selected and then it was updated. So since it was updated, we can know that maybe you want to make an assertion about one of these updates. Probably not all of these fields changed, but maybe like one of them did. So you can delete the assertions you don't care about and keep the ones that you do care about. So again, it's a starting point for kind of um helping you write that entire test.

All right, I think I'm out of time. But if this is interesting to you, um please come chat at the booth. We have tons of t-shirts if you're interested in that. Um and obviously would love for you all to give this a go and and see how you get on with it. It's all free also. It's also very alpha, so be kind. All right, thanks every Everyone, uh appreciate it.