Keynote - Testing in Django by Ana Balica

This video features Ana Balica at DjangoCon US 2017 in Spokane, Washington, USA.

Keynote - Testing in Django by Ana Balica
0:49:08
Published September 6, 2017
3,885 views

DjangoCon US 2017 - Keynote - Testing in Django by Ana Balica

The Django documentation section on testing starts with this: “Automated testing is an extremely useful bug-killing tool for the modern Web developer.” Nobody can argue with that. Testing is an integral part of modern software development, and Ana’s talk will offer an in-depth overview of how the Django testing framework evolved; showcase some common techniques, tools, and best practices; talk about speed improvements; and guide you through a real-world example of testing a Django app. Testing is fun, isn’t it?

This talk was presented at: https://2017.djangocon.us/talks/keynote-2/

LINKS:
Follow Ana Balica 👇
On Twitter: https://twitter.com/anabalica
Official homepage: https://ana-balica.github.io

Follow DjangCon US 👇
https://twitter.com/djangocon

Follow DEFNA 👇
https://twitter.com/defnado
https://www.defna.org/

Summary

Ana Balica traces Django’s testing framework from versions 1.0 to 1.11, explaining the test runner, test case hierarchy, test client and request factory, database transactions, test discovery, parallel execution, tags, and subtests. She compares tools and techniques such as Factory Boy, Hypothesis, mutation testing, mocks, and code coverage, while warning that coverage alone does not prove test quality. She argues that fast, maintainable tests depend on choosing the lightest appropriate test case, separating unit tests from integration tests, and moving complex business logic out of views and other Django components into plain Python objects that are easier to test.

Key takeaways

  • Django’s test framework evolved from a simple management command into a system supporting discovery, multiple databases, parallel execution, tags, subtests, and richer client behavior.
  • Use SimpleTestCase when database access is unnecessary, TransactionTestCase when real transaction behavior must be tested, and TestCase for faster database tests with transaction isolation.
  • Factory Boy and Hypothesis can generate varied data, while mutation testing can reveal weaknesses that ordinary code coverage misses.
  • Keep unit tests separate from slower functional and integration tests to maintain a fast development feedback loop.
  • Mocks are useful for isolation but should not be used reflexively to hide poor application design or merely to make tests faster.
  • Move complex, non-Django-specific business rules into plain Python classes or functions so they can be tested directly and reused more easily.

Summarised automatically from the transcript.

Chapters

  1. 0:14 Introduction to Django Testing Ana Balica introduces the talk and outlines its journey through the evolution and practice of testing in Django.
  2. 3:20 Django Testing Through the Versions A historical tour from Django 1.0 to 1.11 covers test runners, test cases, database behavior, discovery, parallelization, tags, and subtests.
  3. 21:09 The Django Test Suite Workflow The talk follows manage.py test through environment setup, test discovery, database setup, execution, and cleanup.
  4. 24:17 Test Case Classes and the Test Client The relationships among Django’s test case classes and the roles of the request factory, client, and client handler are explained.
  5. 27:22 Property-Based Testing with Hypothesis Factory Boy and Hypothesis demonstrate how generated data and property-based testing can expose edge cases in Django applications.
  6. 32:49 Mutation Testing and Test Quality Mutation testing is introduced as a way to assess whether a test suite catches meaningful changes in the code.
  7. 36:46 Test Speed and Feedback Loops The speaker discusses Django’s testing tutorial, the risks of slow suites, and practical ways to keep tests fast.
  8. 42:08 Unit, Functional, and Integration Tests Strategies for isolating fast unit tests from broader functional and integration tests are presented.
  9. 43:08 Decoupling Business Logic from Django A VAT calculation example compares view-, form-, and Python-class-based designs to show how decoupling improves reusability, extensibility, and testability.

Transcript

7,214 words · auto-generated Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:14

Good morning everyone. Morning. How are you feeling? Great. Are you energized? Are you full of energy? Oh, that's just repeating myself, be full of energy to get started with a day full of learning. Yes? Yes, we are And we're going to get started with a very long talk, a free rough overview of testing in Django and within Django. I'd like to say thank you to the organizers and the volunteers who make DjangoCon US happen, because you and I, we have no idea how much effort it takes to bring a conference to life. And not only that, but also create a welcoming environment where we all feel safe.

1:04

So let's just give them a round of applause. You might be wondering who am I? I'm Anna. Hello. Hi Anna. Hi Anna. And this is my Twitter handle if you want to get in touch with me. There's nothing you probably haven't heard about me because there's nothing outstanding that I did in my life. Genuinely I'm the kind of person who goes to work every day and tries to be the the best developer I can become. I usually say I'm in a long-term successful and happy relationship with Python. I've been doing Django for the past couple of years

1:50

and in terms of community involvement I usually do little things. Currently I'm co-organizing uh the London Pie Ladies meetup and I helped organise the JengaCon Europe last year. I sometimes mentor people and most often I give talks at conferences. I recently joined a company called Street Team as a senior software engineer. So yes, questions. I won't be taking questions on stage and there are two Two reasons for that. First is we actually might run out of time. And the second one is it never works out for me.

2:35

I'll be honest. While I'm on stage, I'm a bit nervous and And by the end of the talk, uh it's getting a little bit tricky to think straight and give a proper answer to a question. And I feel like I'm actually doing a disservice. So let's take it offstage. Uh I'll be at DjangoCon for the rest of the conference until Friday so come and come to me if you have questions, comments or feedback. I'll be happy to have a chat with you Right, so now that we got to know each other a little bit better, let's get started with testing. We're gonna take a trip all the way from version 1. 0 to 1. 11. and see how the Django test

3:20

framework evolved over time. It all started about 10, 11 years ago, sometime before getting Django out of beta. This ticket gets created. Add Unitest framework for end user Django applications. And it lists all sorts of benefits that you can have while having an integrated framework to test your apps. But there's this killer argument. As an added incentive, this is a feature that is present in Rails. And that's how it started. It started with a management command. Test. It accepts pseudopaths at the time that point to test. So you do

4:05

app dot testclass. test method We get a test runner. A test runner is also a config variable that points to a function that runs your tests. That means that from the very beginning, anyone can write their own test runner and test discoverer. The test runner setups the test environment and it looks up for tests in files test. py and models. py. At the very end, the test runner tears down the test environment and returns the results. At this point you might be squinting and thinking, models. py? Is she serious? Well, 10 years ago, apparently doc tests were very popular and they were used to test ORM a lot So models. py were the place

4:51

where model and application level tests were meant to be and everything else was left to test. py Client makes its appearance as a basic stateful browser mimicking stub that constructs requests, passes them to the views, and returns a response. response bypassing all networking altogether and interacting directly with the WISGI interface. At its core, client, it is the same client that we have today, it's just a more limited one If you're writing your test using Python Unitest library, then you've probably heard of test case. Well Django also has its own version of the test case, which is uh an extension of the unitest test case

5:36

because it can also do extra things like load fixtures and clean the stubby mailbox and initialize the client and clean clean the data after each test and it provides a couple of uh asset helper methods for example like a set redirect and template here the ones that we actually still have today. Version 1. 1 comes shortly after with a revolutionary set of HTTP methods. So RESTful APIs are not yet the next big thing, but Django is looking into the future and adds to the client put, had, delete, and options. Along with a test case, we also get a transaction test case. This is where we need to make a clear distinction what is the difference between a test case and a transaction test case.

6:24

So a test case acts like a burger because it wraps each test into an atomic block. It enters the transaction, it runs the test, the test does messy things to the database. And finally, the test case rollbacks this transaction and leaves a clean slate for the next test. Because of that, the test case will always prevent any commits or rollbacks. To the database because uh it wants to ensure that at the very end, when it rollbacks the transaction, the test restores the database to its initial state. And this is a big performance booster to uh the test case and for us to write in our tests. If you want to use transactions, that this is where you need to use transaction test case.

7:10

It will clean up after each test by flushing the data At the start of each test. And by the way, the test case from the previous version 1. 0 was doing exactly that. 1. 2, Django defines a new interface for testing runners. So we're switching from a function to a class. The class should have a method called run test that will act as an entry point. By default Django provides Django test suit runner. So this is not really related to the evolution of the Django test framework, but it's a silly bug that I really like, so I decided to put it in. So the test runner returns the number of errors plus failures all summed together.

7:57

And what Django does, it spits out this number of failed tests as a sys axis. So if you have if no tests have failed then it will naturally exit with a zero which is a success. If you have one test failure, then this is going to be one, which is a failure. that looks correct. If you have 42 failed tests, that'll be fine as well. If you have 256 it suddenly becomes a success. Whoops. This is because Cisa Exit number is an 8-bit number, so it overflows into a zero when you have 256. And talking from experience, it's not very hard to get 256

8:42

fail test So we fixed it. We just say sys exit one. So now we'll always either get a zero for success or a one for a failure. 256 now is a failure as it should be. Let's take a look at the definition of the test runner because something changed from the previous version. There's a new argument, fail fast. It allows you to terminate the test run on the first failure. And I find it very handy. The implementation has been removed ever since from the Django test framework because it's now available with The Python unit test lib, but good to know it exists.

9:29

Django has support for multiple databases with a replication strategy. So you can have a primary database and multiple replicas can be created This is how it looks like in DB settings. There's a default database and there's a replica. If the test mirror setting from the replica dictionary is missing Then there won't be any replication happening during your tests. Just the default database is created and tested against. If you want to test uh replication, then you need to set up a test mirror and if you want to use it today settings have changed so um it should look like this so let's go back to our data databases when the test environment is configured

10:15

a test version of the replica is not created instead the connection action to replica will be redirected to point to default. So as a result, the rights to the default will appear on the replica. But not because there's any there's some sort of data data replication happening between the two, but because it's actually the same database. 1. 3, we can now do a set query set equal and a set number of queries, which is handy for the test case, we also see a split of the two roles that the client embodied. So the role of manufacturing request is turned over to the request factory. And obtaining the response was left to the client. The request factory becomes

11:02

part of the public documented API. So how do you manufacture requests with the request factory? Very simple. You just create a Whiskey request object with a set of WISGI and request specific attributes like the protocol, the request method. payload cookies URL scheme and that's pretty much it doesn't do anything else very simple very factor ish so dog tests In the beginning, it seemed like such a great idea. You get both tests and you go and you get both documentation. In the long run, it became a nuisance to begun uh to debug. It also became apparent that doc tests aren't really good at providing neither high quality documentation

11:49

nor comprehensive tests. That 's why doc tests will be discovered discouraged to be used in the future. And Django is a good example of a big project that came through a love stage with the DocTest and just proved that this relationship can't work. 1. 4. In version 1. 4, the family of the test case classes receives an ex an extension pack. So we also we get the simple test case and the live server test case. Simple test case is very light because it doesn't hit the database. Live server test case runs an HTTP server so that you can use a testing framework like Selenium for your end-to-end test. And it is based on the transaction test case

12:36

Version 1. 5 is the time when Python 3 support landed in Django. Remember we talked about the transaction test case and we said that That it's flushing the database before each test. Well, at the time, other test runners become popular, for example, Nose, and developers figured out that there's a bit of a mess left by the transaction. So it has been updated so that now we flush the database not before each test but after. Let me show you. So if you flush the database uh before the test, you get the following sequence of actions. You flush, you run the transaction test case, and then you can run your test case. What happens is that right after the running the transaction test case, you get some dirty state right

13:24

writing there. So your test cases actually start with a not clean test database. That's why Django Default Test Runner is always rearranging tests so that the test cases run first and then the transaction test cases go. come. And even even though the transaction test case leaves some dirty state in there, it actually doesn't matter because we're going to destroy the database But we can solve this problem in a different way. We can just flush after and then all of our problems are solved and it doesn't really matter in what order we're running our test. tests. 1. 6. So there's one more HTTP method added to the client. This is

14:09

path And if you thought this list is complete, it is not. In version 1. 6, we see various improvements. We get a new test runner that is able to locate tests not only test that by models that by but anywhere from the installed apps as long as it matches specific file patterns. The new test discovery also means that we have no more tasks in models. py. Hooray! Pseudo paths are also removed. Now we can use real file system or dotted directory paths. This also allows running tests that are not inside a Django app listed in installed apps. And finally, doc tests won't be discovered anymore, and we say farewell to that.

14:56

1. 7. So three versions before 1. 7, Django introduced Unitest 2. Unitest 2 is a backport of the Unitest features that work with Django With older versions of Python. And because Django at the time was supporting Python 2. 4, it was vendoring Unitest 2. In 1. 7, Django drops support for older versions of Python, and so it removes the Unitest 2. And we can just use the regular unit test. At first, when the live server test case was introduced, it was serving all the static files as if you were running your server with with debug equals true. From this point on, live server test case will serve only files from the static root, trying to simulate the real production environment as close as possible.

15:46

If you want to serve all of the static files, then you can use the static live server test case. In 1. 8, we get the last known HTTP method to the test client. Can you tell me which one is that? Which one are we missing? HTTP spike someone? Okay, okay. This is trace. This is the last one. And once this one was defined, the client is complete Test case and its burger strategy was a performance booster. And around version 1. 8, a few other intricate changes were introduced that made the test case even better. Before that, think

16:31

version 1. 7, what test case did is wrap each test into a transaction. But it did it in the following way. You would enter the atomic block and you would load the fixtures, you would run the test, you would exit the atomic, in other words, roll back, and then you would close all of your database connections and you would repeat that for every single test that you have. With the arrival of Django 1. 8 We can do something better. So we actually wrap all of our tests from a test case inside of a different atomic block that is going to happen only once And what we are doing, we are loading the fixtures and we are closing the connections a single time rather than doing it for every other

17:19

So now every other every test within also runs within its own transaction block. But we get also the benefit of closing the connections and loading our fixtures a single time rather than doing it every time 1. 9 Whoever works or worked on a large project knows the pain of waiting for tests. finish. From my experience that would take the pain starts with about 20 minutes, sometimes it's 40, depends how large it is. And most of us know that the quick, easy win for that is to parallelize our tests. And also that That's the time when you find out how many non

18:04

-pure tests you have. Gjenko 1. 9 comes with a built-in support for parallelizing tests or using Python multi-processor. Module. In fact, Django's test suit is also parallelized. How does it work? The test runner spawns a number of workers. So if you don't specify how many, it will pick up the max number of holders. that you have. For each process we'll get an instant its own database. So each each worker gets its own database. Therefore, if you have four workers, you're going to get four databases. Test Discovery builds a test suite which is partitioned into to chunks of test case uh sub classes.

18:50

Workers pick up those subsuits of tests and run them once they're finished they pick up the next one and they're finished and they pick up the next one until they're done and they return the result If you're stuck with a version of Django prior to 1. 9, then you can use Nose multi-process plugin. It's also using Python multi-processing module but the main difference between the Django ones and the Nose multi-process is that you get only one database so uh you get a shared state and because of that you sometimes can get bizarre uh failures and errors. 1. 10. My favorite feature that was introduced in 1.

19:37

10 10 is tags. You can tag your tests, creating buckets of certain types of tests that you'd like to run together or you would like to exclude from your test run. Similar feature is implemented by other test runners like Nose and PyTest. If you're not on Django 1. 10, you always have the option to use use Nose 8 rib plugin or the PyTest custom markers. And finally 1. 11. We get all sorts of speed ups and And improvements. In fact, with every version, we continuously try to get better in in different areas of the testing framework. But there's one feature that caught my eye, which I think is

20:24

Nice to know about. So starting with Python 3. 4, Unitest module adds a context manager called subtest. It says a subtest executes the enclosed block as a subtest In this example, what happens is that the test won't exit on its first failure when i equals 1, because 1 is an i odd number. It will actually run through all of the options and it will um re and it will return all of the results for all of the values of the eye. So now with Django 1. 11 you can do that you can write your subtlet When running tests in parallel and Django will report all of the results correctly.

21:09

If you're with PyTest, you've probably this probably looks familiar to you because PyTAS has um parametrized. We went through the history of the Django test framework and we've touched on uh different topics here and there But let's see the flow of running a test suite as a whole or what happens when you run manage. py test So let's split up the test bad into multiple pieces and analyze each one of them and see what happens under the hood. We invoke the management command test and if we zoom in Came back. And if we zoom

21:54

in, that's pretty much a straightforward invocation. So we're getting the test runner and we're creating an instance of it passing a bunch of options and we run the test. Let's zoom in inside run tests So the first line in run test is setting up the test environment. That will do a bunch of things. So the main three key points of this method is setting up the lock mam email backend so that all of the email that we're sending during tests is stored into a stub list that we can inspect and we can uh access It easily. This is a very reduced email backend in its functionality. We also get the Django

22:41

the Django test renderer is replaced by an instrumented. test renderer. It sends signals to let others know about various events that happened during rendering. And finally we deactivate translation so that during tests your project will Um the test will respect your project set language. The next step after setting up the test environment is to build the suite of tests. The heavy load of building the suit is done by the Python unit. test uh library but django extends it um because it allows you to specify uh tags and number of calls and debug mode and all the different things. So it builds up a suite of tests. This is an aggregation of instances of test case classes all collected in a single place.

23:30

And that's and there's nothing More than that. The next step, we set up the databases, we run the checks, then we run the test, which is by the way completely delegated to the unitest text test runner. And for the cleanup, we tear down the databases. Afterwards, we need to tear down the test environment. This is pretty much the reversed order of what we did in the setup Test environment. So we need to bring back the original email backend, the original test renderer, and finally we delete the state and the mailbox. Last step, we return the results of the test suite by combining the number of total failures and errors.

24:17

And these lines should probably look familiar. to you at this point. A quick recap on the different test classes and how they relate to each other. So this is the hierarchy and the simple test case is at the very top, subclassed by the transaction test case, which is subclassed by the test case and the live server test case, finally specialized version of the live server test case. case which is the static life server test case. I've been saying test so many times. Simple test case is very very fast because it doesn't interact with the database. You you can't you can't query, you can't save, can't do any of those things. It has access to the task line though, so think of it as a slightly more advanced version of the usual Python

25:05

Unit test test case. Transaction test case is not fast at all, but it allows database queries and transactions. Test case is faster than the transaction test case, but not as fast as the The simple test case. It will allow you to do database queries but will restrict transactions. Live server test case will act as a transaction test case and it will launch a live HTTP server in a separate thread from the thread that's running the tests. And finally the static live servers is the specialization that in addition to everything else will serve all static files. Client is a very handy utility when you're testing your Django app

25:52

And the three core actors involved in browser mimicking are the request factory, client handler, and the client. We've talked about the request factory being the most straightforward um actor. It constructs requests and it encodes data. Client on the other hand performs more actions. First off, the client is stateful, so it retains cookies and hands sessions for the duration of the test client. It also does a few other things uh to the response. So we talked about the instrumented test renderer. Well the client is listening to those events.

26:37

events like template rendered and request exception and that's why it is able to list all of the templates and template context that was used to render your response. It also sets uh a couple of other things on the response, for example, the original request, uh the client object, JSON, resolve Match, it can also handle redirects and build a redirect chain that we can inspect. Main goal of the client handler is To return a Django HTTP response with the WISGI request attached to it. It will load all the middleware that you've set in your settings. It will disable the CSRF checks, which can be enabled back if you want to

27:22

and it tries to emulate as close as possible server and browser behavior We've talked about all the different tools that Django provides for testing. But how do we achieve quality in our tests? How do we make sure that the tests we write are reliable, future-proof, and fast enough? Reasoning about quality is not Easy. Besides our intuitive human understanding of what quality is, there are some tools that can help us. And let's start with something simple. Introduce Factory Boy together. tests. Lots of people like it because it has these shortcut

28:08

methods for creating models. It generates random yet realistic data because it's based on a different library called fake Faker and Faker has providers that based on certain rules create this random yet realistic data. The important part though is that every time you run your tests, you'll get different values compared to what you get with fixtures and there is a slim chance that Factory Boy might catch uh a bug for you. If you're interested in this kind of stuff, I'd suggest looking at the Python's hypothesis library. It's doing property-based testing and it takes the idea of random input to a whole new level.

28:55

So the idea of the property-based testing came from a Haskell library called QuickCheck and Hypothesis is the Python implementation. You can use, so the way you do it, you use a given decorator which can inject randomly generated data to your tests. you specify what kind of strategies do you want. So in this case we're using a strategy text which is very similar to the Factory Boys providers. And what it does once we decorate a function with the given um function This test will run multiple times, usually about 200, and it will test your function with all sorts of different edge uh

29:45

edge cases. That usually that often easily slip away from developers' attention. Since Django is an established way of doing web development In the Python world, Hypothesis comes with an extra module with support for Django. Similar to Factory Boy, we can create models populated with random data. To be noted with the hypothesis we always save the created model examples. So this is a real-world example of how hypothesis can be used for testing Django apps. If you look at the very top at the four imports that we have, there's one import

30:31

that is the most important. And this is the test case. We're using the hypothesis test case to be able to wrap each hypothesis test in into its own transaction rather than running all 200 of them in one single transaction, because then we'll get a mess. So let's remove the imports and focus on the test. This is a slightly stripped-down example that I borrowed from a presentation made at a Django local user group in London, made by the author of Hypothesis Dave. So we are testing what we're testing here. We're testing that if the project model instance can respect the limit of users a single project can have. You can specify collaboration limits. I on this project I just want to have three people and no more.

31:18

So what we're doing, we are injecting one project Model instance and a list of random amount of user models from zero to twenty. Then we check if the project is at its collaborate limit and we try to to add another user, we expect an exception to be raised because we don't want to accept more users than we have at our collaboration limit. Otherwise, adding a new user should work just Fine. And when you run the test, if I if hypothesis encounters that the test didn't succeed with a set of randomly generated data, it will tell you falsifying examples. So in this case we have a project with a limit

32:03

uh with a collaboration limit of one and we've been able to add two users where the second one didn't trigger an exception, which means there's something wrong. You have a bug. But here's the catch with all of that. Tests are best kept when they're simple and stupid. And one should not have bugs in their tests. With hypothesis, tests can become more complex and you might even reach a point where you will need tests for your tests. Which is ridiculous. So when using um but when used properly and timely, property-based testing can be a very valuable tool.

32:49

Um so keep in mind that test code coverage is probably the most known and widely used metric. So about a week ago I checked Django 's test code coverage. I ran it against SQLite and I got 75%. But here's the thing, test coverage is generally deceptive metric. If you have low code c code coverage then yes you definitely need more tests you're probably not testing your code properly. But if you have high code coverage well might be that your tests are very good but maybe you just kind of wrote them in a way that you hit all of the lines of your code, but you didn't roughly test it it. So unfortunately, from in my opinion, high code coverage doesn't necessarily imply high

33:36

code high quality of tests. We can do better. We can do better with mutation testing. Mutation testing is a concept that was introduced in the 70s. And it involves changing the code of a program in some small ways like little twigs and observing what happens when we run the test on this modified version of the code. And this is a very powerful idea. I thought it's it's so simple and and yet so brilliant. In Python mutation testing is implemented by a Package called MudPy and MudPy operates on the Python AST to implement mutations. So how does it work? You invoke mutation MudPy from the command line and you need to specify your target. And your unitest.

34:21

The unitest obviously needs to uh test your target code. So if your target source source code has an if statement and the if statement has a logical operand and so you're saying if foo and bar then do this. Mat by takes this logical operand and inverts it into an OR. So our target code becomes a slightly modified version of the original and we call it a mutant. Now, what we do, we run the test on the mutant. If the test failed on the mutant, this is good. We say that the mutant was killed by our test which adds points to our final mutation score. But if the mutant survived, then our test isn't good enough.

35:12

It means that it doesn't really matter if we're using and or or the test pass anyway. We must, we're probably not testing what we should be testing. Testing. And ModPi can do a lot more. There's a fairly long list of mutants. It can replace arithmetic operands, break continuous statements, conditional operands, constants. logical operators and it can delete it can even delete whole conditional branches and decorators and do a lot more This is what a stripped-down example of a test run looks like. You get a mutation score, which is a percentage in this case that you 2. 1 and it also tells you the total amount of mutants that it generated, the number of mutants that were killed, survived, incompetent,

36:01

time down. And uh I took I took this tool and I started playing with the Django test and see what results I can get. So one pleasant trend is the correlation of the high code card With a high mutation score. For example, Django Duration Utils has a mutation score of 89 and its mutation score is 100%. This looks really nice. This doesn't always hold true. For example, Django encoding UTLs mutation score is 64%, while the test code coverage is 100%. You can see that mutation score can actually give you better insights into what

36:46

how good your tests are Django tutorial is excellent. Beginners don't need to search the internet for any video tutorials or courses because if you go to the Django official web page you get the Django official tutorial and a whole section of it is dedicated to testing because testing in the Django world is an integral part of developing an app In my second year at university, I discovered Django. It's also the first time that I wrote a test. It was a very silly one. I was just making a request or a URL and expecting a 200 response you know but you know why why I wrote this test is because when I followed the tutorial and I created my Django

37:32

app I also got this file test. py And that caught my attention. The tutorial explained exactly what I need to do to get started with testing my code. So a testing framework is a powerful tool to motivate people. to write test and especially especially beginners. Django testing tutorial contains a section named when testing More is better. It says that yes, soon enough you're gonna feel like your project uh is spinning out of control because it it it grew so big and you have so many tests but you know just let it spin out of control it's fine let them grow when testing more is better there's a trap in there when with the number of tests growing your total test runtime becomes

38:20

slower. And Django gives you all of the tools and encourages you to write very high-level sort of functional style test. So the speed of your test can suffer And this has other implications. With slow tests you get a slow feedback loop. With a slow feedback loop, you get a slow development cycle. Sometimes if tests are so slow, people just stop running them. And that's why here are seven tips on how to make your tests uh not be slower than they actually need to be. And number four will shock you. Number one, use a faster and less secure password hasher during your test

39:05

like MD5. This is what Django does for its own uh Tests. Abuse simple test case when possible. If you don't need the database, if you're just writing simple tests, use simple test cases. Brilliant. any of the objects that you create in it because it's going to be called just once. And here's the shocking bullet point, number four. Use mocks everywhere. And I'm joking, I'm not serious. Just trying to emphasize that yes, this is a joke There's nothing inherently bad with mocks in themselves. Mocks are a really good tool that you need to know how to use.

39:52

But if you're using mocks for with the intention of speed up then there's probably something wrong in your app design and not in your tests. Mocks are really good for isolation so use them for isolation Be very careful what gets created in the setup method. Don't save model objects if it's not necessary. And try to isolate your unit test from the rest of the test , from the rest of your tests. And I would like to talk a bit more about the last three points So last year I learned that this is called the tight loop. So a tight loop like this, where we create some model instances, are red flags

40:38

that need to be identified. uh removed or reduced. Instead of doing create how we did in the previous example, what we can do, we can create an in-memory model just like that. just like this. Similarly, an in-memory model can be created using a factory build method, or you can create a stub object, which is just a couple of attributes. using the stub method. The last two are taken from the factory boy. The caveat here is that you can't always do that. For example, when you have a many-to-many relationship, you just have to save the model. And finally isolate unit tests from the rest of your tests.

41:23

Your test suite may look like this. It's a mix of you know unity unit-ish like tests of functional integration tests. tests and but it would have been a lot more useful uh and would allow creating a faster feedback loop if we separate unit tests from the rest. Because you can run unit tests all the time. You can even run them in a watch mode and they're really fast and they're gonna give you they're gonna give you the results instantly and then you can run all the rest of the tests less often. One way to do it is to use simple test case classes for unitest and tag them with a special label uh if you're in Django 1. 10. If you are stuck with an older version you can separate them in a different folder so that you can

42:08

can run them in total isolation from the rest of the tests. Even better, you can make them simple unit tests or Pi test, and which have no idea of what Jang is and write more of them but don't neglect functional and integration tests because obviously they are the one that validate the contracts between your unit units so you definitely should have both. Say your friend runs a little bakery business and they sell freshly baked bread And other similar nipples, they hire you to help them build an internal app for them. What the app needs to do is to help them figure out what should be the correct VAT value.

42:55

Value for their products and this will save some time for their account because ugh taxes. So we start with a basic naive schema where we get the product model and the product questionnaire Model with a one-to-one relationship between them. The product questionnaire is the key to our VAT decision making. It's a little add-on where we define basic yes or no questions and we build a decision. To eventually land on a defined answer. Is this product zero-rated VAT or is it a standard-rated VAT? We will need a view and we will need a model for it. The view will display the questionnaire and answers will be saved on the product model for

43:40

on the product questionnaire model for possible future recalculations in the same view will converge to a final decision and save it to the product with the known VAT value it's going to be really easy to compute the final price of the product. So I'll be showing example code for different approaches on how to solve this problem. Where at the very top we're gonna have the production code and at the bottom is going to be uh the test code. Solution number one. In a view, once we know that the form is valid, we check if the product is a biscuit and is it coated in chocolate. it. So in that case we're gonna set the VAT to 20%. This is a class-based view

44:26

because of its brevity. If you have an allergy towards class-based views, you can just translate it in your mind Into a function based view. As for the task, what we're doing, we are creating a product object, we are making a request by pass uh passing some data, and then we are referring to Refreshing the product from the database and we're checking if the VAT equals to 20. In reality, we understand that the if-else is going to be much bigger. We can split it into multiple methods but the essential part is that the entry point of this calculation is going to be in the view. And for each branch we're gonna have one test and each test will do the same thing. It will create a product object and it will make the request.

45:14

uh and it will check this VAT value. And that'll be that'll be re that'll be just repeating ourselves and we're gonna we're going to do it for uh biscuits and bread and and flapjacks and other items. So to test each one of those items and to calculate their to check their VAT value, we will also be testing The router will be interacting with the database to eventually send an input to receive an output. And if you're thinking that you can remove the database Based interactions with the mocks. I think mocks are not a solution in this case because we can do better. Solution number two. The conditional check gets moved to a more

45:59

appropriate place, the form. So in the test we're again creating a product model, we are instantiating the form, we're passing in some data, checking if the form is valid We're going to save the form, refresh the product from the database, and check if the VAT equals 20. To test it, we again need to create a product object but we no longer but we are no longer testing the routing which is an improvement over the solution number one via the form we send an input the answers to the questions To receive an output. Third and last solution moves the entire decision making to an old, boring Python class or a function.

46:45

If you want to. It has a method called calculate VAT that does exactly what it says. And in the test, we're instantiating the VAT calculator, calling the method with a bunch of arguments. arguments and we're expecting the result to be 20. Somehow we forget that Django is a web framework, not a program. programming language. Different constructs can live outside of core Django components. The VAT calculator has no idea what Django is and it doesn't need to. When testing, we're not testing the router, we're not testing the ORM. All we do with sending an input to receive An output. And what we did is called decoupling.

47:31

Decoupling has various benefits. Reusability. As a developer, I can pull out the VAT calculator and build a separate app that calculates VAT and helps others maybe figure out their VAT values. Extensibility. If my friend decides to expand their little business to a new country and this place has different VAT values, it's going to be much easier to update them in one place in the VAT calculator rather than going for all of the views and forms and trying to update it there. And finally, testability. You saw how easy it was to test the VAT calculator. This approach of isolating business logic allows creating more and faster unit tests and keep the functional and integration test

48:20

at a manager. Tests have more in them than we think. A test has the capability to drive a better app design. In terms of Django, it's about moving complex logic Away from any Django related components. If it's a complex behavior that isn't directly tied to the database, templates, forms, or views, then build objects that perform those complex actions. And unit test them. Thank you very much for your undivided attention and happy testing everyone.

Questions this talk answers

What is the difference between Django TestCase and TransactionTestCase?

TestCase wraps each test in an atomic transaction and rolls it back, making it faster but preventing the test from committing or rolling back. TransactionTestCase flushes the database after each test and is the appropriate choice when you need to test transaction behavior.

Discussed at 6:24

How can I test Django applications that use multiple databases or read replicas?

Configure the replica with a test mirror so its test connection points to the default test database. Django does not create a separate replica database; writes appear on the replica because both connections use the same database.

Discussed at 9:29

Which Django test case should I use for database-free, database, transaction, and browser-style tests?

Use SimpleTestCase for fast tests that do not touch the database, TransactionTestCase when testing transactions, and TestCase for faster database tests that do not need transaction control. LiveServerTestCase adds a live HTTP server for end-to-end tests, while StaticLiveServerTestCase also serves static files.

Discussed at 24:17

What happens when I run manage.py test in Django?

Django sets up the test environment, builds the test suite, creates and checks the databases, runs the tests through Python's unittest runner, tears down the databases, restores the original environment, and returns the combined failures and errors.

Discussed at 24:17

What is the difference between Django's test client and RequestFactory?

RequestFactory only constructs WSGI requests, while the test client sends those requests through Django and returns responses. The client also preserves cookies and sessions, follows redirects, records templates and context, and simulates more of server and browser behavior.

Discussed at 25:52

Is code coverage a reliable measure of Django test quality?

Low coverage indicates that more testing is probably needed, but high coverage only proves that lines were executed, not that behavior was tested well. Mutation testing provides a stronger signal by modifying code and checking whether the test suite catches those changes.

Discussed at 32:49

How does property-based testing work in Django with Hypothesis?

Hypothesis generates many varied inputs, including edge cases, and runs the test repeatedly; for Django it can also create models populated with random data. When a test fails, it reports a falsifying example that demonstrates the bug.

Discussed at 32:49

How can I speed up slow Django tests?

Use a faster password hasher, prefer SimpleTestCase when the database is unnecessary, avoid creating and saving objects in setup unless needed, use mocks for isolation rather than as a blanket speed fix, and separate fast unit tests from slower functional and integration tests. Django's parallel test support can also reduce runtime for suitable test suites.

Discussed at 38:20

How should I structure Django business logic so it is easier to test?

Move complex business rules out of views and forms into ordinary Python classes or functions that accept inputs and return outputs. This decoupling avoids testing routing, ORM, and other framework details for every case, making the logic reusable and faster to unit test.

Discussed at 46:45

Presenters

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos from DjangoCon US