BDD (Behavior Driven Development) Testing for Django Apps by Le Xiao

Speaker 1: Thank you so much. Thank you so much, Dallin. Um so um my name is Lur and uh this is my first time to give a speech in Django Kool. I'm super excited. So um today's topic is about BD test in Django app. So before we head into our first slide, um I'd like to ask you guys a question. So how many of you have no BDD or even say applied BDD in your development? Just put up your hands real quick. Hmm, good. Looks like San Diego, um PD has a green market in San Diego. So let me do a quick introduction for BDD. So BED represents for behavior-driven development. So it is a methodage which is designed for improving the communication between the tech people and the non-tech

Speaker 1: people. So it's a purely business oriented and uh um it's user focused and uh can be easily understood by um the non-tech people. So uh with this this mess storage, why we need a BDD So um because they say it's a behavior driven, so um it's user um behavior focused and it focuses more on user behaviors than just functions And also it focused more on software development rather than just testing itself. And also because this behavior-driven um development comes with the description, um, that means uh this description can be applied to different platforms. I'd like to break down my presentation into four parts. So first I'm going to introduce how we set up our

Speaker 1: BDD framework. uh in the real production. And the second is on how we integrate our BDD with our CI CD process. The third thing is how we do the refinement, so how we just speed up execution for the list testing. And the fourth thing is we are going to describe what's the limitation for the BDD. So before we set up our framework, we need to collect our requirements from our non-tech business people. So we are using Gherkin language to write down your description. So Gherkin is a um is a language built on the keywords. So as you can see here, We have three levels, feature, scenario, and step, and each level will have its own

Speaker 1: sub-keywords, so which can make it executable. This is a small um example. So as you can see, um the feature is at the very top, you have your feature name, and then um below the feature you're gonna have your scenario name and the Within the scenario, you're going to describe how you're going to execute your tests. And uh also like to add some tags for scenarios so so that I can just keep some tasks I don't want to run And also you can use the data table to have your own specified parameters or your own predefined data. So now that we have our requirements ready, we need to set up our BDD. So we need to get some tools for our BDD.

Speaker 1: So you already um except for your um create feature files you also need a sedanium um which is um for uh manipulating your browsers And also we need a thing on Core to Behave. It's a design framework for BD in Python. And thanks to our Django community, we have a Django test integration, which is Core to Behave Django. And the fourth thing is we need a thing called the page objects. It's a methodology for um just gathering all your uh DOM elements in your page, and then it's easier for you to um Do the management for your uh elements. Now we now can get our hands dirty now. Um so first install all the dependencies I mentioned in the for

Speaker 1: in the slides And Selenium Behave Django and also PDB for debugging. And then you have to install your web driver. So um because Selenium needs web driver to control the browsers And uh we need a chrome driver for Chrome and uh for a gecko driver for the Firefox And make sure you have already installed your web driver correctly. Make sure the pass is correct, the version is correct, otherwise we're going to have some compatible issues with Selenium And uh for the Mac users, you're very lucky, you can use a home brew to install all of them. It's very good. And the third thing is we need to create our features. So following on the Girking style, um you create our features.

Speaker 1: And then I want to emphasize we also applied our own on step convention. I'm going to introduce that in the following slide. The convention here is we are doing it in this way, so with the keywords you see in the Girkin style language, and then somebody do something in the page of the action words So the page objects, as I mentioned before, is where you're going to have all your elements ready and then it's easier for you to maintain and then simplify your UI Of course, and you can repeat um reduce the duplication of the code. And the action words is where we have all our implementation for the steps And uh also it's because we want to um manage

Speaker 1: those steps easily, so we're just using uh action words to represent them. Now, um so we can see that there is an example so uh for logging. So when I log in with the user name and password in the login page of login action, so that's how the step looks like With all those predefined stuff, we can now build our code structure like this. And now I just want to point out if you want to have some um Pre-defined data to load in your task. You can use Ficture. Um and second, if you wanna do some quick ways, you can just use Factory Boy, if anyone knows that, um, to generate some fake data. That's very that's very quick. And uh I'm going to uh go through those files very quickly.

Speaker 1: So first you need uh environment pie. So environment pie is where you configure your um test running. So based on the Girkin style um feature file, you're going to iterate your running through those um levels, so um before the beginning of the running testing and also the feature scenario and the steps. And uh um by default, the behavior jungle will have already set up the database for you before all, and then they will flash out your database after scenario, and uh um they will tear down the database after all. For the action words, we need to load action because one feature can have multiple steps, and that's why you need a multiple action words. And you need to tell the

Speaker 1: BDD framework, say which action words you want to load in. So based on our convention for the step definition, we uh use We just tell the BD uh if we find the keywords inside the stab, we load in the corresponding um action words. And for the actuals, that's where you implement the steps. So is it you just import in your page objects and then um tell and the BDC, you what do you want to perform? So here is an example, I log I import in login page and then I tell BDC I want to log in and input my username and password and the login So for the page object, this is one of the examples. So you need just select all the elements

Speaker 1: you need to perform your behaviors. So you have your user field, your name field, you have your password field. And uh the selecting method will be ID, cost name, and x pass. And uh so uh if you can talk to your front-end designer, is it just tell them, just give you some identifiers that make your life easier for PD testing. And now here's a step height. This is where you are going to match your action words with your step definition So that means actually BD has no syntax meaning, just they will execute all the steps in order. So just have to find a matching between your action words and your your steps. So it's a mapping file Now, after accompanying all of them, you are good to go

Speaker 1: and you can run the commands. So this is how the command looks like. Python Pi is like a Urunture server in Django and then add a behavior option. So you also can keep your database after running the testing. um that's uh easier for you the next one or even say you have some um same database. Now I also did a quick demo for you and uh um I created an app based on a Jenga tutorial And I run in the test um V testing against it. So the demo is about I um just voted for my presentation and I said, uh it's pretty bad and I increased what by the the votes for it increased by one. Okay, so the whole running just lasts for three seconds. Remember this time is

Speaker 1: very important later on Now we're going to talk about our um code delivery process. So everything will be online and uh we need some tools to help us. So first for the BDD we need a feature file management tool, which is we are using hip test. And for the CI platform, we are using Tribe CI and therefore free for the open source. And also we need a cross-path form test too. Here you can see a bunch of them on the market. And we are using Browser Stack. And also you are more than welcome to use headless mode of a browser. So Netcan running very fast. So PhantomJS, Papyr, yeah, whatever you like. And then I'm going

Speaker 1: to talk about the working flow. So first you're running all your tests in your local machine, make sure they're passed. You can run them in Chrome, you can run them in Firefox After they are passed, you can push the code into GitHub and GitHub is ready to integrate with Travis CI. So on Travis CI, we're going to do the parallel test. because we really want to have our testing passed in different browsers, different operating systems. So let's say it's called compatible testing. And uh after running all of them, we're going to have our reports ready on our management tool. So once we see our management tool reports, we're going to check out what uh scenarios I have ready filled. I failed we're going to debug this testing code in

Speaker 1: the corresponding yes, corresponding um platforms. And then you do the same thing again, reproach it to the GitHub and the cursor loop. Now, so we just um did a uh quick review of the VDD, so we now just come to the refinement of BD. As you can see, BDD is very costly. Just one scenario costs us three seconds to run. So we don't want to run everything for BDD. So first the principle here is you have to identify your critical test Where um and the second thing is you have to minimize your unnecessary UI interaction. We are not using BD to test against and the

Speaker 1: UI designing. Also, we can play a little trick with that to accelerate your execution. So we do some prerequisite steps by using the test client from Django testing framework. and then pass on the cookies into the web driver. I'll show the code. So the code here is we are using the client to log in your app So you log in with the username and the password, and then after completing all your prerequisite um steps you just pass in your cookies into your web driver and then use the web driver so that so that the web driver will have the session of the server and then do the uh

Speaker 1: just keep doing the user behaviors you're gonna perform on our page Now remember that we have the previous demo on three seconds for runability. Now let's see how fast it can be now. So same thing, logging with the um client and then do the voting. Yeah, two seconds, or around two seconds. And we say one second Okay, one second really means a lot. If you if especially when you run the testing on cloud and you're gonna run those testing on those cross-platform tools, they charge you by minutes. So save time, save your money No we are having our debugging tools. So somehow we don't have bugging our code, uh debugging code. We have our bug in the testing.

Speaker 1: So especially you need to know how to debug it. And by default, the behave will capture all the standing out, so maybe you are not able to do some breakpoint. And uh um we just need to trick back uh trace back the stack, the memory stack to say where it failed. Here's a code for debugging and uh as you can see we want to have this in a debugging mode. So when you set an option for it, so we say behave um behave debug on arrow, it's turned on. And then if it's failed Here it is in your step. So after step, you're going to trace back your stack and say where it failed to check out the variables, check out your logic. So that's how the command looks like. So you need to customize your command to run this

Speaker 1: testing. So make sure uh you have a no-capture option on your command. Um otherwise you're not able to just capture a breakpoint. Now, here is the last part. The limitations of BDD. So BDD is really good and it's very user-friendly. And but I'm not sailing Snake Wheel here. It does have some limitations. So first the um the logic of a description is linear. As you can see, they have many keywords and they execute keywords in order. So if you want to have some conditions to run BDD like if else, so one way for you to do that is you just have a data table ready and iterate the records to against your BD

Speaker 1: testing. And second one, um, the solution is you just create multiple um scenarios and pass in different parameters. The second limitation would be it's very hard for you to interact with those um complex JS widgets like a data picker. Um because Uh you need to tell um Zenium to click the very specified button, a specified link to uh use So I do suggest don't to don't bother yourself to how to code in um interacting with the complex JS widget. Um the third thing is um the errors raised in the different

Speaker 1: in browsers can be various. So somehow uh you may face an error in Chrome like um the element is not available in this page. However, when you do the same testing in Firefox, it may raise the error for you like this element is not interactable. So I do suggest if you want to test against whether you have ADC element on your um page, don't use arrows to raise up. So just create some um just Selecting multiple um multiple elements with the same class name or same ID and see whether the array is empty or it's not empty. So that can be very good Okay, that's war I have today and listen to my email and feel free to send me any questions.

Speaker 1: And uh how many minutes do we have? Uh

Speaker 2: I had a question I wanted to know if you could elaborate anymore on one of your biggest gotchas I know you had a slide dedicated to it, but uh what kind of vexed you the most with uh starting?

Speaker 1: Uh you mean you sorry?

Speaker 2: You had a slide on um some of your gotchas or things to note. Um you're not selling us snake oil. So I just I wanted to know one of your biggest gotchas and one of the biggest challenges. you had

Speaker 1: to do that. So um for running PD actually you are just you are manipulating the elements on your page. And uh uh it can have some glitches when you do the um BD testing, like uh the page is already loaded, but your BD uh framework didn't catch the elements. and it sort of narrow. So you do have to set up some waiting time for it. So that's the big challenge because if you do it on your local machine, it can be uh very fast. But if you do on online or even push up your um CI platform, um somehow it may not the the the elements may not be catched. So you need to check your network and check out um whether this is happening has been captured and you set up your uh waiting time like uh one second or two seconds

Speaker 1: to help you with that. So that's the that's the thing.

Speaker 3: Uh do you usually use asynchronous weights or do you just use like sleep or something like that. Oh yeah, time sleep with

Speaker 1: time sleep. So when you check a uh check about the framework they are using for behavior, actually they are setting an infinite loop and then Uh if the running time is surpassed 10 seconds, they're gonna stop and throw up a narrow.

Speaker 4: Can you say anything about um Any sort of design patterns between you and UI or front end that makes it easier for you to not test like the UI design while you're making these? uh page locators and etc.

Speaker 1: Yes. So you already if you're um so if you have a very complex landing page, you have many animation on a page, it's very hard for it to use BDD actually, yes. So that's why I'm saying BED is a methodology for you to uh have a uh integration testing between your front end and your back end. And you just need to see whether there's exactly thing on your page the um the data from API, for example. So um When we do this thing, we already talk to our front-end developer, say can you add some I mean specific cost name ID? It's easier for you to do the testing. Because even though um this speed can support X pass. The X

Speaker 1: pass can be very slow if you do the BD testing, so I do suggest you can use ID or use a class name in your front end and can be very helpful for BDD.

Speaker 5: Um so we use a little bit BDD, but a lot of times there's problems with the versions between Chrome and uh Firefox and whatever we're using for CI. How do you handle that?

Speaker 1: You mean how do you handle the errors in Chrome, uh Firefox?

Speaker 5: Like that regards to versioning, like how do you either keep up every Everything up to date or keep everything into one version or uh the like the whole sink with Selenium because it's really

Speaker 1: So that's why we use the cross-platform too. So like a Source Lab or Browser Stack they will handle those settings for you. So usually um when we do the CI at Travis we set up our versions. Um usually we use the latest one. a latest version for the browser. And if it's not compatible with the Canium, they will give you the indication. So like a a browser stack or uh like browser stack, it has a page for you to set up your uh a browser version. If the synonym is not compatible with this version, it won't show up. So that's how we reference our settings. Um

Speaker 6: you mentioned some weaknesses of BDD, but what are some unique strengths of BDD? Um do you guys do both BDD and then also unit testing, or um what combination is one button?

Speaker 1: So Like what I said, BDD is not everything. So you have to do those in the testing, functional testing, before you can do BDD. BDD is a complementary testing and it didn't cover everything Usually we see something you may not see your front end, but you have to consider it, like some error information gonna sew up in your page We mean our test with BDD, but you have to pick sure um there are some proper um showing up in your page.

Speaker 3: Just continuing on the last one, how how do you measure uh test coverage with uh Behave.

Speaker 1: So yeah, so that's why we need a description. So the description will be measured by the uh the user. So actually it is kind of like a mixing with integration testing and user acceptance testing And uh with the description has been proved, um you can just make sure you you can make sure that your uh testing is covered, especially for the user behaviors. And they say that okay, that's what I want for performing the behaviors on your on your on your on your page, then they said okay it's covered. That's what we say. So it's purely user behaved and we only test against the requirements we collect. So that's a that's a coverage

Speaker 2: All right. Well, let's give a round of hand. Uh

Speaker 1: okay.