Setting up your development environment for Django

Well, we're

Speaker 1: Ramon and Greg from DockySign. My um my name is What your name?

Speaker 2: My name is Ramon. I'm coming from Barcelona. I'm working at Dugusain as a sobrachit.

Speaker 1: Microphone.

Speaker 2: Yes.

Speaker 1: We don't have the remote control.

Speaker 2: No, exactly. Uh I'm coming from Barcelona. Uh I work at DocuSign as a software architect. Um now and he is.

Speaker 1: Yeah, my name is Greg Robbins. I'm the technical director of e-commerce at DocuSign. As Ramon said, we worked uh we worked together actually the last couple of years on a lot of projects, both here and uh in San Francisco and in Barcelona, uh e-commerce-related projects, marketing. uh lead acquisition related products. And right now we're working on some exciting Django projects for DocuSign for customers. support and uh lead acquisitions. So we are looking for Django developers who'd like to come to San Francisco and work with us, just so you know, can come catch up with us after the presentation if you'd like more information on that. So for this talk, we're going to talk about a pretty complex subject about uh setting up uh development environments.

Speaker 1: Uh we've created a actually Ramon has created an awesome uh repository that contains a full tutorial with extremely detailed step-by-step uh You know, Ramon dash V V V is kind of how he does things. But it it's really helpful because uh it walks you through every step of uh of uh installing the different components that you would need for this. Uh it also contains this presentation, so I would uh recommend you grab that at some point. Okay, so talking about reproducible environments, who should care? Um well developers care. Developers uh write code that need to run on some sort of development. Developers usually want to write code. We like to write code. We don't like Mucking about with environments. Nope. Well I do.

Speaker 1: But I'm just kind of strange. Um team leads. So as a team lead, I really like reproducible environments because I know that my team can get down to uh working on their their tasks instead of crapping around with broken environments and strange bugs that appear because things aren't consistent from one place to the next. SysOps people tend to appreciate this a lot because the work that we do when we're setting up a development environment for an application can also be applied to staging test production environments as well. And just kind of geeky people in general. I can see four really geeky people there that I know. But um in general, people that are involved in Django projects can can probably benefit from from having uh a way to set up environments.

Speaker 1: So why is it important? Um, you know, a typical project is going to have lots of software packages. It's going to have a maybe a database, maybe it's going to have a web server, it's going to have some caching system, it's going to have uh a queue of some sort. I don't know. There's there's myriad things that you know, Django itself, Python, uh all kinds of different software packages. Um setting these things up manually, uh while there are some people that are kind of strange that enjoy doing this, like myself, uh, it is pretty slow. It tends to be inaccurate because I may do it a different way than he does it. then she does it, then he does it. And we end up with different environments that that aren't accurate. They aren't consistent across across the board. And most people generally find this to be really boring work anyway.

Speaker 1: So um it can really turn into a nightmare when you have a team. Last year, Ramon and I, when we were in Barcelona doing a project, we had a remote team in in South America that They were pretty good coders, but they they really didn't have much of an idea about environments. And they I think a bit if we had had this system that we have now to just Create a git a git repository with all the stuff we need to set per weeks. We would have saved two weeks off the you know lead time off of our project. So And probably a lot of problems going forward too because every time we installed a new like we figured out we needed caching in the system. We'd hey you guys you gotta install caching. We'd have to write this tutorial about how to install caching and then they'd break it. And then It was just, we could have saved all of that by using this exact same system here.

Speaker 1: you know, uh from one from one realm to another, like development's not going to be the same as staging. It's not going to be the same as production. Maybe in development I have some sort of debugger running uh where I probably wouldn't have that in production. Although I I do know a guy who did that once but that's another I can tell you over beer later if you want, I'll tell you. It's horrific. Uh and these these develop these environments also change over time, right? I mean our applications are awesomely fast, but when they're not fast enough, maybe we need to introduce some sort of caching mechanism. So that's a change that maybe happens uh later on in the software development lifecycle. So we need reproducible environments. I think I've uh belabored the point quite a bit. Um I want to spin up these nuances

Speaker 1: in in just minutes and not hours or days or days or weeks. I want consistent results from them. And I want all of this in version control so that we can track it, share it, update it. So when we talk about setting up the reduced reproducible environments and this presentation here, we we're going to talk about the tools, what they do. some best practices, at least according to us. According to us. This is what so we're not uh

Speaker 2: gurus of this.

Speaker 1: No, we're not we're not gurus. We're really nice guys. Um but we're not, you know, world-class experts on this. We're just sharing with you the uh system that we've used that has worked for us and has given us pretty consistently good results. You were sharing with you the Git repo that has our the complete written tutorial dash VVV that Ramon put together, which is awesome. And we're not going to really look at a whole lot of code right now. So as I said, we're not covering Django development here. This is more about environments. We're not going to look at the code, and we're also not going to talk about DocuSign too much more. So the um what are the kind of problems we have that we need to solve? We have multiple dependencies. We need specific software versions. We probably don't have installation and configuration skills across the board on our team.

Speaker 1: Some people do, some people don't. They shouldn't really be that concerned with it anyway. Setup can take a long time or we have deadlines. So the solution, the solution is having a single place where the entire environment is defined. I want this under version control, able to be able to make and apply changes over time and um you know have it reusable as much as possible for different environments as we go from dev to staging. production. Vagrant and Chef Solo is what we need. Yeah. And well probably a visual designer for our presentations because That's not what we're good at. So uh Ramon, you want to talk a little bit about the toolbox?

Speaker 2: Okay. Okay. So the toolbox. The toolbox uh the tools we're gonna use, uh mainly Burrow Box, Brewer Box, it's a software that will allow us to create uh virtual machines, uh Git, everybody knows Git, next. Uh and Vagram. Vagran is uh just a kind of software that wraps around uh virtual box. It allows us to script How do we want to provision a machine? Uh Vagrant starts with uh vanilla boxes. Vanilla boxes meaning that they are bare bones. Uh they have nothing installed but the main operating system Uh one thing that it's awesome is that it offers a shared directory so you can run your application from the virtual machine, but you edit your application from your host with your

Speaker 2: preferred uh with your preferred editor for example I don't know uh PyCharm or Vim uh subversion Vim yeah Emacs

Speaker 1: No, not Emacs.

Speaker 2: Okay, okay, okay, okay. Is

Speaker 1: Nina out there? Yeah, we don't like Emacs.

Speaker 2: No, Nina was there.

Speaker 1: She left and we said no.

Speaker 2: Yeah. Uh so that you can code in local, uh run on the virtual machine And then uh the two preferred commands, Greg commands.

Speaker 1: My favorite commands.

Speaker 2: Yeah, your favorite commands.

Speaker 1: Yeah, so it's like the whipcracker on my team, right? I like All you have to do to get these development environments up and running is download the repository and go in and type Vagrant up and it will spin up The fully working virtual machine, you don't have to spend any more time configuring mucking about. Just two lines or two words, and you've got the machine up and running And the other one is vagrant provision, which is when we've made some changes to that environment, vagrant provision will apply all the uh It will make the virtual machine match what you've defined in your Green Chef code.

Speaker 2: And I think now would be a great time in order to spin up our demo machine. Yeah. Okay, so we can see all of that Uh I'm sorry, I need to this okay. Uh just that uh in the tutorial is explain what does note means, etc etc but Basically what we're doing is uh Vegrand app and saying uh as long as you are waking up the the virtual machine, just provision it. Okay. By the end of the by the end of the speech, we will see if it has worked or not.

Speaker 1: It might even work. Yeah.

Speaker 2: It might do it. Okay, so that's uh Vagrant and why we like Vagrant. Uh we like open source tools. So Vagrant is an open source tool and we like it. It allows us to do uh whatever we want with virtual machine. Uh it's very well uh adapted to VirtualBox, which is also uh an open source tool. There are a lot of ready-made boxes, Ubuntu, CentOS, whatever flavor of line-ups that you can imagine. If you want, you can also work with Windows, so it's cool And you can create your own base boxes in an easy way, in a very easy way. The other tool we have in our toolbox is Chef.

Speaker 2: Okay, what does Chef? If Vagrant is able to spin up a machine, Chef, what does it provision this machine? Install some software inside that uh in that into that virtual machine But in a way that we as developer like maybe uh more than if we were just C subs. It's uh doing code With Chef, you can do whatever you can do in the command line. You can create files, directories, delete them, whatever, I don't care. It's easy to keep track of your configuration changes. Uh it's easy to keep track uh in Git, so you can then uh share those changes with all your team and you get sure that all your team will have just the same configuration uh for their development machine, uh

Speaker 2: for example. And Chef uh makes all of that accessible through through the network uh using a server. Okay. So What's the main difference between Chef and Chef Solo? Chef Solo is just an open source port to Chef. It has no server, uh no server side, no server component. It's much more easy to get started with and it has a plugin for Vagrant that just works. So it's great to get started with that.

Speaker 1: So I'd like to comment, you know, there that So Chef is not a simple system. I mean it's um I think that we've given you the we'll we'll be giving you the steps in the in the tutorial and the repository of How to get up and running, and you can kind of see it laid out in a single tutorial, how other pieces fit together, which is something we couldn't find out on the out on the internet. But anything that you know we could do to simplify at the beginning was a big help. So you know removing the server component from Chef simplified Having said that, you know, we do like that that Chef has the server because as we evolve our projects and we evolve our environment's definitions, we'll probably want to use some variation of those. on staging, test, and production environments. So all that work that we've done is going to be usable later on in in Chef Server, which is what we we do use it at DocuSign.

Speaker 1: We use Chef Server to provision uh different environments for our for our product. Sorry.

Speaker 2: Oh that's cool. So why do we like Chef and Chef Solo? Mainly by the same reason that we like it Chef uh Vagrant because it's open source. As a big company, there are some times that we may need support. It's uh a paid support, it's available for chef. So that's cool. Uh it has gets integrated very well with VirtualBox. It's very widely used. There are a lot of it works with lots of operating systems Uh you can have a lot of cookbooks ready made in order to just get started with it. Chef solo, it's great for local environments so you can start learning right uh after download that component. Yeah.

Speaker 1: One thing that I you know like for it is uh you know if I'm running a team or if I'm if I'm running an organization is You know, again the fact that you can get you can get professional support from Opscode, which is the company that makes Chef. I mean they make this uh actually they've been bombarding my email box for the last month constantly. Trying to get me to buy this. But um and they also have training available so that you know if you do get serious that you want to move forward with this, you know, across different parts of your development organization, that uh they can provide uh you know training by experts on it so that make sure that you're using it in the in the most proper way. Yeah.

Speaker 2: So after all that, uh what we're trying to achieve What we're trying to achieve is just to spin up a virtual machine that runs the poles in Catalan my model tongue that means lice. Uh yeah. Uh the pulse application, yes. Uh and to do that, but

Speaker 1: it's just just thinking about that word, sorry.

Speaker 2: Yeah. Polish.

Speaker 1: Yeah. Well nice.

Speaker 2: And for that we just use uh an Ubuntu server twelve oh four. Uh an Nginx. It's not used in the uh

Speaker 1: scratching too, so Yeah.

Speaker 2: Wow. In the original tutorial. But it's interesting to the to have one of those. Uh Python, Django, and PostgreSQL as in Nginx. It's just To make things really interesting. Okay. So what do we need to do all of that? We need to have installed in our host machine we need to have installed Ruby, the Chef Development Kit, VirtualBox, uh Vagrant Git, all the tools that we have been talking about. You can find in the tutorial the links to those applications. And what are the two main parts that are implied when creating a virtual machine? We need the Vagram file. The Vagram file is

Speaker 2: the file that is created by Vagram when you use uh when you initialize it. And it will tell Vagram how to create the virtual machine. Okay. And we need that Chef Kitchen. Chef Kitchen will say what are the software that we need to get installed into the virtual machine. Okay. looks like

Speaker 1: all this yeah

Speaker 2: in a minute. Thank you. And of course the application we want to run. Okay. So what is an Agram file? A Vagram file just defines a Ruta machine and How how is a virtual machine defined? Just putting the required plugins that you may have installed for Draegrun, just saying how you want the network configuration for your virtual machine. the processors, whatever you can define in uh in Blue Robots, you can define it in a script in a Vagram file and to summit to it uh a Vagram file is a Ruby script so you can use all the power of Ruby Just just for that file. So

Speaker 1: just script it up.

Speaker 2: Yeah, it's scripted. So as it's a script, it's very easy to keep track of it and whatever I think that part you like it more.

Speaker 1: Oh, because I d this is my uh attempted artwork here. Uh I'm actually a pretty good guitar player, but uh visual design is not my thing. So um so talking about Chef, like Chef has these kind of main concepts and they're really cute names that all fit together. They have kitchen and then they have cookbooks and then they have recipes and then they have the knife. So we'll just talk a little bit about what those are. A kitchen contains many cookbooks. A kitchen it's like a project, right? A single project usually has a single kitchen, right? Cookbooks are collections of recipes. So you can get predefined cookbooks from for common tasks. Actually, you get them from the chef supermarket, as it's called. Or from Git repos or you can create your own. A cookbook is like a collection of

Speaker 1: related tasks. So you could have like an Apache cookbook, right? Install Apache, set up virtual hosts, set permissions, create the log files, blah, blah, blah. Or a MySQL cookbook that would do all these things related to setting up a MySQL server. Recipes are just specific instructions that are contained in the cookbook to do certain things, right? And then there's also the the command-by tool for using Chef, which is called Knife, right? And I guess, you know, following along with this, the application that you're going to serve up would be the main course. Right. Um do you want to talk about the Birgshelf and Birksfile?

Speaker 2: Yeah, let's talk about Bergshelf and Birksfile. Uh Birgshelf. No. But Birgshelf is just a dependency manager. Think about Birch shelf as you could think about PIP, uh, APT get or whatever. Uh and the birch file just lists what cookbooks do we will need, uh do we will need in order to install inside the Bluetooth machines. The the structure of the file is quite simple. You see there's a source, uh source line that says uh I want all those cookbooks uh downloaded from supermarketketchef. com. You define what cookbooks do you want in the verse file Uh for example for the APT cookbook we want uh uh the two dot five dot two version. Uh but not all every cookbook has

Speaker 2: to be downloaded from the source you have defined. You can say, hey, I want the PostgreSQL cookbook to be downloaded from a Git repo. Okay? Or uh even a path a path in your uh a path in your dire uh in your hard disk okay in your disk. In your local. Okay.

Speaker 1: So like we made one there that's uh cookbook that's going to install the polls app, right? So like it as you can see just points to the current directory site cookbooks.

Speaker 2: Exactly, it comes off.

Speaker 1: So you can grab QuickBooks kind of from everywhere and Berkshelf is where you manage those dependencies.

Speaker 2: we have said uh before we need it in order to start uh our adventure. Okay. So

Speaker 1: uh I c I created this another uh fine attempt at graphic art. uh where I kind of lined up the cute chef names with what they really are. So, you know, you can take a look at that. Kitchens are projects, cookbooks are like a story. uh you know a group of related tasks. Recipe would be a specific task. Knife is the command line tool and the brick shelf is dependency management for

Speaker 2: new to the to the to the chef wall it it can be quite confusing the terminology that it's used. So

Speaker 1: I would like

Speaker 2: to have that to have had that at the beginning. Okay.

Speaker 1: Yeah we uh we suffered a little bit with that. You know it it's pretty clever the way they thought of the naming conventions but It's a it's a little it's a lot to digest at first.

Speaker 2: Yes.

Speaker 1: Sorry, sorry.

Speaker 2: Okay.

Speaker 1: Sorry, sorry.

Speaker 2: So once we create the kitchen, what we will find in it? We will find environments and notes basically and some of the things that we're not going to talk about. would be a very long uh speech. Uh data bugs and roles. Uh

Speaker 1: the main things about you know the way that we're using it is environments and nodes. Yeah. Yeah. So we go

Speaker 2: So what's an environment? An environment defines a type of machine. It can be a development machine, a production machine, a staging machine, whatever, but it's a type of machine. Uh it's defined in a file named whatever you want. json, so it's a JSON file. And you can see an example here. There's nothing too much more to say about that. Okay. You can set attributes for the cookbooks you have inside the environment.

Speaker 1: Yeah, we have a pretty well defined like you know files in our in our git repos you can take a look at those.

Speaker 2: Yeah. So

Speaker 1: it's just JSON.

Speaker 2: Yeah, after environment you can find nodes. Notes uh as environments were a type of machine, a node. It's a specific machine, particular machine. Okay. So in our case we have a machine that's named balls. example. com. It's a JSON file as it was an environment. And it has a very cute attribute named Runlist and run list is a pretty important attribute in fact because uh It will list every res r every recipe that we want uh that we want our notes to get installed. Okay? And not just which recipes but in which order do those recipes have to be installed so keep that in mind because it's very important. Every recipe that we want to get installed has to be inside this

Speaker 2: run list. So cookbooks. You are pretty much good cook the cookbook.

Speaker 1: Alright, yeah, I like the cookbooks. So so the cookbooks. These define the recipes, specific instructions. They have uh some other components as well as an in a in a cookbook. A cookbook also uses templates. So I suppose you could use it for a lot of things, but we've been using it for like configuration files, right? So we have a a prefab, a pre-cooked configuration file, right? Like for Apache, for example. And in the the Apache files, basically the Apache. com file is going to be more or less the same. Uh except on different environments I may want to set different configurations, different amounts of memory and different paths and so on and so forth. So those things that change from one environment or one node to another. uh I would set an attributes file, right? So you can see it's got a quite a bit of uh

Speaker 1: quite a bit of um of configurability depending on how you want to set up and how you want to use it across different environments. There's also the concept of resources. So resources are they're like reusable class methods from other cookbooks, dependencies. I think when You see in our in our code where you set up the uh the PostgreSQL uh like it it requires the Python cookbook in order to set up some of the

Speaker 2: To create a user for the data for drive.

Speaker 1: Exactly, right. So this is kind of what what cookbooks do. And you know, once again, you can download existing cookbooks um you know from the supermarket or from from GitHub. You'll probably find a lot of them. Or you can create your own. Um creating your own kitchen is really easy. Um I'm not going to go through code here. Right. Yeah. So this is where I'm breaking my promise that I wasn't going to use code. But uh just to show how easy it is, I was able to do knife solo init dot knife being the command line tool for Ruby, and it set up the entire directory structure for the for the kitchen. Right. Which was great. Um setting up dependency management, which is called Birkshelf. It was also really easy. All I did was create a this Birks file. It's called Birks file, the root directory of a project.

Speaker 1: I list the source, which is the supermarket, and then I put in the different dependencies, different places, uh, different cookbooks that I want to grab in order to to set up my machines. And those can be local, they can be remote, they can be from the supermarket, they can be from GitHub, etc. And then creating your own cookbook is real easy too. Chef and Chef solo give you are are are really pretty good. So uh I was just I went into this there's a directory called Site Cookbooks which is where you keep your own uh custom made cookbooks or your or your extended cookbooks. Um and just by typing, you know, the command Burke's cookbook My cookbook, it set up the entire cookbook structure for me. And all I have to do at that point is add a little bit of code to the uh to the recipes

Speaker 1: default file, default RB. Uh and that's the one that's going to be run when the cookbook is called. So like in this example I think we're uh setting.

Speaker 2: In fact, this is uh this is a resource. Yeah. We've been talking before about that. Uh chef uh has Many resources. One of them is Baj. It allows you to to run it allows you to run uh command line. Uh command line uh Yeah, command code, commands, right? Commands. But there are more. I'm sorry, I'm not uh as you can see English native speaker.

Speaker 1: Your English is great. What are you talking about? It's better than mine.

Speaker 2: And we have uh more resources. We can we have the resource git, resource template, that every one of them is quite self-explanatory.

Speaker 1: There's there's a really pretty full set of uh you know commands that that Chef uses that you know it's really helpful to create files to uh set permissions to um Um you know, run bash commands, uh set up networking parts. I don't know. There's there's a whole lot that you can do with it. So the point of this is just really to show that it is really pretty easy to get started. Um The next thing you need to make sure and do, and we we put this in because we suffered horribly about this, because didn't realize that you had to do both of these things. You have to add it. You have to add your new cookbook to the dependency manager, otherwise it doesn't know it's there and it will just ignore it. And if you want it to actually run, then you need to add it to the um

Speaker 2: To the run list of the note.

Speaker 1: Right, to the run list of the note that he mentioned before. So those we thought that was uh we had enough suffering on our side that it was worth putting that into its own slide in our presentation.

Speaker 2: And tell it to you so you won't have to suffer.

Speaker 1: Yes. Okay, so we're we're kind of getting towards the end here. I want to talk just a little bit about wrapper cookbooks. Um this was an interesting concept for me in particular. As I've been saying, Opscode has all these great cookbooks in the supermarket that you can get and use. HTTP servers, databases, caching systems. Uh Python, PHP, Ruby, whatever you need. And a lot of times they pretty much do what you need to do, but you might need to just tweak a few things, or you might need to set some sort of configuration. that they hadn't included in the version that that they've currently got published. So like my first temptation, actually the first thing I did was start cloning these from GitHub. And then I realized that you know I'm kind of digging myself into a hole here and I came across the concept of wrapper cookbooks, uh

Speaker 1: which is actually a lot easier than trying to cone and man clone and manage your own your own cookbook. You simply Define a wrapper around one of the existing ones and add in the new functionality that you want to use. So it's really, you know, following the dry philosophy that we're all so fond of. Yeah. Um if you do start using this technology, I recommend you take a look at this uh uh blog entry called Doing Rapper Cookbooks Right by Julian Dunn. Uh I guess I forgot to put the link in here, but just Google that and it'll come up. Everyone refers to it. It's very helpful.

Speaker 2: It's in the tutorial.

Speaker 1: Yeah. So in uh summary we're pretty much at the end here. You know, we hope you give it a try. Check out VirtualBox Vagrant and Chef for development environments. Uh they're re reproducible quickly and accurately Everything's going to go into version control or get. You save a lot of time. It's cool and there 's skills that that you know you can use in In any project really. We haven't really been talking about Django, talking about environments. So whether you you have another project that's in some other language, PHP. uh Ruby, whatnot. Um yeah, it's all it's all valid for that. And uh we do use it at DocuSign. So um There's the link again. Do you want to check and see if that actually worked?

Speaker 2: Yes

Speaker 1: environment you're trying to see if set up.

Speaker 2: It worked. I'm sorry.

Speaker 1: There it is.

Speaker 2: Okay, first has to go out. Looks like it has ended the right wave. So we just

Speaker 1: for it

Speaker 2: pray for it and that's the amazing pulse yeah tutorial application whats up what's up not much the sky both thank you five volts against five bolts so bad again yes please So that page again, so it has worked. So

Speaker 1: once you get the components, you can download, you can do Vagrant Up, you get this amazing application.

Speaker 2: This very amazing application.

Speaker 1: If anyone has any questions, we'll try to answer them. As I said, we're not we're not gurus on the topic, but uh you know we'll try to answer them best we can. And anybody who's interested in coming and hanging out with us in uh San Francisco and jockey at DocuSign and doing some Django development, we'd love to speak to you afterwards

Speaker 2: I think the people wants to want to go to have a beer. It's a last speech.

Speaker 1: I know I do. Yeah.

Speaker 2: We'll do it, we'll do. So we'll do that. Okay.