Wagtail in the cloud

Speaker 1: We're okay to go. Sorry about that. Okay. All ready to go? I hope you can hear me clearly enough. I think I've got this microphone correct. I haven't done a very good job of um hiding the wires. I think I would make a very bad police informant or a you know a gang gangland snitch. So um uh I'm uh Daniele Procida. Uh I'll be talking about um Whitetail in the cloud. So that's me, that's what I look like on television. Um I am Python software. My favorite photograph on that. I work for DVO. I'll be talking about that during this. We do cloud hosting for um Python and Django, so

Speaker 1: people like you. I'm a member of the Django core team and I'm vice president of the Django software foundation. Which sounds glamorous but which I'm glad about but it's not really. And the other thing that I do a lot of is I'm involved with Python in Africa. So I've met Cody and Lisa. I've met Cody first in in Namibia in 2016. So these are the some of the PyCons that have uh I've been involved in in different ways. Uh I've been to uh Namibia four times. I'm going to got to the first PyCon Ghana this year year and we have plans for the first Pan African Python. So um maybe you've never thought of going to a Python event in Africa, but consider it they're different and

Speaker 1: Huge amounts of fun. These are my friends from uh Namibia. So that's one of the nicest things that I'm involved in. Very lucky that my company I work for gives me the opportunity to be involved with that and is uh very supportive. Some contact details, so this I don't really use Twitter and the best way things just come and talk to me. You know it by me. So Wagtail in the Cloud. Now that the title is problematic actually because I have a friend who's, as Tom mentioned, is a naturalist, so he gave me a bit of trouble over this last time. He wasn't happy with their servants. It was ornithologically incorrect. And um You give me a certain amount of uh of of grief about this.

Speaker 1: This was when I was going to speak at the Wagtail Space Concert. conference in Arnhem which I've done a couple of times uh I've been to a couple of times and um uh then he heard that the conference was called Wagtail Space it was really disgusted. So then he informed me that you know there are no Wagtails in the New World, so Wagtail US is uh Um I'm going to be talking about Django deployment or as we say in the business, you know. Because Django deployment, you know, Django is lovely. You can do a tutorial, you can get a website running on your machine very quickly, and uh for a developer it's It's a wonderful thing. Actually, not just a developer, even if you're just an a programmer who is learning a bit of programming, what you can do with Django

Speaker 1: extremely quickly is amazingly powerful. And again, as Tom mentioned, you try then try to get that thing on to the actual web in a way that's successful and so on, and that's a different kind of experience altogether. So in Django and Python we offer this wonderful entry level for programmers. And then a horrendous stumbling block when it comes to putting it on the web. And you compare that with the experience of um people who work with PHP and how easily they can do that and how um the um the progress from a website to uh from

Speaker 1: sorry from development to having a website is completely seamless because they're working already with a website that's being served, putting code into the same files that are just serving mainHTML. So you can understand how all those people got started with PHP. so quickly some um some years ago. So it's a fairly horrendous experience. And one of the things that uh we, and I say we as in this Wag this uh Wagtail community because it's very important for uh projects like Wagtail It's very important for anybody who's got an open source product in Django to be able to improve that. And um it's important for us at DVIO because that's what our business is partly about.

Speaker 1: It's about in in improving that so that it's nobody needs to s to swear about it. So I'm going to just go straight into a uh live demo to show you some things that we can do. possibly we're wrong, so um right and to show you how brave I'm being I'm gonna go So this is our control panel. Now let me see if I can make that a little bit bigger for you. So how's that for sizing? Okay. So this is our control panel. I've got three projects in there. I'll come back to that in a moment. What I'm going to do first is open a private window.

Speaker 1: And I'm just going to do uh DBL. com slash white tail. Take a minute. There we are. Okay. Let's just make that a bit smaller because it's a bit hard to see everything. Sorry. Does everybody know the Django uh Whagtail Bakery project? It's a very nice demo project with content populated for uh white health. I'm just going to click a robot and not and I'm going to launch this demo. And this is Of the Wagtail Bakery project

Speaker 1: and I can go through it and um see all the pages, I can go sorry, miss that button, I can go straight into the admin because I'm already logged in and do all the things. So This means I mean you saw how long it took to spool that up. I I started at DVO uh four years ago and We were pretty excited then because we were able to get a project deployed to the web in about between 16 and 18 minutes. And I just did it now, and a few seconds later I had a live project that somebody can now work with.

Speaker 1: So um Tom said you you could have a link somewhere on the Wagtail site. So there's gonna be a button on the Wagtail site that says try a demo of Wagtail, you'll get a 15 minute demo of this with the the content and it's the best possible way for somebody to experience a project because They can see screenshots of it and they can read the code and the README and the documentation, but there's nothing compared to seeing something with actual content in it as opposed to a blank sheet or a blank Screen because it doesn't give you the context. So this is extremely valuable for users to be able to understand what it is that you're trying to offer them.

Speaker 1: So I won't spend too much time on the whitetail bakery project, although it is actually a really nice demo and I understand from Tom that many Wagtail sites that are actually deployed look suspiciously like the Wagtail bakery demo. And there's a really good reason for that. And I don't think it's a bad thing at all. I think it's Makes perfect sense for somebody not to start with a blank page, especially when it's their first thing. It makes a lot of sense to take something and adapt it and make it their own. And if people are doing this with the bakery project, I think that means that actually you've got a really good demo here. And then you you that that should be um You should cherish that because I think that's working for you if that's happening.

Speaker 1: I lost my train of thought. It it's the it's the demo, it's gonna last for fifteen minutes. Here's our little uh timer and um it'll invite you to create an account so you can actually have uh uh projects of your own. Because after fifteen minutes this is going to it's going to to expire. We can oh by the way, why don't you go now right now to this website? You can do it yourselves. You can go to dv. com slash wagtail. Like that. And uh that's what webtail. Demo. com slash web tail. We can serve, I think Something like 80 of these 15-minute demos per hour. So as long as not every single person who reads hacker

Speaker 1: news jumps on it. We'll be able to serve up these uh all these um uh demos for them. So we're really pleased about this because we we don't just do it for um um Wagtail, we do it for I'm I'm a Django CMS core developer, so uh Django CMS is um uh uh a DVO product uh uh product and I want to do do uh is it um let's do this Let me do it in a no, I'm gonna have to do it here, sorry. Um I'm sure that's not I'm sure that's gonna give me a four of the F, of course it is.

Speaker 1: Sorry, I've I've got it in here. I can never remember things when I'm standing in front of an audience. So excuse me. Excuse me. There we are, that's what I wanted. So the idea is that we can do this for Other projects as well. Anything like Django CMS or Wagtail that gives you uh an app uh uh um a self-complete website. So content management systems like Wagtail and Jank CMS are really important because I think that between us, Django CMS and Wagtail, have brought more people to Django than probably any other kind of

Speaker 1: application because that's what ordinary people and businesses actually want. They want to be able to do something. There are plenty of very interesting and uh uh important Django libraries, but it's the content management systems that the these um comp Systems that are complete in themselves that solve problems for people that actually bring people to a language or a framework. Okay, so um I I hope the demos work for everybody who's tried it. Yeah, great. Okay, well there you are. Um my colleagues did such a great job on that. Um It's really amazing to see how the the the times for a deployment have have slowly come down over the years and I can't really say that much about what goes on inside

Speaker 1: to make this I can say a bit about it but most of it's a little bit beyond me but it's pretty impressive. So I know some of them are watching them so I hope they're pleased to see that The other thing I can show you right now here I'm logged in. So I can do the same thing as I just did. I could go to DVO. com slash uh Wagtail. The difference is now that I'm already logged into my DVO Cloud account. And what I going to happen here it's going to send me to our control panel and now this time instead of launching a demo it's going to create a demo project for me in my um account so let's uh create the demo project

Speaker 1: and give myself a bit more here that's uh so call this um wagtails base live dinner And we've got various other settings here, but I'm just going to leave them all alone. What's happening here is that this project that I'm newly creating is being duplicated from our um uh source uh wagtail demo project. Now what's going to happen first it's going to provision the project so it's going to set up uh the database and the containers because it's all based on uh Docker which I'll say a little bit more about that later. We've got uh uh a test server and a live server so it's just preparing it.

Speaker 1: It's going to commission the AWS media buckets. Everything's based in Git, so you can just just by creating it I've um uh I've got three commits in in in the project. Down here we have um of interest for example domains to attach it to live domain. We've got options for different server live server settings. We have our backman backup system, we've got access to environment variables, the kind of thing that you'd expect from a deployment system. Let's just give this a moment or two to finish. There might be a

Speaker 1: Q and that's why it's normally it's a bit quicker than this. There you are. Okay, so That's probably done. Okay, so it's now ready to deploy. So I'm going to deploy the the test server. This will be my private test server that only I get See, hit the button. What it's doing now is building the uh Docker images for the Docker image for the web container. Because it's a cloud system and we're using Docker, for example, we can use uh we can have multiple containers serving the same application, which means it's a good way of serving scaling rather than throwing just more RAM

Speaker 1: and CPU at a website. You can just have it handled by multiple containers. Um this will normally take about three minutes. So I while that's going on, I can um what's worth my showing you here. Um I can show you that uh uh we can have we have the concept of organizations so I can have different collaborators working on uh the projects in this uh organization with me so maybe my colleagues or somebody from client or something like that. I can find out everything that's in the in the activity stream. Everything that's actually happening. I can get the git logs of that if I want.

Speaker 1: Let's see if it's finished this The network's a bit slowly. Yeah, I think let's give this a moment more. Oh I know what I've done. Of course I've I've made the same mistake before. I I I created a new project in the wrong. I've created it My personal space is a mistake. So

Speaker 1: I can't find that project. Okay, it was um it was this one. Okay. So this is my newly created project. And by now it should have. Yeah. It's finished deploying. And I can launch it. the uh tests server. We've got a single sign on. And now I'm And there you are. So now this is the same bakery project, but now this is my project. This is

Speaker 1: in my account. I can uh it will persist, it will be free, not forever, because nothing lasts. quite forever but it's going to last for um a long time and I'm going to be able to do all the things that uh I need to do with it and I can start adapting it and change the content. I can uh get into the admin and there's everything. Um had before. So now if I'm a new user I've got an actual working site, a deployed site that's also ready to deploy onto A live server that other people can see under a domain in just a few minutes. And that's quite empowering for people because That's a good way of getting over that last stumbling block from

Speaker 1: between development and actual deployment. And especially a reliable deployment because even if you do get the thing online, you tend to do it all wrong if you're not a DevOps expert. So I think this is going to be of real value to people Who want to do things not necessarily from scratch because they're not because they they haven't done it many times as they would in an agency, but they just want to get started with something Now we can also work with this locally because in my control panel we've got a local development tab and we

Speaker 1: have a local development environment that matches the cloud environment. So I'm just going to copy this command here. And what I'm going to do, let's make a little bit larger. I'm going to paste that. Let me go through my CD and X projects. And this is going to Set up the entire project right here on my own machine in a Docker container and it's going to pull down the database and the media files and I can run it here. We've got some commands like the DBO project setup command that allow me to work on it here. And this process is the exact same process that happens in the cloud deployment, which is quite good because one of the other problems that happens is is that people they have something that's working nicely

Speaker 1: locally and then they set it up in a uh live environment which is different and then everything stops working or starts working uh differently. Oh it was quite quick So um it's just waiting for the um database and as long as I remember to shut down the other server we should be fine so it's there you are it's running the um database migrations that's probably a little glitch in the demo that I need to fix that we have said says we have some changes that are not reflected in a migration of a spectrum artist still.

Speaker 1: Putting down the media files. Okay, and let's go into the uh white tail space live demo and I can do Docker Compose up. And it's that's web is the uh local web container and now if I go back to my web browser and recall her And that's exactly the same thing that's running now on my local machine inside Docker that was running on the cloud. And if I open up the project, it's a pretty standard uh Django project.

Speaker 1: Um we have to it's a little bit opinionated so we have to work on the structure a little bit to make it possible to do these things, but it's pretty standard and I can for example let me see if I can Uh I can't make this text larger, but I can go into the static, I can go into there's the CSS, so I can um let's say that I want the headings instead of I want them to be in Um click sounds, of course. And now I've got a modified file in my git repository, so I'm going to do git as static git git repository Change CSS

Speaker 1: and then I'm going to git push um origin developer. And that's just pushing it up to the server. If we look here on the dashboard , we will now see the That there is one commit not deployed, which is the one I've just pushed up. And I can just deploy that. So it's going to run the deployment again. And You can see the change that I've made here, by the way, I didn't show you that there unexpected, wasn't it? As soon as this is finished deploying, that will also be reflected on the site that's running up.

Speaker 1: um over here. So it's a it's a workflow that developers will be fairly familiar with. We also have a DVO application which actually does the same thing so I can um um Have access to these commands to pull and push code and database and media, but through a graphical interface for people who prefer that. How am I doing for time? I'm taking a little bit of your time, so I'm conscious that I'm between you and your lunch, which is always a dangerous place to be. Okay, you're not going anywhere. A captive audience is my favorite kind, so um so we've we've got this very quite nice workflow that um

Speaker 1: uh also I think helps bridge that gap between the person who's managed to get something working locally and who needs to get it out there into the um into the real world. Maybe it's finished. What else can we do with this? Let's go back to the Uh no, I'll tell you what I'll do, just to make things nicer. I will add this. I'm going to move this to the project where I should have created it in the first place. So this is going to be in my wagtail. space project because I'm gonna transfer it. And now I'm going to go

Speaker 1: Sorry for the screen quite tell So I shouldn't have now I've got four projects there. Now I'm going to create a brand new project because of course sometimes people don't want to start with a wet wet uh with a wagtail bakery project. site they might want to start with a brand new project so this is going to be a plain old whacktail project and And I'm going to put it in this organization. I'm going to have it on our US region, Python 3 of course, a Wagtail site. That's it. That's quick project. And you've seen this happen already. And I'm going to show you this because it will introduce one of the things that makes all of this

Speaker 1: possible. So here's a plain Wagtail project, provisioning it, and let's just give that a second Now the th actually what I should do here is uh We've got what we have we call this our add-on system and it's a way of this is what we do here is possible because we are able to package Python Django applications in such a way that we can stitch them together and deploy them through these various APIs and Um okay and I know what's happened here, excuse me that

Speaker 1: I I hit a button too soon. Let me just So this is a white-tailed project. So this is what determines what of which of our add-ons can be used with this project according to the control panel. Of course, when you're working on your code locally you have freedom to do anything you like, but right now we're working with the APIs and the control panel. So let's see which add-ons are available for this project. And one of the things I want to do here is let's have a look at what's installed. I've got Django installed and I want to update that to Django 1. 11, so I need to do that first. And let's change the version

Speaker 1: because 1. 11 is not currently our um default version, we have to be Django itself is quite conservative and we also have to be even more conservative. So it means that by the time you've got two or three uh parties in that chain, it means that You're working with things that are several versions out of date and people don't always realize this when they are producing the software how many steps there are to the actual users. Actually Django I think is pretty good at maintaining this uh chain of backwards compatibility so that people are not always racing to um Keep up. So I've updated that. And what I would like to do now is I want to install a couple of things.

Speaker 1: You know when you create a brand new web Project you have to create a little home application. So I on the train from New York I turned that into a New York Cloud add-on, so I'm installing it now and I'm just going to install. I don't know if you know about this, you probably do. It's just a very basic extension of the um uh Wagtail page model so that you can add additional fields and actually do things in your uh uh pages. And does anyone know Wagtail Puput? It's Yeah. It's a web blog for um uh white tail. Uh they sent me some stickers, so there's some stickers there.

Speaker 1: Now unfortunately I wanted to show you the reason I'm doing it this way, I wanted to install Puput into the bakery project, but they both have a model called something like blog page or blog home or something like that. Anyway, they're incompatible. So maybe that's something that I'll be able to fix during this sprint that I couldn't add it into there. So I've this is why I'm doing it in this plain old Django project. So now on my dashboard For the project, I should have a few undeployed commits and including those two applications that I uh just installed, so you can see all the commits that have added up to that. And I can deploy this. This is going to take a um and

Speaker 1: um there it was started now. So that will take a um a few moments. If I wanted to set that up locally right now I could do the same thing. Let's go If I refresh this, that should now be in comic comic sands. That was the one I deployed a few moments ago. I can do two things at once. So I'm not gonna make you wait for this one. I've lost it already. Oh it's it's gonna be finished deploying it. in uh a minute or so, but uh yeah I I'll move on because um I don't want to keep you too long.

Speaker 1: No sign of the pizzas yet? Um they're on their way. They're on their way, okay. All right. Oh, okay, we need that. We'll we'll we'll wait for this one. It's just about there. As soon as that turns green, it's ready to be. Deployed. There you are. Okay. So this is the brand new plain old Wagtail site. We're back to our single sign-on. In a moment. There you go. And I'm logged in now Or will we? Okay, so I can just go to the

Speaker 1: admin and There's the home. Oh look, this is something that I meant to show you. We're on Wagtail 2. 0. That's the latest version of Wagtail on our system. One of the things I want to work on is to get it up to the latest version of version on our system at this uh the spritz. So here we are we can create a new child page and there we are we've got the two applications the uh weblog and the little homepage application. So that's how quick it is to when you have the applications to roll out new sites. based on that. I think that's all I wanted to show you in terms of the actual live demonstration, but of course I can show you more things if

Speaker 1: you have questions. So I'm just going to go back to my slides and I'll I'll quickly wrap through and uh Finish off. Well nothing went really wrong. So I should be three things. Digo. com slash Wagtail for the demo is so you can either have an anonymous an anonymous demo that launches in a few seconds, or you can create a Wagtail bakery project of your own and then do with it as you wish. Or you can just create a new Wagtail project or a Django CMS project or a plain Django CMS project. Jagger project from your own account. So that's DVO Cloud. That's an artist's representation of it. I think that's actually what it looks like. Inside, but it's Dockerized Django hosting with all the bits um joined up uh together.

Speaker 1: I gave you a little glimpse of the development tools and the workflow that we have. It's all open uh standards uh and uh uh tools so it's pretty easy to take away a DBO cloud projects and deploy it somewhere else if you get a sickle bus, although that doesn't happen. For deployment, we have this end-to-end wiring, so everything is stitched up and you don't have to worry about it. So the idea is that if you're a developer, you write the programs and we'll deploy them. Uh site reliability that's probably our most important thing for us that we have uh uh engineers across SREs across the world, so there's always similar This end-to-end

Speaker 1: wiring, this is what made what I showed uh possible. So the the add-ons system, which is uh extremely clever and allows applications to configure themselves in Django projects and I can there's more to it I can show you that if if you're interested. And really importantly for projects like Django CMS Wagel, we can put these web applications right in front of new users, the ones who won't necessarily follow a readme on GitHub or the documentation to get started. Some of them will be small site owners, you know, people who don't have a development team, or maybe there'll be the business people or corporate web managers who want this necessarily technical but have an idea, have a feel for what it is that they're looking for. For the developer it does streamline

Speaker 1: development because it takes deployments. uh makes deployment somebody else's problem and it allows us to do some um pretty interesting things. Why Wagtail? Because Divo is behind Django CMS, so Wagtail Well, first of all, I've got a very soft spot for um Wagtail because it was launched at the first conference I organized back in Cardiff in 2014. And um I didn't actually know that um Matt and Tom were going to launch uh White Hill. Well that was quite a validation of of my event, so to have this new uh very shiny uh CMS launch there. Um we are seeing a growing

Speaker 1: Wagtail user and developer community and more people are asking us about uh Wagtail for their uh clients, development agencies who use uh DVO Cloud. So it's important for us to support it. It's important for us to give those agencies more easy options and I think just code diversity all by itself is an important thing that we should not be a monoculture of one uh project and um Why you might be interested in video cloud? Well we have a partnership program, so these are different

Speaker 2: Green Blue Deployment.

Speaker 1: Well we we have a a stage server and a uh a live server, test and live, and we're going to be soon having an arbitrary number of servers in the pipeline. So you first deploy it to Locally on your machine, then to test and then to live. Yes, we do have rollback because everything's in Git. We've got ever backups are made automatically so you can revert to uh You'd like to say, okay, well I'll mention that to our I'll mention that to our our our team. But it yes, it is designed for that, so we take snapshots and uh uh it's quite easy to unmake a mistake.

Speaker 3: I I always thought that DigitalOcean was the big host for it. Are you competing with them and

Speaker 1: Well wis um DVO is dedicated to Python and Django and nothing else. So you can actually deploy non-Python and non-Django applications on our platform, but you're you're you might as well do that somewhere else that's not a specialist uh provider. And I think that um even with DigitalOcean, for example, although they provide many of the steps along the way, they don't join everything up. You're still responsible for that and you would be responsible for the uh site reliability uh you know if your site goes down and when your asleep I don't think maybe they do look after it for you but

Speaker 4: mostly they just provide a server so it's just yeah yeah yeah Yeah. They have a couple of application types, but yeah, it's pretty much here's a here's a server, go do what you want to do with it.

Speaker 1: Yeah.

Speaker 4: You've got to be a s admin if you want.

Speaker 1: So, you know, it's a slightly different space, but we hope that people might think that this is a good idea because then they don't have to worry about the deployment and the DevOps and and and stuff. so on. Um

Speaker 2: if you have an existing application somewhere, is there a process for moving that into your framework here?

Speaker 1: Yes and um uh sometimes it's very very easy and sometimes if the setup is a little bit Uh uh more tr sometimes it can be more tricky, but it's basically a Django it's basically a Django project. If I showed you what the project looks like, you would recognize it. Some things, for example, need to be uh shifted into parts of our system so that our system can take care of them. So some of the things that you might have done manually might now instead of configuration in settings it might need to be uh set up as environment variables but nothing very nothing very major. We've got some documentation on how to do that.

Speaker 1: But it's actually one of the things that I really want to improve in our documentation because Because if you're coming to it for the first time, it can be tricky. So that's one of the things that if you've got a project, I'd be happy to look at it with you, because that will help me understand where people's pain points are. Thank you.

Speaker 5: It's a simple question. Like if I if I have a domain in Google domains and I I want to create a WideKale bakery Example and then like the main pointer there. And you have to sign up for an account.

Speaker 1: Everything's free until you decide that you're going to Deploy the live server with your own domain. So you can deploy the live server, but it'll still be on our domain until you start paying for it. But other than that, it's free.

Speaker 2: The example of the story that like the local work went to test eventually. And how does Cinema optics get handled?

Speaker 1: You can set up different tracking branches for the different servers if you want as an option. But the standard workflow is it goes first to test and then you can choose whether to deploy it also to live.

Speaker 6: I saw that the Docker compose file got copied down Does that mean you can add containers to that and not a lot of the candles behind the scenes on your end? So adding a Redis container, you could then configure your application. Does that scale as well?

Speaker 1: I don't think you can add arbitrary containers. containers and deploy them on the cloud. No. Depending on what exactly you want to do, there might be an option to do it. Okay, unless someone's absolutely desperate for a last question, I think something quite important has arrived. So um I'd be really happy to hear how you get on with it and if there's any suggestions you can have. Thank you very much and um and I would like to say uh thank you very much to Tim and all the volunteers and for the uh the university for hosting us here. It's really nice to be here. Thank you.