Sharing is Caring: Contributing to the Wagtail Ecosystem

Speaker 1: Alright, sorry about the delay. We will try to figure this out. Okay. So my talk is called Sharing is Caring, Contributing to the Wagtail Ecosystem. My name is Andy Chosak. I'm a back-end web developer and I'm a member of the Wagtail Core team. I work for the Consumer Financial Protection Bureau, which Which you may know by its initials, CFPB. As I'm a government employee, I have to put this disclaimer up here, which you can all read basically that this presentation is my own opinion and not representing the CFP. So this is the website of the CFPB, consumerfinance. gov, which of course

Speaker 1: runs on Wagtail. We love Wagtail. We have a large multidisciplinary team of front-end developers, backend developers, UX designers, graphic designers, content specialists. specialists that all work together uh on our website. And uh our website is also open source um on GitHub. So if you want to poke around and see uh how a US government agency is using Wagtail You can see it warts and all. We're not quite on Wagtail 2. 0 yet, but hopefully we'll get there very soon. So today what I want to talk about is the Wagtail package ecosystem. So first I want to give an overview of what

Speaker 1: what I think of when I think of the Wagtail ecosystem. And then I want to talk about a few of the packages that the CFBD has contributed back and open sourced. And then I want to give a little bit of information and encouragement to the rest of you to contribute your own packages. Back to the community. So, first talking a little bit about the Wagner ecosystem. So, when I first started thinking about this talk, my motivation was mainly to talk about the software that we've written and open. sourced and why these are cool plugins and packages to use for Wagtail. But when I started thinking about it, I realized, and today is another good demonstration of this, that you know the ecosystem is more than just the code. The code is is just sort of one piece of it.

Speaker 1: Even if you had the best well-written CMS in the world, it would be nothing without the community behind it. So the ecosystem consists of the users, the people that are using Wagtail. you know, developers, the organizations that sponsor um the the development of of Wagtail and conferences like this, uh the the greater community, that's people that are participating in the Slack channels and mailing lists. And then over here on the right are some of the more tangible things. So there's the source code documentation, blog posts, videos, and so on. And then and then also third-party packages, which is one I one I want to talk mainly about today. So, you know, most of us in this room, I guess, have some experience with using Wagtail as part of a Django project. And people may be generally familiar with the Django package ecosystem.

Speaker 1: So these are things like our Django Rest framework, for example. There's a lot of very popular third-party plugins which are not part of Django but are but are also frequently used on on Django sites. And so Wagtail also has quite a nice set of packages as well that people might not be familiar with or maybe only partially familiar with. with and I I want to demonstrate and briefly talk about some of these. So if hopefully this will work if I go over here, let's see Okay, so a couple of resources that I want to talk about. And I should say I take no responsibility for any of these. These are just great resources that I think everybody should know about and take a look. So first is there's this uh repository called Awesome Wagtail, which is hosted by Springload, which is a curated list

Speaker 1: of many, many both Python packages but also other resources um all about uh wagtail. There's a there's a similar one of these uh for awesome Django, but this is more Wagtail specific. Um and you can see there's there's a a a a large list of um many many useful packages. That you might find when you're working on your Wagtail sites and you're thinking about doing something, you know, whether it's about something about accessibility or maybe a plugin for Draft. js or um things about static sites, whatever it is, you know, come to Awesome WebCal and take a look and see, you know, maybe somebody else has already tried to do something similar. There's a lot of packages in here which which are very useful.

Speaker 1: This page also has toward the bottom a number of things that are not packages, so some resources, blog posts, some of the recordings from other Waco space. Conference very, very useful, lots of very useful packages listed on this page. Another place to look is the Python package index or PyPI, which if people are not familiar with this When you on your console you're installing things and you type pip install something, this is typically where that comes from. And if you go on PyPI and you search for Wagtail, you see there's 180 projects. So of course, you know, the top result is Wagtail, but there's many, many other, and there's a large overlap here between the things that are here and that are on Awesome

Speaker 1: Web. tail um but uh this is also you know a great place where you can come and see um you know when things are updated what their latest version is look at the readme for password and so on. Very recently, uh I guess within the last few months or so, we've also gotten Wagtail added If you come over here on the left, there's this framework drop-down. And it's very nice that now Wagtail is listed here as one of the frameworks you can search by. Now, unfortunately, we don't quite have too many packages which have included there's a little bit of metadata that needs to get included in the in the package to kind of get out to show up here. But eventually hopefully we can get more people using that metadata so that way you can search

Speaker 1: because you know you may potentially have packages that don't have Wagtail in their name and so this This is uh you know just another way that you can find packages which people have published specifically for Wagtail. Uh another place to look is um GitHub topics. So uh On GitHub you can have repositories get tagged with topics. And so this is another place you can look and find open source Wagtail projects. And what's nice about this and what's a little bit different from the previous resources That I've mentioned is that this also includes Wagtail sites. So not just packages or plugins that people have made for Wagtail, but also full websites which are running on Wagtail Wagtail.

Speaker 1: So this is a nice place to look for examples of other people and full sites end-to-end that are using Wagtail. And then finally, there's another curated uh list here on the Made with Webtail site. Which has a very nice browsable interface to find sites, many of which are not open source, that are built with Wagtail. And it's really cool to be able to go in here and see just the diversity of uses that Wagtail is being used, you know, for nonprofits, for industry, for all all all different kinds of users. And so I'd encourage you if you if you make sites with Wagtail to contribute them back to this just as a you know demonstration of the growing

Speaker 1: uh you know the growing use and adoption of of Wagtail. So um so yeah definitely check check these things out. And so the CFPB website is is one of these websites um And I want to talk a little bit about some of the some of the things that we've contributed back to the terms of our packages. Oops. Okay, so uh I want to talk about three of our packages uh that we published uh in alphabetical order here, uh webtail flags, inventory and sharing. And we have a couple of others that are kind of coming down the pike. These are all things that we we had specific needs for things that we wanted out of Wagtail that weren't built into the core function

Speaker 1: And we built them into these packages and we found them very useful and hopefully others will have interest and find them useful as well. So the first one I want to talk about is Wagtail flags. I have to give most credit this to to Will Barton, one of my CFP colleagues here. And this is a package that lets you use feature flags on your Wagtail. sites. So people may or may not be familiar with the concept of feature flags. Also sometimes people call these feature toggles. And the idea behind using feature flags is A way to more iteratively, or continuously rather, deploy code to your site.

Speaker 1: So instead of let's say I have my site version one and I want to launch some new feature, what people will typically do is you'll have version one of the code running without the feature, then you have version two of the code that has the feature implemented. And then you deploy that new version of the code to put that feature into production. So you're kind of doing two things at once. You're both deploying the new version of the code and you're also turning the feature on to your users. So what feature flags let you do is the idea is you instead of doing both those things at once, you continuously deploy the the code, but it's just hidden. It's sort of turned off to the public. So the code is out there, it's being deployed, but the path in the code that would expose that feature to your users is turned off behind a flag.

Speaker 1: And then the act of actually enabling that functionality for your users is just a feature togwin code. It's just an on-off switch. So all the code is already sitting out there in production. It's just a matter of time Turning it on and off. And this is really neat because it lets you do a couple of clever things. So for example, if you have a site with a lot of traffic, something you might want to do is deploy a new bit of functionality to only Only a small percentage of those users for for some testing. So you say, let me try this feature out on 1% of my users, and then you can kind of see how this works before you roll it out across the board. You could also do it, you know, more complex complicated kind of A-B testing if you're trying to test out two different changes potentially. And so there's all sorts of things you might want to do like

Speaker 1: You might want to say, I want to have a feature that is scheduled to go live at a certain time. So you have it behind a flag and then you have some logic in there that says turn this flag on at this particular time or you might want to say I only want users that are coming from the EU to you know maybe get uh you know certain cookie banner or something like that. So all sorts of different sort of segmentation you might want to do. So the the idea behind this Wagtail Flags um package is to um allow you to do this kind of thing in web Wagtail. So let me do a quick demo of how this would work. So this is the repository on GitHub CPB Wagtail flags. And I'm going to just do a quick demo. This is the same Wagtail Bakery um demo site that uh danielle uh demoed earlier which is is really great i totally agree with uh

Speaker 1: doing quick prototypes and experimentation So a very, very simple kind of thing we want we might want to do is let's say we have our our bakery um e-commerce site and we want to have a sale that we want to start. I'm working, I'm a developer and I've been tasked with maybe developing some new um Some new content to put on the site about a sale that's going to be starting in July. So I want to be able to share my draft content with the rest of my team and test out that it works and so on. But I have to do all that before the content goes live for the end user. So The idea behind uh uh doing this with a

Speaker 1: with a uh a flag is so let's say I've got my my template here, let me make this a little bigger. Um Right, so here's my template. And what I want to do is in the header of my page, I want to let's say have a banner that says, you know, there's a big sale on bread, you know, click here too. to my sale page, but I only want that banner to um to appear in certain uh situations. So only if let's say my my July flag my July sale flag is set Um so what this Wagtail Flags package lets you do is it defines some template tags and some related things like that where you can

Speaker 1: sort of do some checks and say only include let's say this this snippet if the if the flag is defined. And then when I'm in my Wagtail admin now under here under the settings, I have a new section for the Wagtail flags. Oh no. Shoot. I think I have to. Oh, that's embarrassing. Um I'm sorry, hold on one minute. Let me just Just uh ,

Speaker 1: Okay, cool. All right. So this is what the interface looks like. And what it lets you do is you can say, define a feature flag and set different conditions. For when it's enabled. So here I've got my flag July sale, but I don't want this to go live on the site until after July 1st. So if I come to my my homepage here, you know, I'm not going to see that that. uh that banner. But let's say I want to do a little bit of testing and I want to say, now for the general public I don't want this to go live until July, but I'm doing some testing, so I want to be able to see it. So I can add a condition that says um You know, if if let's say the admin user is looking at the site, so now there's two conditions for everybody July 1st, but also it's also enabled if the admin user is looking

Speaker 1: Now, when I come to my site, I'll see this big salon bread batter in the top. So, and there's all sorts of different configurations you can do. You can say, you know, there's a bunch of predefined conditions which are built into the package and then you can also define your own. And then there's a bunch of other different helpers as far as you might want to not have the sale page be accessible until the flag is is on also stuff like that. So we use this a lot for doing uh sort of internal testing of features before before they go live. And uh We found it very useful. Hopefully others will as well. So that's Webtail flags. Okay, the next thing is

Speaker 1: um Second thing I want to talk about is called lag tilt inventory. So a bunch of people today have mentioned stream fields. We really love stream fields at CFPB. We use them a lot. Some would say we abuse them For those people who are not familiar with stream fields, what you can do with them, the basic idea is that you can define these custom block types. So you can take the different base kind of blocks like a text block or an image block and so on, and you can combine them together. together. So here 's an example from the bakery demo repo, a block quote. So you're going to have a quote and you want to be able to display the quote and also who is the person who you uh who said the quote who you would attribute to the

Speaker 1: quote to and then you also are defining um a custom template that when this block is rendered in my page what it's supposed to look like Another example is an image block. So again, I might want to have an image and the caption for the image and then the attribute. for for where that image comes from. And so by defining these as a block, I'm now making a single unit that an editor on my site can stick into a page. So let me show you what this looks like. Um okay so again here's the Wagjail inventory repo. And so here's my um Here's what the code looks like in the um the Bakwe demo code base.

Speaker 1: Here's my block quote, right? And so On my page, it's gonna look like this. So I've got my blog post, and now you know I want to insert this nice quote about how great vegetables are on my diet. So that's what what that block is going to look like. And then here's here's what my image block is going to look like. I've got my delicious waffles and then I've got my caption and then I want to say this is a creative Commons um uh uh image. So in the um In the editor, for those who are not familiar, it looks like this So here you can see

Speaker 1: this this whole highlighted part, this is one block that my Wagtail editor is going to see a block quote type and here they're going to enter my text and then my author. And then right below it here, you're going to see the image caption and attribution. And now this is all built into Wagtail. This is just custom blocks within stream fields. These are great. But now the problem that we ran into and what I think others may have run into if you're using a lot of these is it's very hard to do uh queries or lookups of where you're using blocks on your site and this is something that we've run into a lot which is especially if you have a lot of blocks and you have a lot of pages it's hard to figure out how many pages on my site Am I using this block or am I using this combination of blocks?

Speaker 1: And let me give you a quick example where you might run into that. So in this case, I've got my image block, and you notice that There's no little red stars here because the way the bakery demo is written, the caption and attribution of this block are not required fields. But what you'll see is if you look at a page like here where I haven't defined I haven't given it a title or attributions image. This doesn't look great. I just see this little dash here. It doesn't have a graceful fallback. So somebody that's doing maybe a code audit or somebody might notice business and say, hey, wait a minute. You know, there's a problem with with this. And now you might say, all right, well, how many pages do we have where this

Speaker 1: where this happens? And so you need a way to know, well, how many pages am I using this this image, custom image block that I've defined. And Wagtail doesn't give you the way to do that out of the box. So this is what we've we've built here in Wagtail Inventory. And so this is a new section here under settings. This is called block inventory. And what this lets you do is it basically lets you search and filter and find All the pages that include or exclude some combination of blocks. So the blocks that are listed here include both the core Waghale stream field blocks as well as the three that are defined in the base. the block quote, a heading block, and image block. And you can do something like say, okay, show me all of the

Speaker 1: pages on my site that have a block quote. In this case there's only one. Or show me all of the pages on my site that have an image. image block. Or you could say show me all of the pages that don't have an image block. Or you could do some combination. So this is really useful. We have a lot of a lot of cases where Where we're thinking about changing a block, like we might think to ourselves, oh, you know, why don't we put a border around our image block? And then we want to find all the pages on the site where that image block exists. And so this is very, very useful for um for for finding that. We have some ideas for how to make this even more useful. You potentially might want to find stream filled blocks that not only have or pages rather that not only have a block type but also have a block type with a specific content in it.

Speaker 1: There's also an open PR uh against Wagtail which would make some changes to the back end to make uh to make this what the way we're doing it now. A little bit easier. But if you're doing things with stream fields, if you have a lot of these kind of blocks, check this out. We'd love to know what you think about it. Alright, so that's Web Tail inventory. And then the last thing I want to last one I want to talk about is called wagtail sharing. And this is easier sharing of wagtail drafts. So a few people have alluded to this problem today. One issue with Wagtail right now is so these are the options for page visibility. So if you want to publish a page in Wagtail and share it with somebody else. else, your choices are you can either make it public, you can make it public and put it behind a password, you can make it viewable within the Wagtail admin by anybody that logs into the Wagtail

Speaker 1: admin or you can make it viewable within the admin and limit it to people within a group. We had um Uh we had a a a use case where we often have content that needs to be reviewed by a large or changing or indeterminate group of of people similar to the le you know we have guys a government agency we have a certain amount of legal oversight that has to happen of our content And so we need the ability to give in some draft content, send that out for review and approval. But we can't necessarily give everybody that's going to be reviewing the content a Wagtail login. Because there may be too many people. We don't want people to share login. They may not have access to the system.

Speaker 1: And so we wanted to build a way for people to be able to To basically preview draft content in Wagtail, and that's what Wagtail Sharing lets you do. So The idea behind web tail sharing is um it lets you do two things which I think are really useful. So one thing you can do is again, let's say I'm here at my bakery demo, and um I I'm thinking as the owner of this bakery that I want to add a new line to my bakery which is going to be gluten-free bread. And I want to make a new page on my site about this. And before publishing this, I want to send this out for review to people.

Speaker 1: But these are people that might not necessarily be able to log into my Wagtail site. So I need a way to basically expose this view draft functionality without um Without somebody being logged in. So what WagCalSharing lets you do is we have this concept of a sharing site, which again here is in the settings area Where you can basically say, given you know in this case the default Wagtail site, I want to expose this on a different host name. So here I've just used sharing. localhost, but you could imagine you know using some other subdomain of your site. And in this case we would be using some kind of network controls or so on to prevent, you know, to

Speaker 1: limit traffic to this domain only from. in our case sort of internal network. But what this lets you do is now if I come to um so here on my page, if I try to go to this page on the local host um You know, here you see localhost breads, gluten-free bread, the page is not going to be found. But now if I access it via the sharing site, sharing. localhosts, then um the page is is accessible. So it basically is letting kind of giving sort of another route to let you preview the draft content even though um you may not be logged into the Wagtail ad Another thing you can do with this functionality is

Speaker 1: you can preview draft content on a page. So let's say I want to edit my web, you know, my home page And I want to, you know, change this, uh change this text to say, you know, welcome to, you know, let's say My until space. Right, so I'm saving draft. So now somebody that's coming to my Sharing site will see the draft content, whereas people that are coming to the main site will still see the regular content. So we found this very useful as a way to um to preview draft content. And uh another thing which um Tom

Speaker 1: actually uh uh kind of motivated us to do with this project is um we have hooks now so when you're previewing the content um you can kind of add arbitrary hooks to that so this is sort of Another example where when you're previewing the content, you might want to give the people some kind of markup tools. So when they're reviewing a draft, they can do markups. So this is all also sort of built in on the uh kind of the ability to add this to the to the Wagtail sharing view. Okay, so that's Wagtail sharing. So those are our three three packages. If anybody would love to look you know would love any feedback if people want to look at them. So now I want to talk about um contributing your own packages. Why this

Speaker 1: uh So why is this a good idea? Why would you maybe want to do this? So one um one potential reason is because it helps you to define the interface to your code. So a lot of times what'll happen when we encounter a limitation in Wagtail is you s you just start hacking, you just start thinking about how you can modify your code and hook into it. And so if you think about it as a separate package, And you know, it kind of makes you be a little bit more

Speaker 1: disciplined about defining what the interface is going to be, what are the limitations, how much do you want this to do, how do you want it to work. And that can be that can be very useful. It can also help with testing because if you've got a you know kind of a s a strict set of functionality that your package is gonna be be doing, it can help you sort of define tests against that in isolation maybe from From the rest of your website. Especially it's a little bit easier to test the an isolated set of functionality against multiple versions of Wagtail. That's something that we've done a lot. And then of course also sharing with and getting feedback um from the community. So um of course all this easier said than done. So how do you actually do this? Main thing I want to suggest

Speaker 1: Is is copying or getting inspiration from other existing packages. So when we first started doing this, I didn't have a lot of experience at all with publisher. Python packages or knowing how to do that and just looking at other examples that are out there can be really useful. So just last thing I'll show really quick. So here's here's an example of a gray package. the Wagfell two-factor authentication. This is back on the PyPI, the Python package index site. And you know you can read and say, oh this, you know, this This looks like a great package and I know if I type pip install wag til2fa this will get installed, but if I want to make my own package. You know, how how do I know how to do this? So when code is open source like this, you know, you can go to the repo, like in this case, this is the setup.

Speaker 1: py file and just look and see you know how how other people are doing that. Look at our code, look at other packages that are out there. And this can be really useful. Just copy, you know, just just play around. The documentation for Python packaging is really quite um you know not not that that hard to understand and um we definitely have learned a lot just by looking out there at at others um and trying to um you know look at their documentation, look at their examples can be can be really um really useful. And another thing to think about is of course um you know write good documentation if you can to give people um instructions on how to use your package. Screenshots are definitely always appreciated, especially for changes that affect the UI to give somebody an idea for

Speaker 1: um you know what it's gonna look like and then um you know other things to think about are just uh you know choose a good license for your uh for your package think about you know releasing it open source if you can And then finally, of course, submit a PR to the awesome webcale so everybody can use your package and give feedback and we can all make each other's code better So that's it. Thank you very much, and please check out our stuff and give us issues and pull requests if you want.

Speaker 2: Um these look like some really great packages. I'm excited to know about them. definitely solve some problems that I've been experiencing. I'm curious if Wagtail Flag supports Django applications without Wagtail.

Speaker 1: Yeah, that's a great question. We were just talking about it and we've actually had a somebody found an issue against the repo. asking that. We would like to make some changes to that repo, either to make it able where you can install it with WagPill as an optional dependency or potentially splitting out the the kind of more pipe on Django pieces from the Waghill specific.

Speaker 3: Yeah

Speaker 4: Uh also about pactail flags. That requires a constant schema. You're gonna roll out code for a feature, the schema with the feet flag. enabled and without it have to be the same.

Speaker 1: You're talking about the model schema?

Speaker 4: Yes.

Speaker 1: Yes. Yes.

Speaker 4: The model schema.

Speaker 1: Yes. So that's that's a good question. Um Right, so typically typically a good pattern or one that we followed is to try to make your changes additive. So if you've got a model try to add the new field, let's say, add the code behind the flag that refers to the new field, then wait until until you've switched over to use the new field before you remove the old field. Yes, you do have to kind of make sure everything is there to support both the on and off version of the code at the same time.

Speaker 4: What do you do you ever sort of unflag a feature and return it to the mainline?

Speaker 1: Return it to

Speaker 4: A feature is enabled by a flag in this workflow. Do you ever sort of say we're going to promote this feature into not being controlled by a file.

Speaker 1: Yes, typically that's what we'll do is we'll add we'll add the feature, we'll add the code for the feature, then we'll we'll turn it on in production and then we'll have a cleanup PR that goes in and removes is the reference to the flag, just to keep it from hanging around forever. Right. Thanks. I

Speaker 5: like uh Andy was our last