Wagtail as a headless CMS for JavaScript frontends - Tommaso Amici

Okay.

Speaker 1: No, no, it's just presentation.

Speaker 2: All right, great. So my talk is going to be about how to use Wagtail as a headless CMS. For JavaScript performance tense, because that's what I'm mostly interested in, but it works just as well if you have an iOS app or Android or any other need that's not in the browser. So last year at Ripencc we rewrote a site that was implone to Agtail, and we did it with Django templates, which are great. And this year we're in the process of rewriting our main site from Plone. And I propose why don't we use a JavaScript front end? Because personally I like working with them. And so, you know,

Speaker 2: we investigated. This is the result of our investigation. Eventually we decided not to go with it, but that's more down to team composition. So why would you want to use the JavaScript frontend? To me, the most important thing is It comes down to developer experience. So it's about making smaller components that can be unitested and linters that actually work. uh and are reliable. I know there's a an experimental curly lint for Django templates, but I tried it on our code basis and it still has a few false positives. If you throw in TypeScript, you get type checking, which is great.

Speaker 2: These sort of things are just not there in a template in languages as of now. Maybe you know, two years, three years, it'll be much better. It's also good if you want to make a clear front-end, back-end division, if that's how your team operates, if that's how it may work better for you. This is not the case for everyone, I know. So for those who may want to stick with Django templates, why would you? Well, you already know them. There's one fewer moving piece, and it works out of the box in Wagtail. So you have to think of nothing different. Speaking of um the WagTeal API, um it's good, it's mostly good, you don't have to reinvent the wheel, but it still assumes HTML first.

Speaker 2: And it's not a one-to-one mapping of your front end. For example, you can see there that I'm looking at page with ID5. But if you look at the URL, it's deep nested in the tree of my site. So that sort of thing is a bit lost when you deal just with IDs. So what I'm looking at is something like this instead, where I can just have a one-to-one mapping of my front end with all my paths to the back end. In this case, for example, specifying a content type header, I get the JSON back instead of the HTML page. There are different approaches to routing. One is the one I just showed with content type. One could be to prefix all your calls with an API, uh, well, slash API

Speaker 2: slash. or with a JSON suffix. I still don't know which one of these would be best in practice. You could also have a separate subdomain with your CMS. At the end of the day, this really depends on your setup and needs. What I do in this case is I have a front-end and a back-end upstream in Nginx. and based on content type uh I route differently. Now let's look at Wagtail. So if you look at the uh page uh model in Wagtail Core you can see that the um the return of serve is a template response. So while Wagtail has an API, it's still

Speaker 2: not headless first. It still expects templates. And if you hit it and the template doesn't exist, you get a 500. So what I'm looking at instead is a base page that looks more like this, where the serve method uh is overridden to return JSON. You can see there there's a serialized page. That method doesn't exist on pay on bait on page, but we'll we'll make it Um so serializing, as you know or may not know, is the process of turning uh Python uh data into JSON and that can be sent uh in your responses. So let's look at it a bit closer.

Speaker 2: This is how the serialized page that I would use looks. So it checks if your serializer exists. These are all based on Django REST framework. It comes with Wagtail. So it's technically it's an extra dependency, but it's already in there. And then we'll return the type of the page, which is basically the name of the model. That's going to be useful on your front end if you want to say, okay, I have a layout for articles, I have a different layout for articles. um I don't know if people pages if you have employees and they have their own page or something like that. And then in data we use the serializer we we will define to serialize our model

Speaker 2: Additionally, I've also used the serializer for context because Wagtail mixes uh If you use the model a little bit, it blurs the line. So if you use context, you'll just make an extra serializer and it'll work just as well. So let's look at uh the base serializer. Oh, one thing to note is that base abstract models cannot be serialized in Django REST framework. So keep that in mind. Nonetheless, we'll make a base page serializer. And we'll reuse stuff from Wagtail, because why not? So the stream field in particular is a bit tricky. So we'll import the serializers from Wagtail. The rest you can see down at the bottom.

Speaker 2: So ID, slug, title, search description All the basic types are handled automatically, so numbers, strings, that's not a problem. Okay, so with all that said, let's make a real uh example model. It's going to be an article page with all the defaults from page and the rich text block in a stream field. Of course you can expand that and we will do that later. Our serializer is very simple. It extends the base page serializer and it just adds body to the fields. Here's our uh example, it's got a heading, it's got a picture, uh, nothing

Speaker 2: more. And if we send a request, uh, it looks all right. We have our type article page, we have data, it all looks good, except if you're careful, you'll see Something weird in there. And that's because as you know, rich text is um stored uh it's not stored as uh expanded so a link will be an anchor tag with different attributes and then with um different functions black till expands it later So what did we do? We used a serializer the Wagtails provide. We didn't want to reinvent the wheel and use it.

Speaker 2: But now our links and images are not expanded So we're back to the drawing board. Let's look at rich text. So we just want to make sure it's expanded and links are links and images are images instead of the internal representation in database. So we'll use a custom rich text block for blocks in the stream field or a custom rich text field in a serializer for models with rich text fields instead And you just want to call the expand dbhtml function on the value of string field. Note that in Django templates, this all happens when you pass through the rich text filter. So now we apply those changes,

Speaker 2: we make a request, and here is our image. It's not weird anymore, and it will render fine in the browser. But now that we're on we're on that topic, let's talk about images because they are very important and they'll be there whether you like that or not. I like images a lot. In 2022, this is what an image may look like in the source of a page. So it's gonna be a lot more complicated than you used to. And it's got modern formats. if the device supports it. I know Avif is not in Wagtail, but uh you never know, it may be at some point. Uh WebP is in there. Then you will have lots of sizes for different device pixel ratios

Speaker 2: or simply responsive layouts. If you have, I don't know, an article card and on a mobile, it takes up the whole width of the screen and on desktop we'll just take a small square. So you it's good to have different uh widths uh and different sizes in there. Also, we'll have a fallback to common formats. So what we did previously in here is not enough in my opinion. This is in rich text. So let's see how to improve that. Let's add an image to our model. Here it is both in the stream field and as a key. sorry as a field on the model itself. We extend the fields on the serializer

Speaker 2: and then we test it And uh you know we get the ID of the image, we don't get anything else. So we need to write a serializer for this as well. On this note , there are some consideration you may want to do. So one is If you have the means to do it, or you know, you go the cloudinary way where you have dynamic URLs. The problem with this is if you take width as a parameter in your URL, it takes uh, you know, in an ideal world, that's great. It's simple for uh your developers it's simple for everyone to use however as soon as uh

Speaker 2: someone decides to be malignant they can just run a for loop and fill your hard drive with images that go from with one to with a million and then that's a problem. So what you could do is I suppose have extra validation on the route. So you only have Some selected number of options. If you put in with 199 , it will throw an error or you go down that road. You can have signature checking like serve view from Wagtail. The problem I find with signature checking is that you still cannot use it on the client really because you need to have access to the key to generate the signature. So you I think you're left with predefined renditions

Speaker 2: , which is okay. In a template you would do it like this. So you define them and you just use image URL with whatever filters you want. And similarly in Python you would have an image rendition field. In this case, what happens is I passed a filter spec, which is an array of sizes I want. And then in the represent to representation method, I generate the source set and the webp source set as well. What happens, this is how I use them. So I go back to my serializer, I add fit image as an image rendition field. I pass in whatever values I need that will depend on your front

Speaker 2: end. So uh You will know that. And now if I make a request, I'll get source sets back instead of just the uh either one image or the ID. Something that's very important for editors is of course preview. And there's a Wagtail headless preview package from Torchbox. As you can see, it's very simple to add to your models. In this case, because we have a base page everything inherits from, we can just add it there. Then there are some extra settings to add to your settings. py and some extra views in your router API router, but it's it's pretty simple.

Speaker 2: And then when it comes to the front end For example, in Next. js , you would fetch all your paths from the API. In this case, this is just from Wagtail API. There's nothing custom in there. You will then make a page for every path. So as you can see here, it fetches that URL, it gets back all the JSON , which is then passed to a page proxy. Which is basically a router for uh templates, let's say. Um, as you can see here, it checks the type uh of model. And then it goes and sees, all right, do I have that type in my lazy

Speaker 2: pages and then imports a component, which is then used to render the whole page. So in conclusion, I think headless wactile is possible as it was just shown by Patrick earlier. You may actually want to go down the GraphQL road one day if that becomes viable. I think to operate a head headless wagtail, you need Jason as a first-class citizen. You would not be interested in templates or in your models serving template responses. So I'd say do away with them and have a one-to-one map of front-end path to pages. You will need a few custom serializers for rich text for images. or for any

Speaker 2: other block you make that is not uh made of simple types uh like strings or numbers. And with that , I'll leave some space for questions. You can find me on my personal website There's a repo that contains the source code of what I've shown here, still not updated with the latest changes. I'll push soon. There's also a blog post which I'll update that uh it's in written form if that's how you prefer to check this.

Speaker 1: Thank you so much. Thanks. And then just a shout-out that there is uh we had our last two talks talking about headless wagtail. It's very big among our users. There is a headless wagtail channel in our Slack make sure you view there for other ways of um headless wagtail um and we have a few minutes um for questions um before we break for lunch um so let's start with uh in the room and and we'll also check to see if we have any in zoom or in the channel. Yeah, why am I here? Sure. And can you repeat the last part? Can you explain the

Speaker 3: whole last slide?

Speaker 1: Cool. Can you explain more about page proxy on one of your last slides? And maybe go back to that slide if possible.

Speaker 2: Yeah, definitely. Are you still seeing it?

Speaker 1: Yes, we're seeing your screen.

Speaker 2: Okay. So what happens here is um let's go back one uh slide actually. Sorry, are you seeing the notes? Because I see the weird zoom green border.

Speaker 1: We can see all of what you need.

Speaker 2: Okay, yeah, that's okay. So in here you can see It fetches the page. And if we go back a little bit, I had this type article page alongside the data properly that's attached to it Using that type, what we can do is um because our path our dynamic route is uh catching everything, uh it's not yet uh The division is not yet made. So if you need custom templates, you can go two ways. One is say, I know all my, I don't know, slash shopping slash product ID pages will have the same template.

Speaker 2: That's one way to do it. In this case what we're doing is catching everything under the same pay uh yeah page let's say um and then based on the type that comes from the request It's getting the component to render. I don't know if that made sense or if I explained it properly. But the idea is that instead of making uh the routing with file system uh routing like so many uh frontends do we do it in a dynamic way based on the type we get back from the response

Speaker 1: Thank you so much. We have a good bit of time if we want to take more questions or go a little bit deeper into some of these explanations. So if if there are more questions in the room, yeah, great.

Speaker 4: Um so the the YTL API has a an endpoint to find the page based on the path.

Speaker 1: So the Wagtail API has an endpoint to find the page based on the path.

Speaker 4: Uh why Do you or are there shortcomings with that that lead you to do the the approach that you outlined in the first part of your talk instead?

Speaker 1: Are there shortcomings to that approach that uh led you to do things the way you did in the first half of your talk?

Speaker 2: Yes, um so I'll I'll go back there so it's on screen. The diff the main difference between this approach and the Wagtel API approach is that You will have the same path on the front end and on the back end. And the Wacktal API instead requires you to do some querying with query parameters. I find it's not um as clear as having basically two layers and one is JSON and one is the rendering. So obviously the Rackle API exists and it works. So I'm not saying it doesn't. But if you if you think

Speaker 2: um in terms of headless uh and you can actually make the API first class citizen um and get rid of templating entirely um They still coexist so they live in the same um system. I like it better this way. I I guess it's done to preference. So there 's no real right or wrong

Speaker 1: Awesome. And can someone help me with um any Zoom questions if there are any? Yeah. And if not, we could take some more from the room.

Speaker 2: Uh one question from the chat was the page prox is basically the switch for the page components, and the answer is yes. And another question was, I like the one-to-one map to front-end path idea. I wonder if we could implement this in Wagtail by checking if the request contains the header. accept application JSON and call in a surf JSON method if that's specified. I can't answer to that, but I'll repeat it for everyone in the audience.

Speaker 1: Awesome. Thank you. And we do have another in the in the room.

Speaker 5: So specific technology group. Now all of them really want to do the app at the front end. That's kind of speed as uh setting up of like a model.

Speaker 1: Sure, we have a Wagtailer who works in the civic uh tech realm and a lot of them want to use React on the front end and Wagtail on the back end How much coordination does it take for the teams to uh for the wagtail on the back end and react on the front end? And did I did I get all the points Cool. Yeah, that's the question.

Speaker 2: Yeah, so uh this setup is uh it would work very well for that. Um The only thing you really need coordination on is images, as I said, because the rest, there's really not much of a discussion, sorry. You can, you know, if you have uh a title, a slug, you know, any anything else, you'll just show it. Images are a bit tricky because you will have to coordinate with the front-end people to see what sizes they need and pre-define them up front unless you're willing to go down the a variable uh dynamic URL uh route. Um so that that's the only pain point I can see if you need to coordinate.

Speaker 2: As far as the rest goes, you know, you expose the API. Most of the fields don't really need discussion over them. And then of course you need to coordinate if there are changes. But aside from that, I think it would work fairly well to split the work and have front-end only and back-end only if you want.

Speaker 1: Awesome. Do we have any more questions in the chat? Any questions in the room? All right.