Headless Wagtail Strategies

Yep, here we go. All right. Hello everyone. Thanks so much for coming. So I'm going to talk about headless workl. That is I guess a hot topic. and uh that has um probably different solutions so I'm going to to talk about which one we we retained. Um so I'm Remy, I'm uh co-founder at uh WIT, we are a web agency in uh Madrid. We have about 40 people and uh we work for uh lots of uh brands to create uh like kind of public-facing websites. that are going to be um having you know some deep integration to their system but usually the the content uh is a dependency of what we do so we need um let's say a go-to

solution to to be able to have content management for those websites And obviously that is uh Wagtail. Um so first of all, I'm I'm going to open uh already open doors, right? But uh what is headless, just to be sure on the definition. The idea is that you're going to put in between your browser and your work tail a front-end framework like Nuxt, React, whatever. um and uh the CMS becomes a repository for the content uh and then it's available through some kind of a API and then it's rendered by some uh other framework for uh that's more dedicated to frontend. And it's definitely something um hot, you know, in the web development. Lots of customers ask is uh from us.

But um is it really a good idea? Um like Uh because I I know I I've had the debate many times with people that are like um uh come on, it's uh this JavaScript thing is it's not for me, I don't like it. And that's perfectly understandable, but um Let's say my perspective is that uh I have um an agency to run, right? So if you look at the the cost of different things uh for me It's uh switching people from project to project, uh configuring different infrastructure for different projects. I have many, many projects to run at the same time, right? And so the JavaScript bundle bundle size is uh cost zero in comparison to everything else So if um using a front-end framework uh solves you know um the ability to to move on different things it's

uh uh it's pretty important. So It's important for me that sales project managers, UX designers and so on, they know what to do and what to offer to the customers. They also, you know, know how uh the pricing works. Um and uh obviously the the price for this kind of stuff is uh unfortunately not uh the same you know added value as it was ten years ago so we have lots of juniors working on this we don't need to be we need to be able to get them uh working uh without training them for uh too long And uh of course we need to consider inside the agency that the talent pool is uniform. So if we have many uh different uh people uh frameworks. The people we need to train them from thing to thing and we cannot switch, you know, like if there's an origin

bunk, not everybody can jump on the project and uh and save it. Um and of course the customer is always right in matters of uh features. Like if they ask for something, uh we say yes because otherwise they go somewhere else, right? But uh project might start simple and uh ten years later you end up with a huge pile of uh different features that you did not anticipate and that can blow up. So If you have a stack that does not allow this, you will end up uh let's say stuck uh with two choices. Either you uh decide to go with uh some hack uh like band the stack over and uh make it work somehow but then you create technical depth uh very quickly

Or um you need to rewrite everything to another stack that uh works for you. So bottom line is um I'm really looking for a full stack that will allow me to do everything that the customers always ask me to do and uh that is for the back end uh wagtail because uh few reasons uh that you probably all uh know but the Django models. And the migration system is amazing and it allows me to have uh you know a lot of guarantees on my data, the stream fields that I use a lot, probably a lot more than intended Um the the ability to fall back to Django for uh anything that Rectel does not support, so that is you know great because it's not uh uh WordPress that exists just to create uh content. It's a

full uh framework behind. Um it is secure and it is defendable. We have lots of security audits. Uh it's very easy to say yeah we use Wagtail, it's a secure solution. uh state of the art, blah blah blah. And of course the talent pool you can uh find people that are able to work in Wagtail or at least in Django and upgrade them very quic quickly to Wagtail And uh again in front of customers it's uh very important. Uh why uh SSR framework um like next in our case? Um because the it's the same template in the front end and the back end. So you get like classical type SEO. and performance because the content is already generated as opposed to SPA where you need to download the JavaScript and then generate the content.

Um you can have any kind of interactivity you want. So if the customer goes, ah yeah sure Uh now can we turn this into a Facebook like comment section where you can like the things and so on? It's very annoying to be doing the template in jQuery and uh mimic it's the template from the Django and so on. So It's very important to have the same on both sides. Comes at the cost of uh smashing the CPU and the network a little bit, but uh it's perfectly acceptable And also it solves some hard work problems that are um the organization of the code, um, which uh I guess is a matter of uh taste, but uh single file components in um in front end from ox basically you put javascript html and css in the same file

or together and basically it means that if you delete this component, you delete all the dependencies of this component I mean you delete uh all the content of this component from one file. You don't have to go find it in your view file, in your template file, and so on and so on. And vice versa, when you you dynamically include the dependencies. So uh the compilers are able to make the tree and make bundles that are optimized for uh exactly what you are going to display. So if you have a homepage you're going to uh statically um to load uh you know just the content of this homepage and then when you have a mapping component that is uh taking uh one megabyte of JavaScript. uh then it will only be loaded when you go on the map page and not uh in the whole javascript bundle.

And doing this with uh just Wagtail is uh is a bit cumbersome. I mean it's um it's more you know uh bring your own solution to this usually. So what's the catch? Well the catch is First thing is uh routing because voices routing is kind of hard-coded. Um this is problematic for uh regular pages, but m most importantly is very problematic for previews uh because the rules for previews are a bit weird and they are also session based uh so that is um not intuitive how to make it work And overall Wagtail is already messing a little bit with Django's routing, so adding some more uh layers on top of that is uh not a great idea.

Um I am not using Wagtail Headless Preview, that is a package that is uh made for this. Uh for several reasons. Uh you'll see that uh basically the solution that I I offer that does not um need it. uh because it is uh token based uh instead instead of session based so you need more awareness uh in the in your code you need to pass the token around and you need like all the parents of the code need to to know what they are doing and I wanted to avoid that. The second big problem that I have is uh the images, okay, because the front end must take image decisions because it's the only one that knows uh the size of the image, how it wants to display it uh what are the different formats and variants of this image. You want to apply filters and so forth and so forth.

And So that really depends where the image is displayed, you don't know in advance where this image is going to to be, so you cannot pre compute the formats and so on But on the other hand, the front end cannot take image uh image decisions because of the security risk. If someone asks for the same image of uh one pixel, two pixel, three pixel, and so on. you can very very easily uh smash down the server uh by asking a million random uh formats. Uh so and there's two solutions for this but uh it's it's always generated by the server right you can either uh do the render from the server and uh then give the url or use the dynamic uh render url but it's it's the same issue. So uh I scratched my head a little bit when I had to make my first um you know

headless project and figured this. So for context it was uh conversion uh of uh classical work tail project into um headless one. So in the end you you need to consider that the front end is a proxy, right? Like um you can use um to give an example you can use Cloudflare to compress your images. So all the JPEG images are going to be compressed into WebP um dynamically by Cloudflare. So the same URL uh is just content transformation Uh so why not do the same thing with the with the front end, right? So the browser will uh ask for this page to next. Uh uh hello can I get slash foo And then Nux will ask from Wagtail can I get slash foo?

Then Wagtail returns the content and Nuxt uh transforms this content into HTML. uh that is uh properly um you know um presented with uh CSS and so forth. So that that's kind of the mentality that allows you to also solve Um things like okay if there's a redirection on this URL then you you proxy it. So you like you reply what the server is saying. If the server says uh Redirect you have a direct if it's an error, it's an error, etcetera etcetera. So it's making it uh a lot more simple to see it as just a proxy So like basically you make a catch all a route on your uh front end framework and everything that is asked you proxy through Um the the other advantage is that when you do this, you

are on the same domain for Django and for um the next. So the cookies they go through. um and there's never need for calls you can call the API as much as you want. There's only one uh trip to the API in a two, etc. etc. So it's really simplifying things a lot. So basically you just uh say everything that is uh I don't know slash API goes to the API and everything else goes to the front and that's it. Uh previews. are a little bit tricky but um not that hard in the end. Um so how does it work? Like basically when you make an edit in a Wagtail it's going to first make a post to the preview URL to send the new content. And then uh Wagtail is going to store it into the session and then is

going to make a get, like the you know the live preview on the site, it's going to make a get. on the page to display the preview. So it's pretty easy in the end. With um like for example I use a HTTP proxy middleware that I put in the VID server that is you know rendering um next and so forth. I have a regular expression to catch preview pages and if it's a post I send it directly to Wagtail and otherwise I put it through the rendering process. So that is actually relatively easy to do and with that you get the session-based previews. They are rendered with exactly the same other code as the rest of the website and basically Wagdale and Nuxta are both oblivious

to this except for a few lines of code here and there Where I lied a little bit is that uh the content is uh HTML, right, in a previous example. So I have different strategies, but that's the first one. Um you're going to say it's a bit wet, but um bear with me. So the idea is that as I said, the first project that we did was uh a migration So um we already had the templates. We wanted to just redesign things a little bit uh and make it uh make it server uh headless. Um so what we do is that we do uh single file components but without the template and the template comes from the webtail page. So basically instead of serving JSON

uh from the API Um we serve HTML from the API that is the regular template. Actually we we did not even modify the Wagtail uh source code for this thing. We just made uh JavaScript that was uh able to um you know pass this HTML and inject it as a template into the view uh single file component and like generate dynamically the components with the template from the server and um and uh javascript from uh compile uh compile time. And that is that gets rendered into DOM. So basically it's uh it's kind of a hack Uh some would uh call me uh um an abomination, but uh pros and cons, right? Pros

it works. Uh so As long as it works, right? It is workable in the sense that uh it's not creating uh tremendous amounts of technical depth. Uh you can uh you can deal with it, it's not uh blowing up over time. uh it's it scales uh without uh being annoying. Uh I mean without getting more annoying than it is initially. Um it's lowering kind of the cost of converting an existing project into a next project. And you can still get the core web vitals passing, etc. etc. So that's really not that bad. It's kind of um trade-off uh that sounds a bit uh demonic, but it's it's okay. Uh cons though is that it's really getting hard on the CPU because you need to compile all those templates at uh load time.

So that's uh kind of annoying. It's not extremely uh future proof either because uh it's kind of hacky, you know, so it's really on the edge. Uh and the last thing is that You need a runtime compiler, so typically for Vue it works, but for Svelte or React I guess uh not because uh they are compiled at uh compile time and uh Vue is uh you know initially uh parsing HTML itself So that's a way to go, but that's not the definitive way to go, I would say. So another um strategy that we are developing is um to serialize the pages into JSON using the serializer, the page serializer from Rugtail that is pretty easy to use. Um we override the serve method to return JSON instead of HTML.

So we like we just call the stylizer. render the page according to this JSON and put the right components and so on. The only issue is the the images, right? So how do we solve the images Uh because we still don't know the formats in advance. So by thinking about it a little bit more, um turns out that uh you need about six, seven uh formats that you can statically define. Because these days it's not, you know, um like there's all the screen densities, uh screen width, and so on and so on. And so basically at the end of the day you you cannot predict the formats that you're going to need anyway. So you kind of need to take a probabilistic approach to say, okay, if I have seven sizes, is

going to be uh there's always going to be one size that is uh ten percent away from uh the size I really need and that's going to be good enough. And you could think also that is preventing you from uh cropping and uh zooming and so on. But on the other hand it's uh an opportunity to do it in the front end. So instead of having uh the cropping done statically in the back end, you can have the cropping done uh in a responsive way. Uh so when you rescale, you know the the page is still cropped according to the feature from the WagClear thing. So that will require to develop a specific component, but it's also a pretty good opportunity to like improve the uh image management. Um so as a conclusion the front end is a proxy

That's really the core idea behind this approach is to consider that it's just a pass-through with some modifications on the content. So we we kind of give style to the worktail content There's two approaches to this. There's the server templated component approach that is kind of a hack but definitely works. So you send HTML, you transform the HTML into better HTML, let's say with JavaScript and CSS. And there's the API-based approach that is overriding the serve method and uh that uses a fixed set of images uh to to do the rendering. This whole thing is open source, so um the documentation is uh not uh excellent probably because it's an entire project, but we're trying to improve it at every release.

We have uh what we call a model W that is um you know the same as model T you can use any uh framework as you want as long as it's the same as in the uh model W. The idea is that we use this internally at the NGC because everybody can uh knows it and we have tooling around this. So basically you can check it out on GitHub and try it. And you can have the project maker. So that with this one liner it starts an assistant to basically create a project uh that is already uh scaffolded with a next and uh wagtail and that allows you to have like everything that I just explained with the server templated components That is already running, so you just need to start coding basically like all the modifications have been

have been made. Um and uh the Zvel -Kit version that is going to be um uh dealing with JSON instead of uh HTML. Uh we're going to release it in October. Um so it will you know be released there. So we like we release every four months. So it's not going to make the cut for July but it's definitely going to make the cut for um for October. So thank you very much. I don't know if uh anybody has uh questions about this. I think we have um thirty seconds more.