Wagtail as a Platform - Matt Westcott
Published March 30, 2022
This video features Matt Westcott at Wagtail Space US 2020 in Online.
Matt Westcott argues that Wagtail should extend Django rather than recreate or replace it. He uses the “inner platform effect” to explain problems such as overly rigid edit handlers, the all-or-nothing Wagtail Images app, and excessive reliance on hooks, while advocating layers that let developers drop down to lower-level Django APIs for unusual cases. He applies “Chesterton’s fence” to decisions such as Wagtail’s page model, StreamField migrations, scheduled publishing, and a possible React-based admin, urging the team to understand what existing designs provide before replacing them and to establish solid client-server contracts before adopting new frontend technology.
Summarised automatically from the transcript.
Automatically transcribed, so expect mistakes in names and technical terms.
Speaker 1: Hello, I'm I'm Matt. I'm one of the uh the core and original developers of Wagtail. You might have seen me around on GitHub as uh as as Gasman. Uh and uh two years ago at uh Wagtail Space in Arnhem, I gave a talk about my experience of opening up the uh development of Wagtail uh learning to let go of the ownership of the code once it was clear that it had grown to a bigger project than just one person. And today I really feel that we're on the brink of the next phase of that growth where I'm not just letting go of the code, but the decision making too. Oops, nope. And uh that progress that process has uh already begun with features like workflow in uh Wagtail 2.
Speaker 1: 10 as it's the uh upcoming release, which is a really major piece of development from Jacob and Carl that I wasn't really involved in their design of at all. And that's going to continue with features like Carl's upcoming work on multi-language support. And it does scare me a bit. I think I have a good feel for what decisions are and aren't a good thing for Wagtail, but uh gut feelings don't scale to multiple people. So in this talk I really wanted to share some of the thinking bet behind past design decisions in Wagtail. some of them good, some of them not so good, to see what lessons we can take for those of you who who'll be picking up that baton in future. And the story begins uh further back than you might expect, in the early two thousands, when a trendy young agency called Torchbox had a proprietary CMS like every other trendy young
Speaker 1: agency. Ours was called Rational Media and was written in ColdFusion. Ask your grandparents. And Rational Media was powered by XML because XML was the cool technology of the time. It was basically the the blockchain of its day. And What XML gave us was not being tied to a fixed database schema. So by storing the page content in this article XML structure structure and yes it was called article XML not just article to really drive home the point that yes this is XML anything that you could write in An XML tag could be your page content. And to define what kinds of tags were available to write in this document, we had another XML document, which was managed through this
Speaker 1: pointy clicky interface that we were really proud of as it meant that our clients could define new page types without writing code. And this All works really well until they wanted say a listing of events ordered by date, at which point we went, oh hang on, the the date that we want is sort of buried inside this uh XML string so we can't just do a SQL query sourcing on that field. So we had to come up with our own ways of efficiently querying XML and I was the ambitious new graduate at the time who thought he knew everything, who'd been brought in to work on this project. And I was perpetually thinking, oh I wouldn't have designed it like that. Because I figured you don't really need XML for this. Okay, you couldn't have a fixed database schema because we needed this pointy-clicky interface for people
Speaker 1: to define their own page types, but you could still build this in Pure SQL. You'd have a table of templates, which are page types, event page, news page, product page, and for each of those you have a definition of the objects objects that are available on the page. So for the event page that will be the date, location, body text. And now that we've got those definitions in place we can start specifying pages that use those templates. So Our Christmas party page would have a pointer to the event page template. And finally the actual page content would be in its own table a sort of key value store so that you could say The date field for the Christmas party page is 10th of December. And yeah, this was maybe a bit more abstract than just having individual tables for events
Speaker 1: pages and use pages, product pages, blog pages, but this way nothing was hard coded. The concept of these pages was itself defined within the database and we could have our pointy click click interface for defining them. Perfect, I thought. And it turned out I wasn't the only person at the company with uh ambitions to reinvent our CMS. A certain Andrew Godwin who was uh trying to get us interested in this uh newfangled thing called Django was uh showing us how the concepts of a CMS like this would translate to Django models. So you'd uh define a model called event and have fields for title, date, location. And I went, whoa, whoa, hang on. How are you going to implement the pointy clicky interface? And he answered, well, well do you need that?
Speaker 1: With those four words, do you need that? He shattered my whole world view. I just not questioned up to that point that yeah, this this pointy clicky template editor was the added value of a CMS. This was what made it a content managed website rather than a website with a database attached. But then when I thought about how many times a client had ever taken advantage of this feature to define their own page types in the history of rational media, it was uh big fat zero. Of course it was. They they'd hired us as an agency to build this site. The the it w it was our job and honestly that was probably a good thing because if One day a client had gone, you know what? I fancy just finding some pet templates today. Our reaction would have been, oh, oh, oh,
Speaker 1: are you sure? On a live site, uh may maybe you should let us do it. So if this wasn't built for the client's benefit, was it for our benefit? Well no. We would have much preferred to write our page develop our page definitions in code in our comfortable text editors where we had copy and paste and version control and the ability to deploy your work to production just by uploading a file rather than having to log into the pointy clicky interface and have to re-enter all of the definitions that you you'd previously entered in your development environment. So it was it was rubbish really. We'd we'd well and truly fallen for the uh inner platform effect. which is the tendency of software architects to create a system so customizable as to become a poor replica of the software development platform that you're using.
Speaker 1: I I do find that yeah being able to what attach a name to concepts like this, once you've experienced this and oh yes, that's what this thing is called, it really clarifies your thinking. And yeah, I I know there's a risk of alienating new developers who have to get up to speed on all of this cultural jargon as well as the technical stuff. So have to learn about yak shaving, bike shedding, rubber ducks and little bobby tables. But I think it it does serve as a real useful mental shorthand, uh so that uh you can say, Oh, this this problem that we're up against right now, this this seems like the the inner p inner platform effect, rather than saying You know, this problem feels a bit like that time when we reinvented the concept of a database inside a database. And as as developers, it's seductive to want to build these big new systems that automate and abstract away all the boring work.
Speaker 1: I don't want to write database schemas for 20-page type. Let's make a system for that. So you forge ahead with this new system, carving a new path and inventing a new better way to do the boring repetitive thing. And maybe along the way it'll encounter problems that people have solved in different guises in the past, but because this is your system, you'll solve it in your way. XML, blockchain. And you build this system, you put it in production for the first time, and you immediately run into the one special case that doesn't fit your design assumptions that you'd built this system uh yeah i it it with. This it's like the the square peg in in the round hole. Oh, I didn't think of that.
Speaker 1: And when you're faced with this, where do you go from there? Well you have two major options. You can hastily hack in support for that special case into this system. Or you can say, well, that's out of scope for this system. You're just going to have to revert to the old way and roll your own solution, write a SQL query, hand code some HTML, do whatever the platform you're building on. dictates that you need to do. That's if you're lucky and left that escape route open. If you're unlucky, you fell into the trap of the inner platform Effect. You built a database inside a database or a framework inside a framework, and now you're stuck with having to solve that problem within the context of that framework you've just built.
Speaker 1: And This to me is the acid test of any framework that's going to offer a new way of working. Is this a platform that's just built on top of the old platform that's replacing the functionality of that platform? Or does it actually extend the old platform and coexist alongside it to provide new features of its own while still leaving the old feature set available? And there's uh there's a catchphrase I've seen uh co news a lot when on the uh support channels when uh people ask how to do say newsletter sign-ups or e-commerce sites in Wagtail. Well Wagtail is just Django and That's a much better answer than oh, I didn't think of that. Because Whitale doesn't replace Django.
Speaker 1: You still have all of the capabilities of Django available to you So it doesn't matter if no one has yet implemented newsletters or e-commerce as an add-on to Wagtail itself. You've still got that platform below that you can tap into. And I think a place where Django has really done this principle of extending not replacing really well is the Forms framework. So if you have a totally vanilla edit form where you're submitting a form to update an object, you can do that with just two or three lines of code using the update view, class-based view. And as soon as you need to do anything a little bit non-standard, like a multi-stage form or a form that updates two objects rather than one,
Speaker 1: or a form that has extra fields depending on whether you're a super user or not, then yeah, you'll have to write some custom code. You have to leave behind that nice convenient shortcut. But you're always able to peel back a layer and implement what whatever you need for your special case within model form, or if that doesn't work, just a plain form. There's nothing to say that you have to use this whole stack. If your and if your use case really doesn't fit Django's concept of a form at all, you've s you can fall right back to, well, I I've got the request object, I can handle the post body myself. And if Django hadn't been so smart about this design, allowing you to drop back to the layer below whenever your use case didn't f
Speaker 1: fit the standard pattern of doing things, then they would have just given you model form or update view. This is the way of doing things in Django. And then when those non-standard cases came up they would have had to go, oh, we didn't think of that. We we better invent a new multi-stage update view on top of update view or a new form with optional fields for superusers view and they would have had to pile on these new special cases on on top of the stack to handle all of this. And uh unfortunately I think that's a pattern we've perhaps not been too good at following in Wagtail. Uh because when you're constructing a form for the page editor, you're going to have to deal with these edit handlers. or or maybe they're called panels.
Speaker 1: I'm never really sure. I don't think anyone really understands what edit handlers are, least of all me and I invented them. They they need to be there because when you have inline panels there's several forms in play and there needs to be something to direct the creation of those forms because Wagtail is doing this, it it doesn't want You don't want the White Tail developer to have to build all of these forms yourself. And this is a layer on top of Django Forms and Wagtail as a framework mandates that you have to use it. There's no option to go, actually, what I'm trying to do would work better just as a normal form. And as a result of that, when people want to do something non-standard, like having extra fields that are only displayed for superusers, there's a a lot of uh, oh, we didn't think of that, and oh we
Speaker 1: better build maybe we should build another layer on top of uh on top of edit handlers that can handle returning a different edit handler for whether you're a super user or not and it's all all a a bit of a mess. We've had to kind of make this up uh as we go along because we don't have that whole Django stack to really fall back on without a a lot of uh digging into the uh the the the here be dragons of uh edit handler internals. In hindsight, I really wish we'd found a way to make this work in a manner closer to vanilla Django with these forms and templates and You might be thinking, well, I don't know about that. I don't fancy having to write a template for the edit form for every new page type I create. But the point is 95% of the time you wouldn't need to do that.
Speaker 1: You'd have these shortcuts that the equivalent of update view that would do the work for you and build these automatically. And the other 5%, the special cases you'd have the option to drop down, peel back that layer, and work on the level of these forms and templates. So So another example, this was a classic long-running issue in Wagtail's GitHub, adding SVG support to the image model Now SVG doesn't really fit into Wagtail's concept of creating renditions of images with different sizes and cropping. What you need for SVG is something with 75 % of the functionality of the Wagtail images up, so you've got the uploading and the choosing, but
Speaker 1: uh but but not not the renditions and uh so yeah So you need 75% of the functionality, but no one can really agree on which 75 % there's going to be some customization for everyone who thinks they want SVG. And the the problem is we haven't really provided a way to peel back a layer of the Wegtail Images app. It's grown into this all singing, all dancing thing that you either have to use in its entirety or throw it away and then create something from scratch, maybe using the model admin module. And ideally what we'd have is another layer the the just below Wagtailed image and document apps for the general concept of things that have file uploads
Speaker 1: And then that would provide components like the multiple upload view, the uh and the the the uh screen fields block and the choosers. If you have something like SVG that almost fits the model of images but not quite, then you should be able to peel back a layer from the Wagtail Images app and build it and build a new thing on top of this. gen generic uploadable object app. But that uploadable object app doesn't exist right now. And doing that sort of refactoring on an already running project like Wagtail is hard work. I'm uh going to steal a metaphor from uh uh Masie uh Siglowski, the uh creator of uh Pinboard, uh who in uh who r
Speaker 1: in his case is this was a a blog post talking about performing server upgrades on a running service. He says it's like performing kidney transplants on a playing mariaki band. The best case is that no one notices a change in the music. You chloroform the players one at a time and try to keep a steady hand while the band plays on. The worst case scenario is that the music stops and there's no way to unfix what you just broke. Just an angry mob. And uh yeah, as developers, we're inclined to shy away from these kind of deep-rooted changes. We'll go for the low friction option. And in the case of Wagtail, that often means reaching for adding hooks. If White TailImages does 90% of what I want and I just need to customize this one thing, then I'll just add in this hook that will customize its behavior.
Speaker 1: And I th I think we're relying too much on hooks. I think it they make sense if you've got something that is designed to be extensible, like a toolbar, a menu, where you want other apps to be able to insert their own things. But when it comes to customizing the core behaviour of a particular app and you're adding bits on to handle our s the your special cases rather than Moving those special cases to the outside and building those in the most appropriate way according to the underlying framework. And this way, I think white tail images as an app, it risks becoming our new inner platform. In theory, you're not locked to using Wagtail images.
Speaker 1: Anything that Wagtail's built-in apps can do could be done by a third-party add-on Um the there's there's no special treatment of Wagtail images, Wagtail docs, th those th those core apps. But uh In practice, we've invested so much in whiteel images, adding the the multiple upload loaders and the screen field choosers, and it's become this beer moth that nothing else can compete with because you're because everything else is just starting from first principles. If you want to build parallel functionality outside of white tail images, you have so much catching up to do that it's a simpler prospect to shoehorn your special case functionality your not quite image functionality like SVG into Wagtail images.
Speaker 1: And uh Yeah, I I think that's sort of kind of a a bit harmful for developing Wagtail as an extensible platform. And as as I was putting this talk together, the the core team was discussing adding commenting to the page editor. Uh we we worked out how we might add annotations to specific spans of text within Graftale. And the s suggestion came up that's that though this is kind of a handy capability to have, maybe we should think about using Graftale sort of more widely for other text fields, not just for actual rich text. And this this raised my alarm bells a bit. So now hang on. Is is does this mean that DraftTale is going to be our new inner platform? If Draft Table has this one
Speaker 1: killer feature that's going to be more generally useful that uh that you can't get any other way, then are we going to end up with more and more of different fields being shoehorned into using Wagtail, whether uh using Graftale, whether or not that's a a good idea, whether it kind of fits what Graftale is trying to do, just so that they can take advantage of this one feature. So what have we learned so far? Well it's it's starting to sound a lot like don't do anything innovative or z or exciting, it'll just come back to buy you. And well that's Not very positive. So how can we make bold decisions without worrying about killing off our Mariaki band? And uh Wikipedia has a principle they call Chesterton's fence.
Speaker 1: And this is another another of those medal shorthands that I th I think it's useful to have, it ri could really do with a snappier name. But uh any anyway for now this is Chesterton's Fence and this comes from an NSA by GK Chesterton which goes something like this. Some people walking in the countryside and they come across a fence that blocking the road. And one of them says, Well, I don't see the use of this, let's tear this down. And the other one says, well, if you don't see the point of it, then I certainly won't let you tear it down. Go off and find out why this fence is here. And then once you've worked that out, then I might let you tear this down. One of the fences that Wagtail tore down really early on was the concept of MVC.
Speaker 1: In fact, just last month there was this question on Stack Overflow asking why Wagtail mixes in view code with models. It got closed pretty swiftly, which is uh a a bit of a shame, because it is actually a really good question. It's a bad question for Stack Overflow, but uh if uh PDC Ribeiro happens to be listening, then allow me to answer that now. So Django is good at building web apps for managing databases of things. And in the the excellent two scoops of Django book, those things are ice creams, which was a really smart choice on their part because Ice creams are nothing to do with computers. Everyone understands the difference between an ice cream and a web page about an ice cream An ice cream has flavour, toppings a price.
Speaker 1: The business logic for calculating the price of an ice cream, it's a very separate thing to the HTML code for a drop-down for of ice cream flavours. And everyone understands that separation and naturally it makes sense to split those up in the code. Now when we come to a CMS like Wagtail, the the things in our database, well they're web pages. So what what are the properties of a web page as a Django model Well, Django already kind of has a concept of a web page as an object in the form of class-based views. It's an object that knows how to accept an HTTP request and return a response. And that's a useful pattern. You can you do a lot with that. As we've learned, it makes a lot of sense to adopt existing Django cop
Speaker 1: uh uh concepts rather than going off on your own, sort of carving off your own pathway and inventing your own thing. So that's uh exactly what we did. So it's a class-based view with uh with some data attached, which means that it's a model. or it's a model with a view embedded into it. So we kind of lost this separation between the concepts, but it did mean that we avoided building an inner platform. Because anything you can do in a Django view, you always have the option of doing in a Wagtail page. That's how we come to have form pages and rootable page mix-ins. We have that escape route where if the default shortcut way of doing things in Wagtail doesn't work out. You can peel back a layer, drop down to
Speaker 1: working the Django view way. If we hadn't done that, then the view code for serving pages would be somewhere deep in Wagtail Core as a fixed function. And then when people wanted to go beyond serving the static pages There would have been a whole lot of, oh, oh, we didn't think of that. We're going to have to build this new thing on top of that. So we didn't just go, MVC, we don't need that. Let's tear that down. We went through this process of understanding why MVC was there, why it's generally a good idea, why it works for a lot of cases, but in our particular case it wasn't the most effective or meaningful approach. and we were able to come up with something more suitable. And I think while this definitely wasn't
Speaker 1: an easy decision to make, it was a bit like leaping into the unknown. I th I think that's uh it's turned out to be the uh a a good decision because we kind of were aware of what we were giving up there. So what are the fences we're faced with today? Well How about or stream field migrations? A lot of people aren't really happy that any change you make to a stream field definition will dump the entire thing into this new migration, even though Nothing is changing at the database level, it's still just JSON. And this isn't specifically a Wagtail thing. It's standard Django behavior to include the entire field definition in migrations. So if we d dig into like why why is the fence there?
Speaker 1: Why does Django do this? And it turns out well it's for data migrations. In data migrations You're you're writing Python code, and in that Python code you want to work with models that more or less replicate the real ORM. So if you have a text field with email validation on it and your you're doing a migration on that data, you want to have that email validation in place when you're working with that model, even though If you're just viewing migrations as a way of getting the database from A to B, it doesn't really matter. It doesn't make a difference what what that email validation is. But still it's useful to have that around for the benefit of the Python code you're writing. So presumably the Django devs thought, well if some of these properties are
Speaker 1: useful to keep around in the migrations, then Let's just go the whole hog, let's keep them all. Which does lead to the weirdness that if you fix a typo in like a help text of a Django field, then that's another migration. And that's a minor tolerable quirk in Django, but yeah, it's a major headache in Wagtail when when you have these enormous stream field definitions. So now we can kind of see the history why why that fence was put there. And it it made it made sense at the time when this was designed. But today, yes, it's probably not fit for purpose for how we're using this. So yeah, do we need this enormous definition for data migrations? Uh probably
Speaker 1: not, but data migrations on screen fields are an unsolved problem, so maybe that's where we should think about looking next if if we're going to sort of tackle this. Can we find a way of doing these data migrations without this uh enormous definition? Well yeah, probably. Um but uh that's uh an exercise for the reader as they say And uh finally I think we're ready to tackle what's probably the biggest movement in tearing down faces that uh Wagtail faces right now, uh React. I've I've tried to l love React, I'm not really there yet, and if anyone's going to be leading the the drive to Reactify Wagtail, it's probably not going to be me. Uh we're using a fair amount of Reacts in Wagtail already in places like the Explorer menu and in Draftale, but not in places that directly
Speaker 1: interact with the views and the forms that Wagtail implementers are routinely building. It's not getting involved in this whole sort of posting forms, getting responses workflow. And I can absolutely see the attraction of modernizing our user interfaces. quite recently. I think there was a of what what would you like to see the core team work on next, there was a a lot of enthusiasm for for using a lot more React and API based uh mm based transactions in in in the Django admin. And because Django forms are built on this very nineteen nineties transactional model where the server knows everything and the client is
Speaker 1: is stupid and that's a Very old rusty fence indeed. If you take our image chooser, there's a lot of classic jQuery era JavaScript here, populating hidden fields and replacing labels as you choose things. But The client side doesn't really have a concept of what an image object means as a whole. It's just a bundle of opaque form fields. The client side doesn't know what these things are. It's just those are all understood by the server. And when you post this form to the server, then the server will piece these all back together. And if we're going to have richer client side richer client-side and client-server interactions like being able to auto-save the page in the background, so
Speaker 1: which means the the client has to compile all of these fields into a JSON blob and push that back. or being able to duplicate stream field blocks so it's extracting the data from one and pushing uh it back to another one an another block, then it's not enough to for the client just to treat these as an opaque bit of HTML and and JavaScript, the client-side code needs to understand these objects in its own right. And that's where something like React comes in. So the questions that I really have to the the React advocates is where is React going to fit into our stack? How do we stop this from becoming the new inner platform? Everything I've seen of React
Speaker 1: so far suggests that it's pretty opinionated. If you change state you have to re render the whole thing. And that that does worry me a bit. So it's yeah. And How how do you kind of escape from that back into the old way of doing things, if you can do that at all? When we roll this out and have our first edge case, our first oh, I didn't think of that moment Will we be able to peel back a layer and get back to pure Django? Or will we be to be forced to fix this by piling React on top of React? Can our Mariaki band keep playing or will they have to stop and learn JavaScript?
Speaker 1: So the Django, the classic Django form approach might be past its prime, but It's handled every edge case that we've thrown at it so far. So let's make sure that we really understand what that's brought us, what the implications are of replacing that. And then then maybe you can tear the fence down. Thank you very much.
Speaker 2: Awesome.
Speaker 1: Okay.
Speaker 3: Thank you, Matthew. That was an amazing talk. Um Matthew slightly undersold himself there when he said he was one of the core team. He's the as you're probably most of you aware from his contributions on GitHub and Slack, he's the He's the lead developer of of Wagtail and um great to hear his position on all this. If anyone has questions for Matthew, then um the best place to s to ask it is in the Slack channel, which we're gonna keep posting. here in Zoom. But if you do have a burning question and you haven't found the Slight channel yet, do use Zoom and we will copy it across. Okay, yeah.
Speaker 3: I'm going to ask one in the meantime. Matthew, there was some really interesting work done by Bertrand Bordage, who's another member of the core team. um that was launched at Wagtail Space uh in Arnhem in the Netherlands the year before last. Um which did was it there was a uh part of an ambitious attempt to to solve some of these problems. What's your take on that?
Speaker 1: Yeah, that's so yeah, that's um yeah really exciting work and uh I I I guess the uh the The message about React, I think that kind of uh it's kind of ri writ written with that in mind. Um I think yeah th the it it's clearly something we uh we we need to do to improve the the wagtail experience but uh that uh I th I think yeah we we do need to to make sure that we're sort of not leaving behind the uh the the experience that Wagtail developers already have as uh as working with uh the yeah Python-based the Django-based form fields. And um I th I think it's while it it's you can yeah, go through this uh the
Speaker 1: process of Okay. And any time one of these edge cases comes up we can figure out a solution for that, but I I I think what uh we really need a bit of the foundational work where we can sort of say, yeah, we can hammer out what what are the what's the API that the Django uh form fields need to follow if they're going to interact well with what React and StreamField need it to do. So that might be things like being able to ha having a definition of pulling out the data from something like uh an image chooser and and doing that client side and turning that into something that can be pushed into another field
Speaker 1: just which is made yeah figuring out what what the contract is that our client side code need to needs to follow. And I think once we figure that out then that will be something that is kind of more generally useful for anything else that we build with React as as a basis, like the choosers or auto-saving. and it's not going to be sort of having to come up with another React-based solution for every one of those things that we build. So yeah, let let's yeah uh let let's go down that path but uh yeah I think uh make sure we've got those foundations in place first.
Speaker 3: Thanks, Matthew. Hey
Speaker 4: Matt, it's Andy Chosak. Uh I have a question. Can you is there an example of a fence that has been torn down during Wagtail's history?
Speaker 1: Oh yes, it's uh I suppose yeah, one that's where sort of Yeah, I I can think of s some that we're kind of in the process of tearing down, but uh I well I think what one of the one of them would be may it may be uh I I suppose yeah the um scheduled publishing would be one uh where I think from the uh where when we started out, um yeah, it's we we kind of realize that um if you have Yeah, it if you have a page scheduled for publishing and then uh go and make some more edits and commit
Speaker 1: and uh n uh and uh and change the the future go live date to something else and you could have several of these it these revisions of a page in flight at once and that seemed like a a bit of a dangerous path to go go down. And in terms of like the making sense of that as a user interface because when you go in and edit it, you can only see the latest version. If you're going back to C to fix a typo of an older version that's currently live, then how do you fix that without clobbering this new revision that's uh that's that's still waiting to go live? And so I think for a long time we ha I had this sort of unwritten rule in my head that, yeah, okay, to to m keep this sane, we're just going to have
Speaker 1: one w what one revision ready to go live at once. And that that would just become this sort of dogma in my head. And then as soon as someone uh uh came yeah c came yeah, wanting to have uh multiple uh iterations ready to go live, then it's it's kind of okay, what's actually stopping us from doing this? And this seemed like one of these fences to tear down. Um but yeah, I think when we put that through the RFC process and we sort of really examined that, we thought actually yeah, there's not really any fundamental reason stopping us. Um it is just something we need to figure out what the UI for that is. And uh yeah, once we uh yeah, once we figured that one out, then I think we're able to go ahead and make that change.
Speaker 4: Thanks.
Speaker 3: Matthew, I've got a question also that's been posed on Slack from William McVeigh, who asks the uh the objection to building out more functionality on the client side is partly React's opinionated view of how things should behave. Would the problem be alleviated by a less opinionated client framework like Vue or possibly Ember with its own MVC opinion that's more compatible with Django 's existing opinions?
Speaker 1: Quite possibly, yes. Um yeah, I I would uh I I think yeah, right now we're kind of at at the stage of figuring out what what the interactions we need behind Django and uh and uh and and uh and the client side are. So maybe once you figure that out then something like like View or Angular might be the the right thing. I suppose that there is That yeah, some element of uh yeah, we can't we can't totally discount following the the technology that client-side developers want to use. Obviously there is a huge amount of momentum behind React. And if we're going for something that's uh that that's uh le
Speaker 1: less uh l yeah. le less less attractive and less seductive, then are we kind of missing out on that expertise? But um I think yeah we have to uh Met and make that call, it might be a bit of a leap into a the unknown but uh but yeah. We have to make that decision at some point
Speaker 3: Thanks, Matt. I just want to clarify something that Sir Paul has asked. Is this about improving the admin interface or generating content management options for a React front end? And I just want to be clear that what Matt's talking about is um the the the the prospect of re refactoring, rewriting the admin UI in React. So Wagtail absolutely remains um An excellent candidate for building sites where the front end is in React or View or any of these other. And in fact, that's uh that's really like the the theme of this event, as it's turned out. There's a lot of talks about headless content management systems, which we'll we'll cover a lot later. And then Matt from Tim White on a slightly different note , it sounded like you said doing migrations differently was an exercise for the reader. Is that really where you were going to leave that topic forever?
Speaker 3: No
Speaker 1: no. No, that that that's where that's that's where the topic is at the moment. We've yeah, we've we've discussed this and uh yeah, I I th I think I I might might be uh um might be in the minority on sort of standing in front of that fence saying, wait a minute and uh and I think yeah the uh uh I think uh Cohen all yeah came up with uh uh some a snippet for to yeah, say if or selectively opting things out from uh f f from uh uh the And and yeah, I th I think as I said something uh now that we're aw aware that this is a fence and it might be valid to tear it down That's yeah, some something that we can work on. Obviously, like everything in Wagtail, this is a work in progress.
Speaker 1: Um, but yeah, we we can sort of yeah, think about more intelligent ways of uh of representing uh the uh screen fields data in migrations.
Speaker 3: Great, thanks. I think we've covered the questions unless any other moderators can tell me I've missed anything.
Speaker 5: I I I am hoping this isn't a duplicate. I've been juggling a couple things preparing for Cohen 's talk, but I believe Jeffrey Babb asked, why do we need these SPA approaches at all and why React? Are these tools the new XML slash blockchain? Hopefully that didn't just get asked. Or did it.
Speaker 1: Well yeah, as as as I say, I'm uh I I I I suppose I I'm a bit from the the uh ol old fart uh I I'm kind of uh yeah very skeptical of uh any new technology that comes out. I remember it even when jQuery was new and it it promised to change the way you write JavaScript. And it's well yeah, m maybe I maybe I like the old way of writing JavaScript and uh But uh but but yeah it it's um I th I think th these things kind of uh to f yeah it tends to to come to to cut coming cycles. Um I think yeah the no SQL was one at one point and uh but then yeah that some people to realize that uh they rediscovered oh actually SQL databases they they did solve these problems in in in the nineteen sixties
Speaker 1: that people are suddenly rediscovering. So and I'm I'm not saying that's the case with with every new technology. So some of them have more staying power than others, but uh But yeah, I I think this is where you need to understand what you're replacing. Um yeah, where yeah. Wha how yeah, what Yeah, whether the path that brought us here has there there's more to it than you think, and whether the new technology is really covering all the bases that the old technology did.
Speaker 3: Although it isn't my area of expertise, I'd just like to m to try to make the case for for the SPA style approach, um but which is really b based on my understanding of what other people have said and it's really that we want to continue making the editor interface the best that it can be. For example, we want to have autosave and we want we would like to think about having live previews in the editor. And I think where where we're trying to m manage state to enable things like that, then it looks like the React type approach can help us solve those problems where at the moment we have what's increasingly complicated. code base of Django templates and jQuery. So I'm not saying it's impossible to do it with a more traditional approach, but I think those are the those are
Speaker 3: those are among the cases that are given.
Speaker 1: In fact, maybe the one case that's closest in Wagtail to selling me on on React is like the the choosers. I've been kind of working on Yeah, I've yeah, a while back slowly on uh rebuilding these so that they are more generic. I mentioned having this sort of generic layer underneath Wagtail images and wagtail docs. And In those you have this chooser interface where in one tab it's you have choosing existing items in the other tab its uh form to create new ones. And if you're managing this as kind of one transaction, one thing that gets posted to Django, the old client survey, and then back again, then it's kind of pretty the uh it's a it's uh it's a uh a bit uh mind bending to try and capture the state of both of these tabs at
Speaker 1: the same time. So yeah that's where you can really want to treat them as their own thing. So this as a more fluid thing where this is one component that knows how to communicate with the s with the server and this other tab is another one. So I think yeah it's it's It's something where you're you'll tie yourself in knots in uh if you're trying to do it the uh the the old sort of Django uh form posting way. And I think yeah, the the the more people we can get on board with building those interfaces in the sort of way that front-end developers are used to, then the better.
It is the tendency to build a highly customizable system that becomes a weaker reimplementation of the platform it was built on. Matt illustrates this with a CMS that recreated database-like structures inside XML and later with frameworks that duplicate Django’s capabilities.
Discussed at 5:37It should provide convenient higher-level shortcuts for common cases while allowing developers to peel back a layer and use the underlying platform directly. This avoids piling more special-case functionality onto an abstraction that has already reached its limits.
Discussed at 8:53Wagtail extends Django rather than replacing it, so developers can still use Django’s views, forms, and other capabilities when Wagtail’s built-in patterns do not fit. That underlying Django layer provides an escape route for custom functionality such as newsletters or e-commerce.
Discussed at 9:38A Wagtail page is itself a web page object that needs to handle an HTTP request and return a response, so combining model data with view behavior is meaningful in this context. Reusing Django’s class-based view concepts also leaves developers an escape route to normal Django view code instead of forcing them into a separate Wagtail-only system.
Discussed at 20:33Django migrations preserve the complete field definition because data migrations need historical model definitions, including behaviors such as validation. That is tolerable for ordinary fields but becomes unwieldy for Wagtail’s very large StreamField definitions; Matt suggests a more selective representation could be investigated.
Discussed at 24:25It needs to understand what the existing Django form approach provides and define a solid contract between Django fields and client-side components. React or another client framework should be introduced in a way that preserves an escape route for edge cases, rather than forcing Wagtail to layer more React-specific fixes on top.
Discussed at 29:55Scheduled publishing initially treated having multiple future revisions as dangerous, but examining the assumption showed no fundamental reason to allow only one revision waiting to go live. After working out the user interface and behavior, Wagtail moved toward supporting multiple scheduled revisions.
Discussed at 34:21Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published July 19, 2024
Published July 19, 2024
Published July 19, 2024
Published July 19, 2024
Published July 19, 2024
Published July 19, 2024