Wagtail under the hood

Hello, my name is Koen van der Kamp. I'm from the Netherlands. I'm a Python developer and a member of the WagTil Core team. I work at 4Digits. We provide Python development and maintenance. Many of our clients do not require a CMS at all, but when they do, we choose Wagtail To contribute to Wagtail, ForDigits organized the conference and called it Wagtail Space. We organized Wagtail Space the Netherlands three years in a row. The name stuck and is also used by other Wagtail events, and that is awesome. I want to thank the organizers for organizing Wagtil Space US. Thank you for having me as a speaker My talk is called Wagtail under

the hood. During this talk we will have a look at Wagtail Inner Workings, what happens when a request comes in and a response is returned. I will recreate WagTil from scratch. This way we can discover it step by step. The names and structure of the code will be similar to WagTil's source code. Some parts will be simplified because I'd like to explain the overall concept, not every detail. And the Wagtail codase is big. I will not recreate all. I'll focus on page request and response. I'll answer the how and why behind some Wagtail concepts. I give you some tips and explain things that might trip you. Wrapping up, I'll explain why Wagtail is awesome.

I'm going to create a virtual environment, install some packages, define models, create a view. write templates, explain URL patterns and add context data. Let's start with creating a virtual environment. A virtual environment are isolated environments for Python projects. This means that each project can have its own Python versions and its own dependencies. Everything we install in this environment will not influence any other project. Let's install Django, Django treeBit, and Django extensions. TreeBeard is used for the page structure to organize the pages, and Django extensions is a helper that makes Django development easier.

I'll now create a project called Project and C D into it and create an application called Core. Now let's fire up uh my editor and this will take a while And I'll switch to presentation mode so you can see what I'm doing. And this is our core project and project folder with the settings. And let's add our applications to installed apps.

So I'll add core and I'll add tree beard and Django extensions There's no Python interpreter configured for this project yet and that's what I'm about to do. PyTarm will inspect my work and give hints and autocomplete and help me with my imports. And the only thing that I have to do is select the Python that I use. And this will start uh indexing my project so it can quickly give ins

So let's create some models. A model class represents a database table. It allows you to query the database with Python code. We need a site and a page. The Wagto site model stores the domain name. and a reference to a root page. The root page is the first page in a tree also known as a home page. When a request comes in, the site model with the same domain name is looked up and the root page or the children of that root page can be served. You can have the multiple sites configured in a single Wacto install. So one admin with multiple front ends.

Let's create a page. A page contains a title and a slug And the slug is a part of a URL, the word between slashes. For example slash blog slash then blog is the slug. Also the content type is stored and I will explain that a bit later. I'll now add a string method and it gives the default

representation of a page. Next, I'm gonna create a home page and normally this is project quote and will live in a separate app. But for simplicity, I will mix Wagtail and project code here. The home page has an intro field and that's text. And I will also create a list page. It will list its child pages but won't contain additional data. And I'll create a detail page. And this page will have an author. And a body field containing some text

These are regular Django models. If we were making a regular Django project, this would be the database definition. Now we would create some URL patterns and views. I'm not going to do that but I will quickly show you a regular Django request and response cycle. A request would come in and match your URL pattern, then a view will be called. It will query the database and get some data. A template will be rendered with the data as its context and the rendered template is returned to the browser. In a pure Django project, a URL will be hard-coded and point to a specific view. Fixed URLs and views defined by a developer won't work in a CMS. In a CMS, content editors create pages.

They can add them to at any point in the page tree. It's up to the CMS to figure out what page to serve. Analyzing the requested URL and looking up the correct page is a dynamic process. We need to be able to store pages within a hierarchy. We call this the content tree or the page tree. WagTil uses tree beard. an MP node, that's a materialized path. It's the fastest way for SQL to work with a tree structure. It's time to create a database and have a quick look at it. Python manage make migrations and python manage migrate and now we can open up the database.

So the core page, our base model, has title and slug and content type and some fields for tree beard to store the hierarchy. and the homepage has a PTR or a pointer in the intro field. Note that page is a table, homepage, list page and detail page are separate tables There's a foreign key relation called PTR pointer and it points back to the page table. This is called multi-table inheritance. This means that WagTil can query all pages via the page table, which is great, a single table to get any page. But if you want to be specific and get a field on um

a subclass like intro on the homepage. We need to be able to go from a page instance to the most specific subclass, in this case a homepage. And that is what I'm about to show. Django has a content type model that has an entry for each model in the database. It's used by Django Permission Machinery and Generic Foreign Key. Wagtil uses the same content type table. Wagtail stores the content type on the page model, this way we can get to the most specific subclass. Let's set the current type during initialization and add a specific method.

I need to pass the arguments and keyword arguments. And when there is no ID, the item isn't created yet, we set the content type. So if we create a home page, this will contain the ID to the homepage content type And I also create a slug here. Normally WagTil does this for you in the front end via JavaScript and the Slug is based on the title. I do it here for convenience, just to make things simple

And now we need to add a specific method. It's actually a property, so you don't have to call it. And it looks up the current content type based on the content type ID stored in the content type field. And it returns the most specific instance. So if we create a homepage and it says saved, then it contains the homepage ID on the content type field. And if you call specific, the home page instance is returned. Now it's time to create some content and I will use Shell Plus.

It comes with Django extensions and does some uh imports for me. So that will save typing And first I'm gonna create a home page and it will add use add root. This is a tree bit method. And it will save automatically the homepage for me to the database as a route item. And now if I query all page objects or the first one I get a home page and it doesn't have an intro field, but if I call specific I get the home page, another

page type, and if I call the intro field I get the actual value. So this specific method works. Next I'll add a subpage and it's a blog page, it's a list page. And the title will be blog. And I have to add it as a child to the homepage. And again, this is a 3-bit method and it's saved automatically to the database. Next I'm gonna add an article and it's called Django is awesome. And we can store the author.

And again I have to use addChild to add it to the parent page, in this case the blog page. So blog at child instances article And I will create a second article and it's called Wactors Awesome. And I'll do the same, I'll add it to the blog. So if we call blog. getChild pages of cat children I have to say

it will return all children. Last thing to do is to create a site And this is a regular Django query again. And the site has a name. My site. Very original. And the host is the local host My current IP where I run the development server. And the root page is the home page Now we have some data and we can query our database and get a page or specific pages. It's time to create a view. In WagDil, the view is defined on the page class. All other page types inherit the same class and thus have the same view. The method is called surve

and it returns a rendered response. It takes the request, a template name, and context. And the template name is based on the name of the class. So it lives in the core app and we take self-class name in lowercase dots. html and we add context and that's the current instance self So it's a quite simple view. Next, I'm going to create

templates. And the first template I'm going to create is the base template. It contains HTML shared by all other templates. And it will show the page title as a heading. And it contains a block content. This is a slot for other templates to put the data in. Next time we can create a homepage template and it extends base. And the content is the intro field. And let's make that a paragraph

Next, I'm going to create a list page. The list page will list all its children. For child in page get children, we're gonna create a link. And let's wrap that in a paragraph. And it points to the child slug. And the name will be just child. And that's the string representation. So this will be the title. And then the last template will be a detail page. And the diesel page will uh contain an author

and a body text Let's fix that. And yeah, I think this looks good. Now I have to wire the URLs. First I have to include The core URLs and it comes after other already defined Django views. In this case an admin view that we don't use at the moment. And let's create a new URL file with URL patterns.

And we'll use a regular expression to match words ending in a slash. This is a greedy pattern. It will match slash slook slash look slash look So it's a word and um ending in a slash and it can repeat many times. and there's an first group capturing group so all together will be matched like you see in green. Now add the surf method. The surf method will look up the site and the homepage or the root page

And it will look up the site by looking at the host from the request. And the slugs is the second argument, and therefore we need to split the path on slash. So this will actually be a list if you have blog slash Wacto is awesome, then blog is an item and Wacto is awesome is the second slug. and it will return the root page and call the route method on that. That method doesn't exist yet, so we're gonna create it

So almost there, let's create a route method on the page. And this receives the request and the slugs. And if this if there are slugs we have to do something, but if it's empty, we have to return the current page. We will serve the current view. And if there are slugs we have to look up the children. And the first child slug, the first slug in the list is the child slug. We'll look up the child page. And the remaining slugs are all other slugs. So

let's try to get the child the subpage And here 's the get children method again. And we want the child with the correct slug. And if it does not exist We will raise a four or four. But if it does exist But if it does exist, we will call that method the route on the child. So this is a recursive function.

It will call itself again and again until there are no slugs left and the current page is served. We've written quite a bit of code. Let's start up a development server and open a browser I 'll increase the font size a bit. Well, this is your home page, and blog returns the list page And it has two articles. Django is awesome and Wagtail is awesome. So this is the complete Wagtail request and response cycle for you.

The last thing I like to show you is getContext. We add this to the page model and by overriding it in a subclass you can add additional context. I'll open up the home page. And here we'll add a form. Let's uh add the Django authentication form. And I need to import that.

And I also need to add the form to the template And now refresh the home page, and there's our form So first some general tips. Use a virtual environment to isolate your project from other projects. Configure your editor, let it do the work for you. Try to learn tools like Django extensions. It will save you a lot of time. Read source code. By knowing how something works, you can bend it to your will. And if ever in doubt what the best solution is to your problem, just make small proof of concepts and try all options.

And what we have learned today is that Wagto PageTree is handled by Django Treebeard. And Addroot and AddChild can be used to add pages to the tree programmatically. Use specific to go from a page instance to the most specific instance. A page combines views, routing, context and an admin definition all in the model class. Get context lets you add additional data to your page. And you finally you can combine any Django view with Wagtail. So why Wagtail? Why not the top 3 CMS? Why not WordPress, Drupal or Yumla?

The answer is because of Python and Django. Python is easy to read and rapid to write. One of Python's greatest strengths is its large standard library. PyPy, the official repository, has over 300,000 third-party packages. Python is a team player and connects with anything. All Python powers are available to you. So search pypy. org and read the Python documentation And next in the stack is Django Framework and Django its third-party packages. They provide functionality that you can use in your WagTil site. So if you search to add functionality to your WagTool site, you might also search for Django packages

and read the Django documentation. So good places to search are PyPy again and the awesome Django list on GitHub. And there's also an awesome Wagtail list too. And there's a site called DjangoPackages. org. If your site has Wacto under the hood, it has Python and Django under the hood too. With all this power you can drive it anywhere. Thank you for listening. This was my talk. I'm happy to answer your questions.