Fact checking with Wagtail

Speaker 1: Hello, everybody, and welcome to our next Wagtail Space um session with Full fact today. Just going to give it a minute while everyone's joining. We're quite a lot of people in the waiting room today keen to get started, which is great to see Some familiar faces Hi Ralph. Ralph's been to quite a lot of the talks, I think that you could write do a good write-up about all of this after. Would be nice Right, we're at quite good numbers, so I am gonna get going.

Speaker 1: So I'm delighted to welcome John and Craig from Full Fact. which is the UK's independent fact-checking organisation. And they're going to share how they've used Wagtail to overcome Django's admin limitations and manage their 10,000 plus fact checks with greater speed and efficiency. Over to you, John.

Speaker 2: Thanks, Lisa. Hi everyone. Yeah, as Lisa was saying, my name is John and my colleague Craig is also with me co-presenting. As a sort of caveat before we start, we are relative newbies to Wagtail. We uh fully migrated to the Wagtail CMS uh in May of this year. Um so we've uh we're gonna share our experiences with you um uh both uh myself and and and craig are uh on a sort of communication side of things, we're not developers, but we'll hopefully give you takeaways that can be applicable um in in your organizations. For those who are not familiar with what a fact-checking charity is,

Speaker 2: We monitor uh the the the internet and the general information environment for potential uh incorrect claims. Um to and then fact check them with the intention of allowing people to have a uh a source of information they can trust so they can make their own mind up on whatever it might be. And we always try and prioritize as as a charity. those claims that might potentially do most harm uh to try and you know stabilize democracy and and and uh all of that. So more about our website. I'm gonna share my screen now. Hopefully you can see this. Uh false packs has been around since uh oh, I miss press share. Full Facts has been around since 2009

Speaker 2: and we've had a website presence since then. But in 2018, there was a significant website development change to Django Adam CMS. And that has been the CMS that Full Factors used up until 2025. It's got a growing amount of content on there. Not necessarily comparable to the likes of the big sort of media outlets, but there are about 10,000 fact checks or other articles that are on the site. It's used on a daily basis by a team of journalists, but then other content contributors throughout the actual uh organization itself. Now at the start of 2025, we wanted to move away from Django Admin and move to Wagtail for a number of reasons. I'm sure you will

Speaker 2: know lots of uh the benefits of of Wagtail and maybe some of you are familiar with uh with Django admin as well, but essentially it was to to build like a solid foundation for future web development projects. There were a significant number of bottleneck issues with publishing content. You know, rich text editors go so far. But uh if If non-standard content needs to be a HTML markup, then the variety of CMS users, uh journalists and others that use the CMS had uh varying degrees of competencies in in using HTML. So we wanted to make it more accessible and we can now do that through greater user management, allowing or restricting certain areas of the website or certain features of the website in different ways. So what I'm going to do is show you what this looks like for us on the back end.

Speaker 2: So I'm just going to exit out of here and go into one of our fat checks. So this is what most of the fact checks look like. I'm going to come back in a moment to this checked claims, this claim conclusion bit at the top of the page. But just talk about how for a second we've moved beyond rich text to embrace stream field blocks, as many of you are aware. And a lot of these stream field blocks might be um generic like the embed function, there's a as a there's an image that's donate uh added into the article further down the page, but we've started to a process of using our um uh a pattern library which has got a number of user interface components and then turning them into stream field blocks.

Speaker 2: So this featured card, if I choose an article Let's type something in. Um this is now turned into uh a String field block and as sort of uh user content block markup on the actual uh web page itself. So it's highly accessible for users that don't need to uh create that as HTML. We've not fully moved away from HTML. We have HTML blocks themselves. The idea being that, so in this instance, we had a guest. um uh SEST uh NMP, we wanted to highlight that in a way we can utilize a this sort of custom HTML block and then at a time when we see appropriate, if we see

Speaker 2: sort of a consistent use for how this can work as a user friendly stream user friendly stream for your block, we can then turn it into that. But beyond that, we're going to talk today a little bit how we've embraced snippets as well to really utilize structured content. So I'm going to talk about that in a few ways. Firstly, obviously we need to convey and try and build trust with all of our users. One of the ways we try and achieve that is through transparency, corrections So when we check a claim, we check it in the abstract and we don't assume intent, uh, you know, misinformation versus disinformation, but we want to also embrace that ourselves by saying

Speaker 2: this is where we might have made a mistake or in this case just updated the article uh with newer information for additional context. Um and we utilize stream field blocks so we can uh pull all of the article updates into one place so we have a transparent visible corrections log through all of our um through uh all of our corrections. And another way we're we're embracing this is through uh checked claims. So here is the claim in the abstract. But it has been cited by somebody. John Healy happens to be a UK government minister. And he uh so whereas we don't assume intent, we want to report the sighting. That is a snippet at the moment.

Speaker 2: And we've got future uh uh intentions to build out the um uh the the sighting itself, but also the what we call intervention off the back of that. So we asked this UK minister to correct the record. There is currently a stream feel block in the article, which is a free text block that says what we've done and what the outcomes are. But with Turning this, which is the next step, this is the next project, into a a snippet, a stream for your block, we can do more things with that information. we can report on trends. How many times have we fact-checked one particular political party or a media outlet? How often did they correct the record?

Speaker 2: And Braig in a moment is going to talk more about topics and how we've used Taly's snippets to knit together an ecosystem of information. um using uh at least uh index pages as one of those means. One of the things we can do, for example, is to take Keistarmo, who might be the Prime Minister of the UK, who might be a topic, as you know, subject matter, but also might be a claimant. So we could start to utilize and present that information in other ways. So this is what we're currently building How many times has Keystama been a claimant? How many times have we asked for more information or asked him to correct the record? And what are the outcomes? of those intervention steps.

Speaker 2: Just to build more ways where we can bring together structured data and present it to users in different ways as opposed to having lots of individual essentially instances of free text. One final thing I'm just going to quickly touch upon before we pass over to Craig to talk about. uh topics and snippets uh because i mentioned it in the um in the description for this uh Just an interesting way of how we've used in in this case like schema structured data. So information, meta description, markups that we allow Google to crawl and repurpose for their own uses. We have we've uh we've always structured our claims in this way

Speaker 2: through a claim and a conclusion because up until recently Google has presented that information in its Google search descriptions in this particular way. On the left hand side here, you can see what the claim is and our verdict. So users in Google search and beyond in other Google products. can have a structured way and we can and a consistent way for how we present our information. Recently in June, they removed that and now there's a wild west of Of how Google might take the various different types of descriptions from our page and serve that to users, sometimes seemingly out of context, showing them misinformation. So it seems like we're perpetuating that misinformation.

Speaker 2: So we've had to pivot recently from that claim review markup. And instead of wrapping the claim in claim review Which we still do. We now also wrap it in a no-snippet structured data markup. So we could tell the Google, whatever you do, don't use this claim, this potential piece of misinformation. and then serve it to more people again. So there's just different ways that we're trying to combat misinformation all the time, but using these modular form on our Wagtail has allowed us to do that. in a in a nice and reactive way. Okay, I'm going to stop sharing my screen now. I'm going to pass over to Craig, who's going to talk about how we're utilizing uh

Speaker 2: snippets for a better uh ecosystem of data and content.

Speaker 3: That was seamless. In the practice it was not seamless. Okay, so our Certainly my initial insight um in terms of how we approach this project came from running the newsletter. So that that was where I started before our support experience team existed. And I was obsessed with this idea of how do we drive up engagement? And then eventually that turned into, okay, if we can do it on email, why can't we do it on fullfact. hook. org? We've got all of this content. How do we do that? So The problem, as we said at the top of the top, was um that we've got 15 years, um 15, 16 years um worth of activity. and 10,000 articles. And when we first approached this, it was on a Django, CMS, and there were no

Speaker 3: topics. It was very, it was all over the place. It was not well structured and our website traffic was declining. And I was told basically that this was a thing that was going to happen, this was unavoidable, and we're just going to concentrate on social media now. And um That to me sounded like a challenge. So um we got stuck in. But the initial reaction, I guess, for both me and John was very much this Scooby-Doo, uh, which is meant to be a gift, but it's not been a gift. So we were to this problem was very much like, oh dear, what are we gonna do So, our first approach to this problem was basically to sit down and do our research. And We are living in an age of AI and AI can point you in the right direction on this stuff, but it is no substitute for doing the hard yards of getting down there, reading through the latest blogs

Speaker 3: on SEO, on search, talking to people who know their way around data architecture and things like that and coming up with kind of a comprehensive vision. for um how your users are going to interact with your data um and with with your articles. So um so this led to uh a solution which was basically that the website should be a filing cabinet in an ecosystem. Now a visual representation of that is on the right, just to give you a sense of what that that that is. I'll show you what that means in the practical reality. But basically this was how do we present our information in a logical, hierarchical, structured way, but also in a way that's much more organic and kind of fits where users actually are. So where bad information is, does our good information also sit within that ecosystem as well?

Speaker 3: So how are we true to our values and how do we utilize the tech and how are we just smart with our research to kind of get the most out of that So one of the key things again, this is me using the term hard yards again, is to talk about taxonomy. So me and John basically spent a month going backwards and forwards in a Google spreadsheet with most of our content mapped out being like What is a category? What is a topic? What is an author? And and working all of all of this out basically And and it it seems very simple at first until you start going, well actually that doesn't fit in there and that doesn't fit in there and and but it is really important. Get your taxonomy down first. Get it documented Do the hard yards, do the discussions with your stakeholders as well to make sure that they're comfortable with it.

Speaker 3: And never forget your users as well or readers as we would call them at full fact. They are they are they are the key stakeholder. If they're reading something and not engaging with it, then arguably we're not doing our job well in terms of putting good information out there. Good information is only as good as a person's ability to engage with it. So taxonomy is important for that. So this is where we got to in terms of that. So you can see the ecosystem elements of our structure. um of our hierarchy um with the website represented by Sir David Attenborough there. I thought he was a good representative for ecosystem and then obviously a nice IKEA um file in cabinet there for our kind of file in cabinet system. So a category, this is your bog

Speaker 3: standard stuff you see on a website. So politics, health. immigration. We didn't have a politics category when we first started for a UK fact-checking website like Full Fact. We used to produce a lot of politics content. There was not a politics category. It is now one of our most popular categories since we've done that. So we went to where users were, we looked, we did a substantial piece of user research. Users were like. . Come on guys and we met them, we met that need and actually it's doing really well for us. And then around the outside we've got topics. That's an attempt to produce clusters of information. So we did a really good piece of work on statins recently. And we had a case study where somebody had believed online misinformation about statins and we packaged that all together in a statins topic. And some of that lived under the category of health, some of it lived under common

Speaker 3: or some of our other categories. But all of that sat together as a package and now that does really well on Google search of in in terms of when people are Googling statins, they can see that there's been an impact from the bad information They can see what that impact was, but then they can also see the evidence to support why that bad information was bad and access that good information. And the final part is authors. In an age where AI is king and it's going to continue to be key. Authenticity is key and for a fact-checking website authenticity and trust are important. So we used to list authors at the bottom of our pages. Now we link through to these author pages which contain a little bit of a bio about the author. They're going to include pictures as well, because that's a really important thing.

Speaker 3: And an index page of their fact checks. The idea being that um Not only do we want our users to be able to see that who this was written by, who that person was, what they looked like eventually, but also that they can then look at their back catalogue of fact checks and go, actually this person knows what they're talking about. I trust them um when they're when they're relaying this good evidence. So that's an attempt to kind of meet the challenges of the day. So where has this gotten us to? So basically it's an a a an iteration of one kind of page which is an index page and because we're starting early these are going to iterate over time but this is where we are. So on the very far left there we have a category index page to that you you produce a fact check and you list it straight away under one of these category pages.

Speaker 3: So this is this is health and then everything is listed to do with health Now on the sort of center left there you have one of an example of a topic index page which again looks similar to category but that can have sort of multiple fact checks from multiple categories kind of grouped together in one topic On this kind of center right there you've got content type. Now this is a little bit more subtle, but this is an attempt by us to basically produce three kinds of output to the user in a kind of very visual user-friendly way So we've got our fact checks, but we also want our users to know when we're maybe writing a comment piece that isn't a fact check or an analysis piece which actually isn't a fact check or a comment piece but it's actually more of an in-depth investigation So we kind of use those more on the front end to give our users the ability to kind of differentiate those kinds of content.

Speaker 3: And then on the far right there you have an example of our um author pages. So this is for Leo who's one of our big health fact checkers and Leo writes a lot of our great content around vaccine misinformation for instance. So being able to establish trust For for Leo is really important so people can say, actually, this guy knows what he's talking about. I can go back through and read his back catalogue. So what I'm ah actually final point before I do this and this is not my strong point is stakeholder management in doing all of this especially when you're in a small organization and especially because we were a brand new supporter experience team Stakeholder management is key. And John is very much the stakeholder manager. I am not the stakeholder manager at all. I am the like here is the vision, go do the vision.

Speaker 3: John is like Actually, let's go and negotiate the vision into existence, which is a really important skill. And I cannot stress this enough when you're working on your own project. If you don't get buy-in from people who aren't techies, who aren't skeptical, and sorry, who are maybe a little bit skeptical about all of this and just think you're attempting to like do something to them. and you're not going to get very far. And a lot of the progress we've made, I think, is in large part due to kind of John's skill at kind of bringing people with us. So So the TLDR as the kids say on Reddit is keep your users in mind but if you've got lots of content try to give your users the ability to choose their own adventure. Basically, so this is what we've done with with Wagtail is uh kind of choose your own adventure version of full fact for our users, and it seems to be working.

Speaker 3: So I'm gonna take you through. Um I've done a spoiler there, but I'll take you through um The uh this is where I tripped up last time actually. Hang on, let me escape. There we go, escape, that's not so smooth So I'm going to take you through the process of how we tag an article. Now we have one of our recent fact checks that we did. This is our one of our big in it's not actually a fact check, it's an evidence piece. But you can see it's listed under health And this is where we went into a very scientific analysis of some of the arguments that have been had around paracetamol, as we call it in UK, or Tylenol. So you can see at the bottom, these are our topics that we talked about earlier. So we've got them tagged under Donald Trump and Autism, but we want to add a topic there of paracetamol

Speaker 3: for our UK audience. So again, this is what a fact check looks like on the back end. If we're going to add a tag though, or a topic, sorry, I use tag and topic interchangeably. um but topics is is is the name we will go to promote and what you can see here is actually we've got Donald Trump autism correlation and paracetamol And you might be asking yourself, well, why are some of these shown and some of them not? Now the reason why correlation and paracetamol aren't shown is that they aren't published um uh index page yet and we have that degree of moderation so we don't just have lots of index pages we aren't cannibalizing traffic and so we can take a bit more of a strategic approach to the SEO with this So very easily, if we're going to add um a topic on here, I'll just go and add paracetamol on there again.

Speaker 3: Paracetamol. We would go and publish. Again, this won't appear on the front end yet, but I'll just do an administrative update and just call it topic. So we've got a kind of version control kind of path behind. And the way we manage topics is under our snippet tree. So if uh under our snippet kind of um So if we come here you can see our various snippets but you can see our various topics. 904 instances, not all of them are index pages again. So if I click on that can see these are all of the topics and the articles that they apply to. Now you might say for instance under ID cards, we do have a live index page there. ID cards are a big issue, digital ID cards in the UK at the moment. So that's an area where we want to have

Speaker 3: an impact and where we want to have a say. So what we're going to look for is we're going to look for paracetamol. Now you can see there are two articles listed here. It doesn't have a topic page, but we're going to add a topic page. So add a topic page and this is where I will end up butchering the spelling of paracetamol. Set or more right in front of there we go. Thank you, spell check. Um there we go. So what we can do is sorry, and then we need to choose the topic. So this this page sits separate to the topic, but we're gonna go in here and we are gonna type it in paracetamol and there we go we can see the topic and in terms of the description the description will uh appear separate to the meta description on the index page So here is one I prepared earlier.

Speaker 3: And I'll just quickly copy that in. Just tidy that up a little bit. And then go to promote and obviously we're going to pop our pop our meta description description in there as well. So that appears on search engines basically but Google overrides that most of the time. And we will then choose an image, very much standard fair. Do we have a paracetamol image? We do not. So what I will do is I will or a holding image for the meantime and we'll just use that one. Now we'll change that once we are done but usually we would have like a a good hero image to go in there. as well. So the only other thing I'm going to do before I publish this is go to content and hide the meta

Speaker 3: description. The reason why I do that is that will become apparent when I hit publish. So if I hit publish That is now live and you can go and check that for yourself if you want. Fullfact. org forward slash paracetamol and then hit live and there you can see. There's our associated fact checks with paracetamol. And this is the description here. And when I tick to disable the meta description, I ticked to disable that or else it would have put the meta description up there and uh the description as well. So that's how we do um how we've used snippets to to to do that. I will quickly migrate back over to slides. I should have just stayed on the same screen So it basically we use

Speaker 3: these topics as landing pages, I think, if you were going to kind of use the kind of marketing terminology. But then they're more than that to us, they're a kind of our being. part of the conversation but and and they have added traffic over time for us where people are coming to these clusters of information and so they're doing quite well for us. So um that's the kind of the end result of that. I'm gonna move us on to any questions unless you want to say anything, John.

Speaker 2: No, I think that's everything. Thanks so much, Craig. Um yeah, any questions at all? There's there's one question that I'll answer in the chat if we've got Lisa, do we have five minutes of questions?

Speaker 1: We have got time for questions. Yeah, I'm just going to read that question out in case anyone hasn't seen it in the chat. So Nigel would like to caveat at the beginning of this question that he does agree with the site's view, so he is likely to trust it, but How do you build trust with people who do not share those views or people who distrust mainstream media?

Speaker 2: Thanks uh Lisa. Thanks, Nigel. I think that's a really interesting question and something at at the sort of the core of what full fact is about. We're desperately trying to fly the fragile flag of sort of impartial uh establishing of facts in a world which is increasingly becoming quite partisan and and losing that tether to truth. You know, you have you have political spin which tries to which is fine, that's what people do to try and take a fact and and show it in a light and more more frequently we're we're getting to a stage where There's a complete disconnect between the established facts to have uh whatever conversation, whatever discussion we want, and and uh and and on what might be posed as something which is entirely untrue.

Speaker 2: It's a difficult question. There's no silver bullet. Otherwise, you know, we wouldn't be in existence. But we have a number of audiences. Like you, Nigel, like people who might uh be more inclined to agree with um with full fact's uh uh conclusion or something because we're hopefully we've built up and demonstrated that trust. We might class you as a supporter or a potential supporter and other people as beneficiaries to try and uh prevent the um the perpetuation of potential dangerous misinformation. I think, you know when Craig was talking about examples of statins or vaccines, is there's a clear line there between um taking um misinformation and then there being a direct harm to that.

Speaker 2: But we we think that the information can be used in multiple ways. So it's not just about trying to change the minds of somebody who might think that paracetamol causes autism, but also, sorry, tidalamol as well. but also to provide at least some semblance of of uh sensibility and reality to those who want to um uh keep in touch with a a a a a source of of of uh trustworthy factual information. You know, we we that we often ask uh get asked like who fact checks the fact checkers. We are uh as as much as possible as clear and transparent in our workings. So you know you can do the working yourself, we list all of our sources and and we sort of present that to the user.

Speaker 2: We just did the hard hard work for you. But uh so for for you to sort of um try and uh uh give comfort or information to a concerned friend or family member, whoever it is. So it's in multiple ways. There's not just one way of doing this. As I say, otherwise we we wouldn't exist as a charity. And I guess that in Ed that is the um the the ideal goal that we're not quite as uh as needed. But uh I hope that gave at least some answer. to your question there.

Speaker 1: And I've got quite, we should be a quick one from Michael who asks, do you use a dev to staging to product workflow?

Speaker 2: Yeah, that's right. We have uh we have a uh a dev site um and we have uh a uh a live site. It It doesn't actually go from dev to prod. It can go to dev as a nice testing ground and then we will deploy it to prod as well. But we we certainly have those environments to try and working out as many books as possible in a safe environment. I realize both me and Craig right now are using the prod site to demonstrate. That was mostly because there's a bit of a lag with dev and we want it to be as slick as possible for this presentation.

Speaker 1: I have got another question, but I'm going to send it to you after, John and Craig, because I don't think you will be able to answer it in one minute and people need time to move on. So just like to say thank you very much both for your time and doing a great presentation and thank you everybody for coming.