Multi-lingual websites in Wagtail

Yeah, let you do it. So yes, my name is Paul. Not gonna bore you with that. Just trying to keep your attention alive a bit more until we finish up here Actually my original talk was planned to be on a slightly different subject, a bit like uh my previous uh uh Speaker, but then I found out that the topic I was going to address was about translations in Wagdale. It's come very far over the last couple of years. The documentation is excellent as Tom rightly commented So I have to speak into the microphone perhaps. So I changed tag

a little bit and decided to take a different slightly different approach. I still want to talk about multilingual websites in Wagtail. What are the issues involved? What are we actually trying to achieve here? So this is not going to be a very technical talk. conclusions that we might come up with later on, but it's about non-technical perspectives. So what are we actually trying to achieve? Who are we talking to as a um who is the user of the website? So a lot of what we heard today was about uh facilitating the editors. But I also want to take into account the uh

Well the visitors of your website, so what are they looking at? When we visit a website, we are actually participating in a very old tradition. We are telling each other stories. We are trying to communicate something, explain something, share stories about. Things you're trying to sell maybe or some other topics that are of particular interest to us as human beings So we like I said we tell stories around the bonfire. So how do we do this? If we are speaking different languages. Of course we have the The whole internationalization setup within Django

and Wagtail. And that setup likes to use very simple technical solutions So that's why why they use this whole I-18N patterns URL structure where each language gets its own prefix. Well, I'm saying language, I should say locale. But for simplicity, let's just say languages here. There are some issues with that. And also it just looks ugly, but maybe that's just me. So I think that any page should be available on any URL. It would be nice to be able to have the same page in different languages

And just make sure that it looks nice. So I I give some simple examples. But this can't use of course the AI uh the internationalization pattern structure. So you have to come up with a different solution. And there's all kinds of solutions going around there. I know I've built a couple of them myself. And it's always like you're trying to reinvent the wheel here. Also, um both from a um an editor perspective and from an end user's perspective, but especially the editor perspective Um we want to make things easily understandable. Uh if you have to try to explain to uh your client

Okay, if you want to build this page in this language and you have like five, six, seven languages on your website. Please make sure the page is properly translated to other all the other languages. And people generally don't understand that. Of course there are again always solutions around this But that's currently the state of the art as far as I know. Of course I'm not a very deep expert. I work with Wagdil already for many years. With Django, Python. I work with other systems in Python before that. But it's always a challenge to explain to clients how they should use the system And if you have to explain it, there's basically a problem.

But first lesson of course is finding out what does the client actually need and what do they expect One of the issues that I keep running into here is that it's it's very tempting to start from a SEO perspective. Of course, this sounds really nice for people who are involved in marketing, but I have some issues with that myself personally I don't like the way that Google is developing itself. The impact they have on the market, on the work that we do, the reliability of the results these days is also an issue, of course.

And so my point is let's forget about SEO for a moment. We don't want to be found, we want to be understood, we want to be easily understandable as a website. for people from different backgrounds, be it different languages, but especially also different cultures. But not everything is terrible when it comes to SEO. Some lessons were learned in this period that we've worked on this. Strive to improve this. So semantics do matter. So it is important that the structure of the website is correct. That the HTML that you generate makes it easy to understand for

humans and maybe also for non-human readers to to follow what you're trying to explain to expose. So there are some very simple tricks to to give some connotation to what your website is trying to achieve. So use a lang uh language hints Point to uh different versions of the same page in different languages. And of course, like I said here, don't forget your site maps. They also need to be included in any stuff like that. So some basic rule rules of thumbs that we have to follow when we build multilingual websites is be useful. Be useful.

Don't make the visitor think too hard about finding content. Work for humans. Think about the customer that's visiting your website and try to place yourself in their position. So what does your website look at look like when they first visit your website? And haven't yet had a chance to see what it what it actually brings them. So like I said, never a good, never a second chance to make a good first impression. So for instance, this is just a an example. This is an Arabic only website. This is the Al Jazeer website, it was just the first one that I could find

It's if you don't understand the Arabic script, it's a bit difficult to understand what you can do to get a readable version. This is another example. This is the Arabic website of CNN. This is even worse. Everything is right to left, except for the cookie wall. Imagine that you're from I don't know an Arab country and you're opening this website, you're interested in information, and you see this cookie wall, but you don't speak English. Nice eh? But of course, to be fair, these are not meant to be multilingual websites.

And open it for the first time and don't speak your language. Can they find a language switcher? Do you offer them a language switch? How do you make it possible for them to visit a version of the page that is actually readable for them, understandable? And of course in the case of the cookie wall, do they even know how to close the cookie wall? Because if I go to the Ultra Zero site, I see some buttons and I can click on them, but I have no idea what I'm doing. So like I said, first impressions they matter. This of course the the basic principle of least surprise You don't want to make things difficult or surprising for visitors who come to your website

because what will they do? They will just leave. Most people don't read English. And I know it's a shock to hear, but most people in the world, they don't speak English. They don't read English. Of course the same is for all the other languages or scripts, Arabic, Chinese, Japanese. Most people don't speak it. So if you're Of ex of course, unless you're in the US, you don't have to deal with that. You can just assume that everybody speaks uh American. But the rest of the world we have to especially in in places like Europe But also I assume in in Asia , Latin America, Africa, we have to take multilingual content into account if we want to reach

people from different audiences. So what what can we do? Well the first thing that we have to do is to look at what the browser used by the visitor indicates about their preferences. So in general modern browsers always send in accept language header. Which indicates okay these are the languages languages that I'm able to read. Um There are some fallback mechanisms you can use here , but I'm going to go into that later on You can also try to redirect immediately to the correct version of a page if you have it available

based on one of the languages mentioned in the accept language headers. Or you can offer a manual redirect like a pop-up, hey I see you speak French but no English. Do you want to open the page in French? Perhaps? Of course, in French you ask them. You can also offer an optional geofencing redirect. So hey, you're from China, perhaps you want to visit the Chinese website or um You're visiting from Egypt, perhaps you want to open the Arabic website of Al Jazeera instead of the English website, etc. But you should always take it into account. What does your website look like for first-time visitors from a specific cultural and linguistic background?

offer them a way out or else they will take their own way out. So what is the conclusion? I think we should somehow support redirecting pages based on the preferred language of the visitor. This is not the case automatically. I know Django AI uh Django I18N tries to provide some support for this, but it's in my opinion it's incomplete. For simple a simple example, it will try to redirect you, as I mentioned before, to the same page. Um But that might not exist, that same page in the other language. So that might what would result in a 404.

So you would have to try to look up, does the page actually exist? And that's not something that is easy to do in in Django at the moment. Well actually I did find a way to do it in Wagtail this morning, but I'll come back to that later. So we have to rethink what we're trying to achieve. And I'm getting to the point of my my presentation here. We want to start from first principles. That is to say, we cannot assume that if it works for you, then it's good to go. Also, best practices as we sometimes call them in the business, in my opinion, are often just the solutions that were acceptable yesterday.

That doesn't mean it's the best we can do. We can do better. That's basically my point. Let's do better. So first off, know who your who your audience is. It's not just the client. The client might be wrong about their audience or of course they might have a deep insights. It's you have to understand, you have to talk to them You have to maybe even use like tools like A-B testing or uh browser spyware in the browser to see where do people click Something that is normally used to uh um uh improve um um conversion on on web shops, etc. You can also use it to see

how are people clicking around your website. Maybe they are looking for a translate button or a switch to another language button Okay, as mentioned before, we always have to determine what is the preferred language of the visitor, either through automated mechanisms like the accept language header. or by just asking them. Take into consideration both first-time users, first-time visitors, I should say. and returning visitors because a returning visitor might have a different language preference than actually specified in their accept language header. Of course for first-time visitors you know nothing.

All you know is the header. And in the header it says, okay, I like these languages, but for returning visitors, they might have made a different conscious choice. You have to take that into account. So again, there are many ways we can handle these problems. And next up is how do we Make that result in a desirable outcome. So how do they how do we we guide our our visitors to a version of a page that's actually interesting for them, readable Of course, as always, anything is possible. We just have to find the right mechanisms, the right policies

, and see how that works. In the past, we've tried many different solutions for this. Uh during the sprint we saw also some solutions being um explored in this regard. So for instance you can have a single URL but give the people different uh content For instance, depending on a on a get parameter or a cookie, show the same page in a different language, in Dutch, in English, in Spanish, in French, I don't know, in Japanese, whatever. That's a possibility. Of course the CEO people will start to tear their hair out when you speak like that.

But it's still technically possible and for human consumption it's perfectly acceptable. My preference would be to always redirect first-time visitors to the correct version of their of the page they are visiting. So if I have specified that Dutch is my first language, please always redirect me to the Dutch version. of a page even when I visit a page that has as its uh a default language English for instance But only if that version actually exists, please. Another idea is to always offer a language selector.

That could be like in a in a modal pop-up or in a language selector somewhere in the screen. Doesn't really matter, but just make it clear that it's there and make it visible, make it understandable. And don't forget to translate the values in there, of course. But the most important part that we have to deal with is the accept language header. It's a well-established industrial standard to use this header. It's well defined in RFC. It allows you to really specify on a really deep level what languages are acceptable for the visitor. With a weighted

value attached to each language. This is just what I copied from my browser at the laptop at the moment. It's just an example. But it already indicates that you can have a lot of languages there. What I often see in these when people actually try to use this, and it's always kudos for those who do , they forget to consider the secondaries. So there might be more language languages mentioned than just Dutch and English or French and English or German and Spanish. I don't know. But always consider the secondary. So if the first language is not available, but the second one is, please redirect me to the second one.

And don't say I don't have the page in your language. It happens. So my idea was originally to speak about how do we do multilanguage support in Wagtail, but That's like a done deal already. As we saw in the sprint, lots of people working on this. It's it's a very good system already. But what is missing in my view is behavioral strategies. So how do we actually show the correct content to the visitor? I mentioned before that I was thinking about creating some kind of a new middleware for that in Django Wagtail. And that's but that's just part of the story.

It's actually not as complicated, I think. I haven't had time to actually work on it this uh this uh conference but um uh it looks quite feasible actually to do it. Uh there are some basic rules here. We don't want to implement a single strategy for everybody There is this old adage in the in the Unix world where you have to separate mechanism and policy. You have to build the mechanisms and allow people to design their own policies on top of that. There is no one solution. But if we build the right tools, I'm sure everybody can design their own solution for themselves. So I have some ideas that that we might consider As I said, we can use the middleware is a is a great place to detect the the preferred language

of the of the visitor. We can also offer manual manual selections like I said in drop-downs, in modal pop-ups, etc. We should consider always redirecting to the correct language after we decided what that was, but only if the page exists And if the page if somebody asks for a page in a language and that page does not exist, please render another page in another language. So stuff like that. I came up with a simple example policy chain here where I say oké let's first look at some kind of cookie where people actually have indicated that they always want a certain language.

If that doesn't work, you might want to look at uh the path if that's relevant for your situation. And if that doesn't also uh give you an acceptable language, uh use to use the header. But always use the header in the end. So we might even tweak that a little bit. So we say okay, in some situations I do want to use the path. And then if there is no cookie, uh redirect based on the path. And then set a cookie, please. Always set a cookie at the end, by the way. There's a default language that doesn't have the um the prefixed, which is a feature of I18N patterns.

I would treat that the same as with a prefix by the way personally. And of course, when there's a cookie set, always use that. And that's just an idea. And again, if you ignore the I18N path, make it even simpler, use the cookie. And if not, use the accept language header. Um but how do we actually do that? Oh wait. Um I missed something. Nope. There's two two places to do this. I found out that of course you can use the middleware to do the initial detection, which is fine. I mean that works. But redirecting from the middleware, it's a bit tricky.

But there is this feature in Wagtail. It's called before serve page. And you can use that one. It's actually trivial. I was looking at in the wrong spot before when I talked to Matthew about this Before purchase serve page is the is a great location to see okay you're asking for this page say in uh um in French But your preferred language header does not contain French. Redirect them to their preferred language. But in the prefer in the before served page hook, which is a hook by the way in Wagtail You can say, okay, does this language translation actually exist and is it published? And if so, redirect them

to it. So I think that if we bring all this together, we can build websites that are flexible in their behavior, that always offer the correct translation to the person involved. Maybe I can work a bit on that more and explain a bit more in code and offer perhaps a simple package containing both the middleware that's involved. and um an example of the of the of the hook of the before served page hook and see if that resolves the reinvention of the wheel as I see it that I that we constantly have to do for each website as it's done at least by us at the moment.

So that's basically it. The end result should hopefully be that we are all kumbaya and uh everybody understands their websites again We don't end up with 404s or undecipherable web pages. And hopefully that will help the Not just the mechanics of the translations because those are excellent at the moment, but also the behavior of the website a little bit. So thank you.