You've been caching your content website wrong

Okay, here we go. So we are going to talk about cash again. Um so um it's pretty independent from uh previous one we've uh we've had Um in the sense that we our scope is more on the browser slash proxy side, um slash uh CDN. But uh with a specific approach that is um the uh e tag mechanism. Um so it's not um time based and uh it's a different approach that suits uh the suit like the needs that I had at the moment. Definitely not for everyone, but I'll show you the pros and cons for it. So the idea of the E-tag mechanism is um something that saved my ass quite uh a couple of times.

For performance is pretty easy. When you reply to HTTP query, you say the tag of this query is uh whatever, random string, you decide. And then when the server asks for the same uh when the browser asks for the same resource again, uh they say, okay, tell me if it has changed since uh last e-tag. And uh you say yes or no. And if no, you just say it didn't change and the browser is like, oh fine, okay, I'll just display the one I had before. So There's a request to the server, but you don't need to generate the the answer if the answer has not changed. And that's a great way to put the burden of caching on the client and not on the backend Like you don't need the Redis or whatever to cache stuff. Support for um

e-tags is great in browsers, great in Django. terrible in uh proxies because it's actually very hard for proxies. Um like imagine in the scenario the proxy has the thing in cache so There's a new client that doesn't have the thing in cache that asks the proxy give me uh this page. The proxy needs to because the proxy proxies, right? So it's going to relay the headers coming from um the client So if the client did not put the if not match header in the request, then it needs to go get the if not match header that it has somewhere else, get a 304 from the server, and then reply with a 200 and the tag to the client. Uh if you did not understand it doesn't matter, it's just to illustrate the point is is pretty confusing. And you see here the

truth table uh that you need to implement, and that's a simplify uh simplified one, right? At the um proxy uh level. So and then I tried many many many different options uh that didn't work out at all. Um so Nginx is uh complic I mean Nginx is uh I'm sure you can make it work with Nginx one way or the other, but I just Did not find it. Squid just ignores the thing altogether. Varnish uh almost does it but uh not exactly. Cloudflare ignores uh the tag, even though they say they support it. Um Apache HTTPD um does not. I tried also um you know all the CADI type uh all this well it didn't work out. In the end I uh like Apache Traffic Server, it's uh

you you go to the website, it looks horrible. Um and then you're like oh my god, this this thing must you know that date from uh Paleolithic era or something like this And it's um it's not going to be nice, but actually it was uh supporting the thing natively and uh very easily and uh So I'm I'm very glad it pushed me to discover this thing uh but more on that later. So um it used to be uh commercial so basically what happened with uh this uh traffic server it used to be commercial software um made by a company that got acquired by Yahoo for something entirely different and they did not know what to do with it so they just said well let's open source it that's it. Uh so it got open source given to the Apache Foundation. Um and it's um out of all the things I've listed except for Vanish, I think it's uh the only one that is intended to be a reverse proxy.

That is made for this use case because all of them it's uh HTTP servers that accidentally do reverse proxy. And um it has auto-comparation of uh content with simple plugins like uh GZ, Broadly, WebP, and so on. um and also different use cases like uh you can authenticate against s3 bucket and have public and signing URLs and stuff like this. So really great piece of software but uh let's go on the attacks Um so we are basically um having two things that is expiring things with it or expiring things with um the time, right? So the attack you still need a round trip to the server, so like the latency is maybe not going to be great. So of course um if you go through uh CDN uh you will do the round trip to the CDN and the CDN has a keep alive connection

to your back end so there's not so much handshaking happening but still you you need to reach the server. Um the in the tag the real uh the content expires in real time like if you change the value of the tag, um it's expired automatically. So um that's that's really convenient if you don't want to I mean if you want immediate um you know, eradication of uh cash. Um if it's time based you need to call an API or wait for the the amount of time And uh which is uh in my opinion like for example I have um the website for which I I did this has uh lot of content. Um expires pretty like that is edited pretty rarely. But if I edit it I want it to change quickly.

So I don't want to put a one week uh expiration on the pages when uh really I want them to to expire automatically Uh the attack is conceptually simple, uh not for the proxies, but uh for you know the browser, it's simple. Um But um for the time-based expiration you need to call APIs and deal with vendor specific uh specificities to you know expire the stuff. The e-tag is fast enough. Uh I had two two second generation uh for my pages and now it's about a hundred millisecond. Uh so that's uh good enough. That's not the best. The time-based will you know no need to run trip to the server. So It's going to be immediate. The e-tag is going to be longer. Um the e tag also works for the browser, so

if the browser and I mean the browser will also cache the different pages. So depending on your use case that can be interesting. Because you cannot uh expire remotely, you there's no API for the browser to expire the stuff, right? So um having the tag in the browser to validate the content it can be pretty interesting. So it's really um a question of how do you want to expire your content and what is the performance trade-off you want to be doing. And in my case it's definitely that was interesting. So how do I implement this? So basically I'm going to show you uh snippets of code that are integrated with uh model W that I uh talked about earlier. But uh you can it like it's the concept you can uh re-adapt

uh in any way you want is just like the different uh things to consider I'm going to list. So on the Django side, um what we decided to do is have one e tag for the whole website to do any edit in the website. and it expires everything because then you don't have to because otherwise you know you change the title of a page, then you need to change it in the menu everywhere. or you change something, you know, it can be a read more section and so on. So there's a lot of things that can have, you know, there's a lot of possible repercussions. So since I don't edit the website that often, uh I prefer just to expire everything when there's an edit made and uh let the cache rebuild itself. So just needed to hook up to Vogtail

to clear the cache. And the clear cache is just so I'm using basically the Django caching framework. just to store the e-tag value um and and that's it. Uh how do you generate this e-tag? Okay. So the tag is anything you want, it's any string you want. Uh so it just generated random enough string And that's it. One thing that is important however is to also expire the tag when you rebuild your um website because if you change your source code And the design has changed or whatever, you don't want to be uh serving an old uh version of your source code. So it's not just the content, it's also what the page looks like. And that's uh pretty easy with CICD to put uh you know

the build ID as an environment variable. So that's what we do. We have the build ID as an environment variable and we we store it there So this key function allows uh allows us to store the e-tag in the Redis and then we get this key. And so basically if we change the build prefix, we are going to generate automatically a new e-tag. And the generation of the tag itself is uh just a random string of uh I don't know fifty characters length or something like this. Just one detail here, you see the W slash in front of the E tag, which means it's a weak E tag meaning that you can apply content transformations on uh the content uh and you'll see later where it's uh helpful Um so then you need to inject this e-tag into every page. And in the context of Vogtail

today it's uh a bit uh artisanal, right? You you uh uh you do it as you want. So what we decided to do is that we can have have uh one model that is um parent uh in terms of class over all the other class page and pass um page classes uh in which we added two different uh attributes that is use e-tag yes or no and cache control headers uh where we set literally the content of the cache control headers uh and it's in the we put it in the promote section in the in the pages. Um so then we have a decorator that we use to decorate the serve method. Uh that like it's conflicting a bit with a rotable uh page mixing, so it was a bit uh complicated to get it uh

right. But basically the idea is that we just uh invoke the e -tag decorator from Django and wrap the serve method with it and that's it. So this uh this attack for this um attack decorator basically will uh handle for you the checking if not match, if it matches then this and otherwise not and so on So it's all provided by Django. It's just some wrapping and uh plumbing. Uh then you need to fix the HTTP headers. So I discovered uh doing this that Django is uh actually very smart in dealing with uh HTTP headers. because the like if you put um if you use the session for example it will automatically add in the vary header so that is the header that uh tells on which other HTTP

header you need to look to know what is your cache key. uh it will add the uh cookies to the very header if uh you use the session but otherwise not okay so and that for lots of uh things uh the JZIP needleware also will uh add its own stuff and so on. So it's pretty interesting. And uh there's just one thing that we need to do from that. That is the how to say, uh to check if the cookie is indeed in the very headers and if so to mark the cache as uh private no matter what because otherwise it means that we are potentially serving some uh private content uh to in in the cache and caching it forever so that could be annoying. So my advice is do not enable the JZP middleware from

Django because uh if you put uh traffic server in front, uh traffic server is much better at doing this than uh the middleware from Django. Um and um yeah that's it. So but the main point of this middleware is just to mark the cache as private if needs to and that just adding up with the other mechanisms of Longo Um there's another thing that is cash busting um the the thing. So if you remember the previous talk, right? There's I say it's a proxy. Like if you ask slash foo on the front end, the front end will ask slash foo from the API. But If the front end that runs in the browser asks for a slash foo, um and that it needs to be the the answer from the the back end

and not from the front end, uh not the pre-rendered you know, thing from the front end. Then it can be very confusing because the same URL and it's completely different resources. Uh and uh you could deal with it with a very header, but actually you could not because lots of uh m uh software d uh ignores the very header. So bottom line is if you ask slash foo from the front end, then it's translated into a slash underscore underscore row underscore underscore slash foo. uh URL, so it's just a prefix. And in order not to trifle with uh jungle roots and uh YTL routing and so on, I just modify here the path on the request. Object so that everybody thinks it's just uh like a request on slash instead of uh slash row and that's it.

Then on the next side, what do we do? So if you remember, the next side is a is a proxy that is uh adding JavaScript and CSS. But um what you need to do when a request comes in, you need to look for the if non-match header and put it in your own request to the server. Otherwise um the whole mechanism is uh broken. Same thing, you need to prefix your URLs with uh underscore underscore row because otherwise uh you will not um you will have the caching issue I was uh talking about And then you need to proxy uh back, so when the server replies, all the different uh cache control headers. So the vary, the tag.

cache control and then if the answer is not modified to reply as well not modified and uh basically next um we create an error uh because there's a way to to deal with it in a next, to say okay three or four not modified um the same as if it was five hundred or uh 404 or 301. It's um non-200 HTTP code basically and we are replying uh not modified. And the browser accepts it, so it's fine. And we there's no rendering process happening in uh next type because we just short shortcut it uh like this. Then how do you configure the forward proxy with uh the cache, right? So as I was saying, um uh traffic server is very easy to use, but the request.

config file is horrible. And when you open the Debian uh version of this configuration file that has all the options already listed and commented, it's like a a wall of text, like there's no space, it's just um you know uh suffocating. So I created the Docker image of Traffic Server that is making it easier to use. It does two things. yaml, uh uh the records. config file, you can fill it up in uh YAML That is much more readable than the original syntax. And you can also interpolate environment variables in any configuration file for um traffic server, so that's you know more the Docker way of life. So we have uh different files. So let's start with um

remap. config, okay, that is uh And what you're going to see here is the full configuration, just so you understand it's very easy to configure. So basically here you map the URLs from uh what's you know you're going to get as an input and you map it to an origin server. So you say everything that is on slash back goes to my uh API and everything else goes to my frontend. Super straightforward. Here you see Django templatey stuff. It's not really Django template but uh inspired syntax. And that is the Docker image that does this. It's not uh stock with a traffic server. Um you have then the records. config file. So I'm not going to explain all the options, but uh three are pretty important.

Uh first one is to limit the RAM used because um otherwise you're going to like if you deploy it on Kubernetes and you did not put any limit on this, uh it will use all the RAM available for you know all the other containers. So You need to not forget to limit it. Um of course I thought of that before pushing it to production. Um then you have to configure the DNS resolution to be uh looking for the default uh domains. Uh again if you deploy in Kubernetes, uh that's how you know the internal URLs of uh API and front end. So um by default not looking into that And then another one that I I find extremely convenient is the normalize AE, that is uh normalize accept encoding because

Um the way the vary header works is that you say it varies depending on this header, this header, this header, but it's not semantic inside the header. So If one client says uh very GZIP uh says uh accept encoding and says gzip broadly and the other one says broadly gzip, it's two different headers. So you can have an infinite amount almost of um you know uh variations. But since you don't even understand all the encodings possible and so on, it it doesn't make any sense. So what it does is that it will uh sort them in a specific order and it will um uh you know always accept Uh I mean remove the ones that it does not understand. So basically when you configure it, you say okay, um you you understand

uh gzip and uh botly and uh and you normalize it this way and this way your page will be um compressed well more on this later um so We are going to use two plugins uh that are compress and uh header rewrite. Okay. Um so first one header rewriting is just to add some uh some stuff. So there's a first thing to say if it's a hit or miss in the cache. And then remove remove the server and expowered by headers because uh it's well known that it's a great security improvement to hide the name of the server. And then we enable test compression and that's really an amazing uh part of it because you get uh is basically what Cloud Cloudflare

does, but it's like the open open source version of uh Cloudflare Um so you say the algorithms that you want to handle, you want to you say you want to cache them and uh generate them, and you say which uh MIME types you want to compress, and then that's it. And it's it's blazing fast, it's much faster than uh GZP implementation uh of Django. It's uh doing broadly in real time. If you try to do it in Python, it's uh it's very very slow. Um like I mean to give you an example, um broadly compression. on my pages it was 700 milliseconds when I tried to do in uh in Python side but uh there it's uh you know insignificant so it's uh very good at doing this and uh I highly recommend uh using ATS also for that. So as a conclusion,

this is based on model W that I presented earlier, but even if you're not using it, obviously you can uh inspire your yourself from the different checkpoints. Like really each each slide is uh a requirement that you need to follow if you want to implement it yourself. Um the tag is very convenient, at least for some use cases like mine Um and uh it's especially convenient if you want to expire pages immediately from and the browser and the CDN. Um of course it has some drawbacks and I'm also very surprised uh in this process to have discovered the traffic server that is um actually a great piece of uh of software and very easy to use despite um some uh looks that are a bit uh uh frightening.

And that will be it for this presentation.