Using a custom template loader at scale by Dane Hillard

Uh

Speaker 1: so thanks for coming everybody. How's your first day? First day of talks anyway. Great. Good. So I'm the only thing, excuse me. Got a bit of a cold that I'm coming off of. So I'm the only thing holding you from uh the talks in the rest of your evening. So thanks for uh thanks for bearing with me. Um And this talk was actually originally accepted as one of the deep dive talks, but due to some of my own gross incompetence, I had a scheduling conflict, so thanks to the organizers for uh accommodating me uh today instead. And thanks to all the volunteers and uh to those doing the captioning up here and um for making this conference great. So I'll talk a little bit about myself,

Speaker 1: give you an idea of who I am and how I came into Django and how Django plays a role in my life now. First, this is kind of who I am on the internet. Uh so you can find me on the web or on Twitter or GitHub, uh, whatever your sort of preferred medium is. And throughout the talk, my Twitter handle is there on the bottom. So if you have a question or something that you don't want to lose track of, feel free to ask me as I'm giving this talk. So I work at Ithaca, and Ithaca is a nonprofit organization in higher education space, and we do uh sort of a lot of things helping with academic research and uh digital preservation of documents and things.

Speaker 1: And specifically I work on JSTOR And JSTOR is an academic uh database, online database, as well as a research platform. And I guess the way that I came into Django was sort of by by personal uh use first. So I am a photographer and uh this photo in the middle here was taken in Mexico a couple weeks ago where I received the worst worst sunburn of my adult life. So visiting sunny California was obviously the next natural step. And the photo on the on the right is also in Joshua Tree, which is not too far from here.

Speaker 1: But I kind of had tried a couple other frameworks uh and a couple other languages um along the way in building my in building my site. I needed something that could allow me to upload photos and organize them in different ways and maybe do blog posts and uh let customers contact me and things like that. And none of these frameworks that I had been trying kind of resonated with me, but I ultimately gave Django a try after having seen a little bit of Python and have kind of just been going with it ever since. Um and it's been a joy to work with and I try to use it where I can. So uh back to back to JSTOR. Around 2014 is when I joined the organization.

Speaker 1: And at the time we were on this big proprietary CMS. And it was a little bit it was a little bit difficult to change. I mean think about uh developers doing these like day long or multiple day long trunk merge activities um trying to get everyone's changes from the last however long it had been. uh into the next release and um people scrambling to make sure their stuff worked so that it could get into the next release um because who knows how long it could be till the next one. And there was also sort of low visibility into problems that might arise. So anytime something was wrong, we had a lot of digging to do. To be able to kind of surmise what was going on. And so around 2014, I joined and we were in the middle of a project to

Speaker 1: get out of this situation. And So we were at the time uh breaking this big CMS up into a variety of microservices. Um And moving a lot of those services onto an AWS platform. And that allowed us to do sort of these microservices for the back-end as well as front-end consumer applications. consuming those APIs and then do a lot better things like application health monitoring and continuous integration and auto-scaling too. So this is kind of where Django enters the picture. Um and uh I can it was kind of a good pairing at the time because I had learned Django and I was I had been using it for

Speaker 1: uh a year or two uh at the time and I so I knew enough to be dangerous um and then I also wanted the challenge of of learning a lot more of these detailed pieces of of Django that I had yet to explore. So I signed on and Uh kind of the approach that we that we took with our applications is to not just have one project with many applications, but actually to have several different distinct projects. And this allowed us to do better fault tolerance and better independent performance profiling and auto-scaling. So when say Google crawls the website and is hitting the the article viewing application heavily, we don't also want search to break under that heavy load.

Speaker 1: So we wanted to maintain some amount of of uh Isolation between these applications. And then at the same time, we also wanted them still to share a number of things. We wanted them to all do authentication the same way, uh handle permissions the same way. doing look and feel and search forms and all that kind of stuff the same way. So being able to centralize some of the business logic. was still was still of big value to us. And so uh it kind of whoops, sorry. It kind of ended up looking like this. And uh so we we made this installable app that had all of our shared templates.

Speaker 1: And We would install that app into each of the each of the consuming front-end applications in which we needed to use it. This might be a practice that you're pretty familiar with. If you use Django, you install a plugin, you add it to your installed apps. You can use all the templates from it and so on. So this felt natural at the time, and we were pretty happy with it for a while. But as we kind of grew and started breaking out more pieces of the CMS, we ended up with more applications. uh more distinct applications and ultimately um the problem we ran into was that these changes to the look and feel required uh deployment to basically every consumer right um And so if you imagine that each app has to have this navigation at the top, um

Speaker 1: if you need to add one link to the navigation, You have to deploy all 37 applications that need to show the application, uh need to show the navigation rather, and that wasn't great. Um So we were starting to feel these pains and didn't really didn't really have a great idea where we wanted to go with it next. But the the way we sort of approached this was to write down uh what we wanted out of it and what what it needed to achieve and then we could go from that and try and find a solution. So the desires really were for all these consumers that we have, all these front-end applications, to uh be able to receive these updates uh without a deployment

Speaker 1: in as near real time as we could do it and in some way that would not actually affect the page load performance for our users uh in a And so this kind of started to sound like a service that would that would that would serve templates. Um we didn't know exactly what that meant at the time, um, but that's that's kind of what we wanted. We needed something that was a central place that all these applications could go and ask for a specific template and then receive the content of that template. And then that templ that service could be the one place to uh to make those updates. so that all these apps could then reap the benefits of those updates.

Speaker 1: And after quite a bit of digging and thinking and uh reading a lot of documentation, we Settled on custom template loaders. And uh so the rest of this talk is kind of gonna cover this concept. And so we'll cover a little bit about what these actually are. and how we use them at JSTOR and some of the things that it helped us achieve, and then also some of the things we have kind of yet to do to reach where we're trying to go. So, first off, uh we'll kind of go into what they really are. So, this picture might be something you're used to if you've used Django for some time now.

Speaker 1: Um You've seen it probably in some other form, either in the Django docs or a number of blog posts kind of cover this, but essentially there's these layers of of the request lifecycle that happen when When someone requests a page on your site. And so it starts by going down through the web server and gets translated through the WISGI layer into something that Python can understand. And then there's this middleware layer. uh that allows you to intercept that request as it comes in, make some changes or add information to it, and then uh within a view is where you have kind of your your business logic happening, and then you ultimately render a template, which flows back up the stack ultimately into this response that users see. And This maybe is nothing new for you, um, but what we

Speaker 1: what we really wanted to attack was like the part that happens underneath that. So that's the part we had never really seen or thought about before. And it turned out that template loaders are that piece. And like most things in Django, it provides a very reasonable and pluggable and um easy way to kind of get to that piece of the request lifecycle. So uh it the the template loader's job is ultimately to accept a template name and then return that template's content. Um and that going back to our desires is kind of kind of where we wanted to head. So there are a few template loaders.

Speaker 1: installed by default, uh well available by default in Django and installed by default uh if you haven't specified your own. And Uh the file system loader will look for templates within your project, and then the app directories loader will look for template directories within each of your apps that are installed. And then the cached loader will wrap some number of other loaders so that subsequent requests for those templates uh don't involve additional processing each time. And uh There's a there's I think a couple others that are also available, um, but they're in the docs if you'd like to learn more about those. And Again, its main job is to

Speaker 1: you know, given a template name, return that template's content. And so a simple uh a simple way to do that is to subclass the base loader class. And there's these two methods, git template sources and git contents, that uh are are sort of the meat of things. Um And the get template sources method is meant to get all the possible locations where that loader knows it could look for a template. It doesn't actually have to check that they're there, but just return a list of locations where it might be. And then get contents will iterate over those locations, check if the template is available there, and if so, return the content. And um If it goes through this process and ultimately can't uh

Speaker 1: can't find any template, then it raises a template does not exist, which will then kind of defer to later loaders uh to try and find the template being asked for. So um this is this is uh kind of the the flow that we wanted to intercept. Um this Is not good. Um I don't know what that slide was. Uh I think this was just to I think this slide was just showing um that you can add that loader into Django by uh There's like in the in the settings. py file, there's the template setting with an options setting within that, and then there's a loaders option that you can configure a list of.

Speaker 1: of loaders. So you would uh just point it to the the dotted string uh path of that dotted path string of that loader So uh how did we how did we use this? How did we get what we wanted using a template loader? Ultimately it looked like this. So uh when we when we get down into that template rendering process. We are now have now have this way to intercept uh the loading process using this loader class. And what we can do in that loader class is Check if this is an a a template that we are interested in loading from this remote service.

Speaker 1: And if it is, then we can check if that template content is already in a cache of some kind. And if it is, we can return that template content that's been cached. And if not, we can fetch it from the service, which will cache it and then return that content. Um and if it's not a template that we care to fetch from the remote service, we can just raise template does not exist because the future loaders uh that are the the default Django loaders, file system loader, app directories loader. Um will then be able to to look where they normally would in the local file system for those uh for those template content. So the templates that we want to fetch from the service, like the navigation and the layouts that we have and the footer. uh will go

Speaker 1: f be fetched remotely and uh all the sort of app specific templates and things like that can be fetched as they normally would be. So that part on the right there is kind of the get contents part , and the part on the left is the get template sources part. Referring back to that loader API. And architecturally this looks Somewhat like this. So if we have an articles application, uh the first thing it will do when trying to render a template and it knows that it's one of these remote templates that it needs to fetch, it will go to the service to check if it's available. If not, it will call this template delivery service and

Speaker 1: that will check the database for that content. and retrieve it and then put it in the cache and then ultimately return it to the application. So uh uh basically we wanted to check obviously if this meet met all of our desires. Uh First of all. And so consumers could receive those updates without deployment. Anytime we wanted to put a new version of a template up, we would simply add it to the service. and that update would propagate in a way that those those apps could ask for it and get the updated content. In near real time, pretty close. The biggest thing is the network call to the cache or the network call to the service.

Speaker 1: And those things can be improved with with bigger machines and horizontal scaling and without affecting page load performance. So with heavy caching and things like that, most times If a user requests a page, they'll be getting cached content. And the increase in page load time there was fairly negligible for us. So it kind of checked all the boxes. And some other things that it kind of got us were this ability to do per user configuration for testing changes to those templates. So we have uh kind of this live configuration system that we use for feature flags and setting string parameters and things like that. So coupled with this, um

Speaker 1: We could pretty easily enable new versions of templates just for ourselves for testing purposes and do things like browse the site with changes to the navigation, the footer, the layout. everything like that. And it made it easy to test impact of changes like uh adding preload or pre-fetch uh pre-connect tags uh to our pages and things like that too. So uh this was a pretty nice kind of fringe benefit. And we built a a system around making this easier for ourselves. Which looks like this. So you can kind of select the layout that you're interested in updating, uh specify which version of the template you want to add.

Speaker 1: make a description for what that change actually does, upload a new file and uh put your name as the as the person who uploaded it just to a bit of an audit trail. Um and then uh you get a list kind of of all the available versions and you can enable that version for your local session or you can enable that for all users once you've determined that that uh change is desirable. And uh situation permitting, I will show you a live demo. Let's see. Gotta make sure I'm on that screen. New tab. The new tab opened on my screen.

Speaker 1: So Normally JSTOR. org shows the JSTOR logo, as you might imagine. But well, this is gonna be difficult with a separate screen. I will show you after, in the interest of time. Uh I'm happy to show you how this works. So uh Going back over here. This also ultimately allowed us to do parallel development of a whole site-wide redesign. So we That's good. Um we about two years ago uh had an interest in sort of rebranding and and changing the entire look and feel of the site.

Speaker 1: Uh we wanted to do this Normally we want to do incremental development and deliver value as fast as we can and in small pieces. But with something like a project of that nature, we wanted to kind of wait until it was all ready and vetted out. and just sew so that we could flip it on for everyone all at once. And using that system that I mentioned before, we were able to really do that. that we one day were able to just turn on the entire uh redesign for all of our users uh because we had parallel copies of all these templates uh that were available and so we just selected all the ones we wanted, flipped it on, and it all worked. So uh we did face a couple problems. Uh one of which was this thundering herd issue.

Speaker 1: So uh initially we had cached our template content at the consumers, and we had only cached it for some limited amount of time. And Uh there were times when that content fell out of the cache and then every consumer went to the service at the same time. Uh and And service got bogged down enough that none of them received a response. And so new requests coming in from new users also called the remote service. Uh and it just spiraled out of control. So that was uh a lesson learned and we cached indefinitely at the service instead. And um the cache loader configuration was a bit confusing. It normally is only on in production, so when settings. debug is false.

Speaker 1: But then if you specify your loaders explicitly, you have to just make sure that it's still only on when you want it to be and off when you want it to be. We ran into issues where local in local development our templates were being cached, so we would make a change in the template and wouldn't be able to see that that update. just confused us a bit. So that file management thing that I showed you is also a little bit clumsy. It's easy to upload the wrong copy of a file or forget To commit a version of a file that you uploaded and things like that. So what's left would be to kind of improve that uh story around adding and removing layouts and templates. But also we may just want to use version control

Speaker 1: for that because it's already good at at doing exactly that. So thanks. That's it. I also need to mention that I'm writing a book. It's Practices of the Python Pro. It's the this code on the screen here can get you 40% off on that for the duration of this conference, but also specifically today. All of the Manning books are $25 for print books, so see which one of those might be cheaper if there's a book you're interested in. Um and that's all I have. Thank you

Speaker 2: Do you have any problems with your caching memory management? Like are you accidentally over caching items and keeping them stored or are you able to make sure that you're deleting everything? Um sorry.

Speaker 1: Good question. We cache these things separately from uh most other things. I think we have a cache specifically for this. Uh and there's a very small handful of templates that we're caching. Uh so it's uh the way we've chosen to do the infrastructure there, it's very easy to say, cash this forever. We don't care. Uh eventually things may f like if you reach The if you sort of saturate your cache, right, things may eventually fall out, but uh they'll be the oldest things, right? The least accessed. So uh we have not run into any any caching issues at the moment.

Speaker 3: So I hope I didn't miss this, but um for your individual apps, do you have a way that they can customize the templates they're requesting, or do they all just get the same? template, you know, every single app, if it asks for template A, it's always the same coming out.

Speaker 1: So it's always the same, but it's still a template. So there it it still has the places in the template that you can uh fill in information. Uh it has blocks you can extend it if you like. It's all the same way that the Django template workflow normally works. We're just getting that initial file from a service instead of the file system. Good question. Sure.

Speaker 4: Well, what did you use for feature flags?

Speaker 1: Uh that's another talk in itself, but ultimately it's it's another Django app. Um we rolled our own partly because of uh the way certain specifics around our authentication and and uh uh things like that work but um I would proc probably recommend like Django uh what is it called waffle uh I think there's one called gargoyle too maybe anyone Anyway, there's a there's a few libraries out there for you, for sure. There's also LaunchDarkly, uh which is like a fully managed uh service and API.

Speaker 2: Have you messed with uh Redis versus white noise and do you have a preference and why?

Speaker 1: I have never used white noise. Um white noise is for like static files, right? Um So I I suppose we could make that work, right? Because they're ultimately just static files. Um But yeah, if we haven't we haven't tried that. I haven't even tried white noise for uh for like personal apps or anything, so um that could be interesting to explore that. So for sure. Yeah.

Speaker 5: All right. Thank you so much for joining us. You can use this towel to get over your cold.

Speaker 1: Thank you.