Documenting Django Code in 2022 with Eric Holscher

Welcome everyone. My name is Eric Ulsher, as Lacey said, and this talk is about documenting Django code in 2022. That is this year for those of you in the future. So I want to start off just with a little bit of a story. I actually went to my first conference in about three years, about a month ago, in Django Khan in Porto. And I was sitting at lunch and talking to people and they're like We have all these frameworks for thinking about code. You know, we're here at a Django event. It's all about kind of this framework. And they were like, I really wish there was a framework for thinking about documentation. Like all of this work we've put into the tech ecosystem around code really feels like the kind of philosophy around documentation isn't as advanced as we hope. And it was very convenient that at that very conference, uh Daniele Persada um was presenting on a framework all about documentation

And that's part of what I'm going to talk about today. And it's really exciting that we actually are starting to have a little bit more of this work going into the larger picture of how to think about this stuff, right? And I think that's a big part of what's really helpful for people getting started. So, why should you listen to me? And one of the co-founders of Read the Docs and Write the Docs is kind of the most important things. Read the Docs is a documentation hosting service that works in open source as well as commercial settings. And then Write the Docs is actually a global community of people who care about documentation. So we've historically had events in Portland, Prague, and in Australia as well, in Melbourne. And so we have this global community of people who are kind of thinking about documentation and doing this work. So today we're going to focus on kind of four elements, of which authoring is going to be kind of

a little more than half of the talk, because that is where we spend more than half of our time, right? When you're actually doing documentation, a lot of the work you're doing is actually writing documentation, right? And then after that, we're gonna talk a little bit more about kind of the other stuff, right? How do we present it to the world? What does the design look like? How do we make the user experience a little bit nicer? And then how do we get it online and in front of the world? So that's going to be kind of the structure of the talk. But like I said, we're going to start off and spend a decent amount of time with authoring. Because that is that is the act of of creation, right? The act of creating documentation. So the first part we're going to start with is structure. And this is actually the framework that I was talking about. With diataxis. And so some of you might not know that name, but you have probably seen something that looks like this.

It's kind of gone through, it's got it gotten a name in the last year year or two, but uh Daniele's been working on this for a long time, kind of this framework of how to think about documentation. One of the things that I think a lot of people is kind of it's important to start with here Is this organized by user needs? So when you look at this, it's actually thinking about like when the user comes, when the reader for your documentation actually comes to that site. How are they thinking about what they're trying to get out of that documentation? And so that that part wasn't immediately obvious to me when I first kind of thought about this. And so I really want to kind of think through that. And so that that big four you know two by two chart's a little overwhelming, so I want to go through and break it down a little bit.

So one of the first things is what do you know, one of those axes is the study versus work. And you can think about this, you know, as the mode that you're in when you're going to engage with documentation, right? Are you trying to get something done or are you trying to learn something? So sometimes you'll be in, you know, study situations. You're trying to acquire skills, you're trying to learn a new framework, you're trying to kind of enhance your knowledge. And then after that, you're actually like doing your job. You're like, I just need to solve this problem. I have this thing I'm trying to accomplish. And that's kind of one set of concerns. To put it in a concrete example, attending this talk, you are studying. You are not actually writing documentation. But attending the sprints at this conference is much more like work.

You're trying to get something done. You're trying to have output. So this is the one I struggle a little bit more with is this kind of practical versus theoretical spectrum. Um practical feels pretty straightforward, right? It's like step-by-step things, it it's very kind of uh easy to kind of I I would almost like say it's actionable, right? It's like there is some action coming out of it. Whereas theoretical is more something that's happening inside of your brain. Like you're learning something. You're kind of internalizing a way to think about stuff. So we're going to talk a little bit more about each of these kind of with examples, but I think it's good to kind of understand, you know, what is that two by two grid. So we're gonna go through each one. So a tutorial

is practical and is study, right? When you come to a project, you really want to figure out like Does this work for me? You're having to kind of evaluate it and learn how it works. But it's also very practical, right? Like most tutorials are like step-by-step instructions. You're typing something into a terminal. You're actually doing steps. And so that is kind of the top left of that image. How-to guides are practical, but they're for work, right? So you've learned how Django works, for example, and you go in and you're like, I want to figure out how to do a model form or I want to add something that you have a field to the admin or whatever. You go in and that's where you go to a how-to guide, right? It's just step by step, it's very practical, but it's focused on work that you're doing, not learning. Like you're at your job.

Then that next quadrant is explanation, which is study but theoretical. And I actually think this is one of the places where Django is kind of Really set apart. Like a lot of software documentation doesn't do this part, but Django actually has a wonderful set of guides, right? Um When I was kind of researching this talk, I went in and looked at the Django documentation, and there's this great performance guide. And one of the like the subtitles in that guide is like Understanding Laziness. And it's like that is that is such a like explanation, right? It's like how do I think about laziness? I just loved that as a as a title, right? But it was like Here's how to think about how laziness works in Django. You know, here's how to use laziness in query sets to make your code do fewer queries.

These kind of things are, you know, that's not a immediately actionable, but it's how do I think about building something in Django? How do I kind of understand the problem I'm trying to solve? And then lastly, uh, this is one that you know a lot of people are pretty familiar with, right? Is reference. This is work, but it's theoretical. And I struggle a little bit with the the theoretical part on this, but really kind of I think what the framework is trying to say is that it's very concrete, right? If you go to like a reference in a page of documentation, It has to be like exactly what the world is. Like it is almost certainly either generated from code or something like that, right? You're not putting it to work. It's much more about just kind of a a state of the world, if you will

Um and so yeah, I mean I think we all know what references are. Like in Django, for example, there's the query set API reference. where it goes in and it lists all the methods. We have the template tag reference. But kind of putting that in the theoretical framework, that's the job it's doing. So I do think it's really useful to think about verbs here. And so these are all kind of examples of each one of these things, right? So a page that teaches you how to build a Django application, guiding you in step-by-step instructions. What would people say that would be in the framework? Tutorial, right. Then you have a page that shows you how to quickly add a field to the Django admin. What would that be? How-to

guide. Something that explains the usage of HTTP caching and why you'd want to use it. Explanation. And then something that lists all the classes of a class-based view. So pretty easy example. And it went in order of the things that I told you to make it a little easier. But um but it it's really just I think those verbs are super important, right? Like that is the job that that is actually doing So that is a very high-level introduction to DiaTaxis, but I think it's a really important way to actually start thinking about documentation, right? I think historically people haven't really put a ton of deep thought into these things. And it really helps you kind of understand where to get started and

how to approach this problem. And one thing I think is really interesting is the impact as an ecosystem, as more and more projects kind of adapt to this, then documentation, how it like how to read documentation becomes a skill that we all learn within kind of the Python and Django ecosystem. So I actually think that's a really cool outcome is if more documents start, you know, more doc sets start being written this way, then kind of readers will know how to approach things and it will actually kind of grow the skills for everybody. So I did want to kind of show that again so that it makes a little bit more sense now, hopefully. There's there's obviously still a lot more kind of depth to this thinking, but I just wanted to give a high-level overview and do some marketing. Uh for Dataxis, because I do think it, you know, documentation doesn't get enough marketing in the uh tech world, I would say.

Alright, so that is kind of the the theoretical authoring uh section. Now we're going to go on with a little bit more tools. And so in the kind of ecosystem, there's two that you'll probably run into the most. One is Sphinx, which is historically restructured text-based, and is a lot has a lot of functionality. It is kind of old, it has been very well loved, it is heavily used, and has a ton of features. And then MakeDocs is kind of the like upstart. Which is a little bit more markdown based, it's a little slimmer, and it works works well for kind of smaller things, and it's kind of growing an ecosystem. But definitely Sphinx is a little bit more heavily used, I would say It's you know Django, Python, a lot of the ecosystem is using this.

So I did a pitch where I would cover both strings and make docs for a 45-minute talk, but I with just 25 minutes I wanted to focus on one And so on the MakeDocs side, I will recommend you check out material for Make Docs. If you're interested in kind of something a little lighter weight, it is a great, great piece of software. But I don't have enough time to cover both today. So we're gonna do Sphinx. And one of the big exciting things that I think a lot of people have have held people back from being excited about Sphinx is Markdown. Right? Markdown is like the thing that we're all using on GitHub, on you know, Slack, on everywhere, basically. Like Markdown has kind of become a standard for Programmer text input. And so that's a big deal, is that the Swinx ecosystem has first-class markdown

support. And that project is missed, which basically just swaps out restructured text parsing for markdown parsing. And so this actually works by, this is kind of the overview of the whole ecosystem with, you know, kind of read the docs, which is working on top of Sphinx, which is working on top of docutils. Which is an implementation detail we're not going to cover too much, but if you see the word as you're kind of perusing the ecosystem, you at least know where it fits in. And then DocuTills has a restructured text parser. And then Mist is actually a markdown parser that generates that same internal AST. So that's how you can kind of think about it. And so what that looks like in practice is here on the left we have kind of the table of contents tree of Sphinx that we're you know familiar with if you're using Sphinx much. This is basically it translated to Markdown.

Looks very similar to Markdown. They use that kind of front matter that a lot of people are familiar with with kind of YAML files, that kind of stuff, as the kind of argument section. So you can see how it kind of maps from one to the other. There's lots of examples like this, but the really powerful thing that they did is they took Markdown and added directives to it. So historically, there is no standard syntax. for directives in Markdown. And so that's kind of the exciting part is you can actually have, you can effectively call, you know, restructured text directives from Markdown. So you have all the power of the ecosystem of Sphinx. Embedded in a markdown document. And so it's pretty straightforward to install it. You just, you know, pip install it, con install it, add it to your Sphinx extensions, and then you just start writing files with MD extensions and

It just works. Um so this works really nicely for kind of pulling in existing markdown content or kind of authoring in a mixed environment. One of the cool things you can do is you can actually mix these together. You want to you know have some standards, right? So maybe it's just like this whole section of our docs is RST, this whole section is markdown. You know, you don't want to be mixing them too frequently because it'll just like melt your brain. But it is really nice if you're kind of doing a a migration process or say you have, you know, release notes that you maybe want in markdown, but then your API references are an RST to make it a little nicer. That is something you can do as well. So there's kind of a a transition path which is nice. So okay, we we've talked a little bit about how to do authoring. You're kind of starting to to make some documentation, but now you have to present it to the world, right?

And that's where it gets kind of exciting is there's been a decent amount of effort in the last few years to really improve the kind of experience of reading Sphinx documentation. Within the ecosystem. And the one thing I do want to highlight the most on the design side is Furo, which is a really nice new Sphinx theme that is kind of that standard three-column layout. It looks really nice. Here is kind of a you know desktop, tablet, and mobile version of it. Um the most interesting thing is that kind of three-column layout, like I said, the Right side is a page table of contents and the left side is the entire project table of contents, which is a nice way to kind of engage with documentation. And it has lots of nice features. It has

the three-column layout I mentioned. It has a nice dark mode. It's really nice to customize. It has that all kind of built in and makes it easy to add a logo or change the colors or do those kind of things. And it also is kind of a becoming a standard endpoint for a lot of extensions. Like they'll support Furo kind of as a first-class design output for the extension, which is nice. One of those other kind of extensions that's come along is Sphinx Design, which at which is a nice way to add UI uh UI elements to your documentation. So we're very used to documentation that is just like incredibly text heavy, right? Like the the internet has gone into this world where everything's really interactive and there's all sorts of rounded corners and stuff, and like I feel like our documentation has just looked

Very static. And so Sphinx design is really nice. It adds bootstrap-based UI elements. And they actually have output for a lot of different Sphinx themes that They do Fior, some of the other ones as well. And it just makes the docs look a lot more modern, a little more interactive. So these are some of the things you can do, you know, cards, grids, styles, all this kind of stuff are kind of the things you can support. But this is kind of a nice example where it's like, that is actually documentation that 's maybe slightly washed out on the projector. Um but the interesting thing here, right, is you actually have this like UI element that looks mu much more like a landing page that you would see. on a marketing site than something inside of documentation. And so you're able to kind of do all those things within your project, right? If you need a button, you need a call-out, you need these kind of

just a little bit nicer kind of interface, it makes it very simple. So just as an example of what that looks like, uh you have like kind of a big card. It's you if you actually like hover over it, it adds like a nice shadow and does all that stuff. And it just is kind of standard RST or markdown uh syntax. Which is really nice. And so that really kind of gives you as an author a lot more power to really express what you want in a set of docs. Like you're not just, it's not just like text and image. images. You have a little bit more you can work with to express what you want. So then on so now you have, you know, really nice looking documentation with a beautiful theme. But we're all Pretty used to just kind of sitting around and and reading words on the page.

And so I do want to talk about a few extensions in the ecosystem They work to improve the user experience. Because I do think this is somewhere there's space for more innovation here as well. Like we're doing a little bit of work, but there is definitely more that can be done to kind of improve the UX experience. And so the first one that I want to highlight is definitely tabs. This is a very classic example, right, where you're like, here's the install instructions. We want, you know, here's the Mac OS, here's the Linux, here's the Windows, or whatever, and you want to have like a nice A nice set of uh tabs with different instructions. That just gives you a very, very simple way to do that, right? And looks very similar, it's kind of integrated into RSD, works in Markdown as well. Another really kind of silly one, but I think is actually really important, right? Is a copy button.

As is, you know, historically something that would have had to been built into the theme itself. But the nice thing with Sphinx is you do have this extension, right? And I mean Having a GIF of a copy button seems almost a little overkill, right? You know what a copy button does. But it's really nice, right, when you have code examples in your docs and just those little kind of affordances that make it a little nicer. So this is one that we've actually been working on uh personally. So is Sphinx Hover Xref, which is a terrible name, we should really rebrand it. But what that does is it gives you those kind of Wikipedia style hover cards. So if you have an internal link to your documentation, you can actually hover over the link and then it like pulls up that content and embeds it nicely in the page. And so that's a really nice kind of affordance. It works really well with kind of API documentation, right?

If you have a it's like here is the return type and you just hover over the return type and you're like, oh cool, like I can get a little quick overview of what this type is. And I don't have to open like six tabs to like, you know, tab back and forth. It just makes it a little bit nicer to kind of browse that. And the other one I want to highlight is again one that we've been working on, which is kind of a live search experience, right? So you're used to search on the internet. It usually is search as you type, has nice engagement, this kind of stuff. And so that's basically what this is, right? Instead of having a search where you hit enter and you go to a page and it loads results, it just has a nice UI on top of that that does searches you type. So I think there's a lot of these examples where we could be building more and more kind of UX into the ecosystem, and it makes documentation better for everyone.

everybody, right? The readers are gonna have a much better experience. Um and yeah, I just think it kind of is like slowly raising the bar of like what people expect and the kind of experience they have reading documentation. And I think that's a huge, you know, that is a way to get more value out of the work we're putting into write docs. Because it is a lot of work. Like writing documentation is hard. And we definitely want to make sure people are getting the value out of it when they're actually going to read it. So the last thing we're going to talk about is deployment. So this is actually my day job that I work on Read the Docs full-time. And we are an open source project. And basically what we do is allow you to easily deploy Sphinx documentation. It's heavily used in the kind of ecosystem. We actually

host Django's PDF docs and a few others. You know, Pip, VirtualM, Conda, a bunch of these projects use our service. And one of the cool things that's kind of new notable, right? This has been around for basically 12 years now, uh, Read the Docs has. But the cool thing is we're now kind of adding a lot more extensibility to allow you to support other kinds of tools. and other kind of workflows. So this is what kind of a a standard YAML config looks like, right? You're just like, what OS are we running, what Python? You know, where's your Sphinx config? And like, how do I install the project? And then we will basically just run and build Sphinx stocks for you automatically. So this is kind of the traditional way that Read the Docs works. And so we have two new kind of additions which are kind of fun, which one is called Build Jobs.

And so this still kind of has us running the build commands. We kind of control Sphinx for you, but it allows customization on top of that. So this is great for small editions if you just want to like auto-generate a little RST file that goes in your docs, or you just have some script you want to run for some reason. So there's a bunch of different places you can kind of hook into the process where we're checking out your version control after you do the build, after we set up the uh virtual m for the conduit environment, uh this stuff. So this is just kind of an example where it's like, okay, after the build, we want to, you know, hit this URL with some data, for example. I mean, who knows what that URL does? It's basically just, you know, a a lightweight webhook. But that's just a very simple example of how that might work. Some other common customizations we've seen, we've been kind of seeing what the ecosystem is doing here.

Uh one is installing dependencies with poetry. We don't have kind of first party support for poetry, but you can just run poetry, right? Just poetry install and it all just works. You can unshallow a git clone. By default, we kind of do a shallow clone on Git uh just to save space and uh performance. But you if you have some extension, a lot of people have these extensions that read the full Git history to generate really strong. notes for example. You can kind of manage that yourself. You can check for broken links. This is a big one. Sphinx has a built-in link checker. So you can just say like, hey, run the Sphinx, you know, make link check. before you do the build and then cancel the build if you don't do anything if uh there's broken links. So yeah, there's a lot of these kind of examples of Places you can go. You can look on our docs. Actually, we have all of those cases kind of with code examples that you can copy.

And the other kind of exciting part is build commands. So this basically allows you to override the entire build process, right? So this is what you think of as a kind of traditional I don't even know. traditional kind of service, but basically it's like you run a shell script and whatever gets outputted in this read the docs HTML directory, we host for you. Pretty straightforward, right? So this is an example of doing Pelican. So we're actually hosting one of our own sites via this method in Pelican. Um and so yeah, this is basically whatever you want, right? You can just run a bunch of bash, you can run a Python script, you can do whatever. You know, stuff happens, HTML comes out, and then we will host it for you. And so this is pretty cool because it read the docs has lots of power on top of it. We do versioning, we automatically index all of your stuff for search.

We have a bunch of kind of features that the platform has that you then you can now use with you know, basically any tool that you want. I was talking to somebody yesterday who has kind of their own pet uh static site generator, and I'm like, yes, come use it, give us feedback. Um so this is a pretty exciting new possibility. You know, for the longest time we were like only supported Sphinx. And it's still the the biggest thing we support, but now you can kind of bring your own tool as well. So hopefully at the end of this talk, you will go back to your company and you will kind of introduce some of these practices, right? You're gonna start thinking about writing your docs with DiaTaxis. They're gonna be built with Sphinx, they're gonna be themed beautifully with Furo, they're gonna be enhanced with a few of these extensions that I've talked to you about, and then deployed online to read the docs.

That is kind of the the happy path, I would say, for the ecosystem if you're you're looking for a way to create documentation today. So that was very fast 25 minutes. But thank you all for listening. I really appreciate it. I don't think we have time for questions. But please do go forth and make happy users with your documentation.