Building JSON APIS With Django / Pinax by Brian Rosner

Speaker 1: Come on, no.

Speaker 2: Alright, I'm going to be talking about building JSON APIs with Django and Pinax. I like to say building Django backends for the modern world because I feel like web application architecture has changed so drastically Over the last several years. Several years ago when the iPhone was was introduced, we had uh no idea what this little device could do and what it would change to our expectations. Fast forward to did it, fast forward to today, and we have an app for nearly everything that we're putting onto the web. Built in a web app is so much more than just the traditional Django model view template. There's so much more that goes into it, the view layer is becoming more complex. And we need to implement APIs to let the smarter, richer clients be able to interact with our data. So before I get uh

Speaker 2: before I get started, let me talk a little bit about myself here. My name is Brian Rosner. Um I live in uh Denver, Colorado with my wife, and uh we have a baby due in October I work as a chief architect at Elderian. Eldarian is a web agency where we build uh web apps using Django and PinApps. I work on Pin X at work. I have the polyverse ability to work on Pinax, but also my spare time. I'll speak a little bit more about Pinax in a moment. Eldarian also recently open sourced a uh a PaaS, a platform as a service uh named Kel. It's a foundation for our commercial Eldarian uh cloud uh uh which was originally named Gondor and uh

Speaker 2: Kelly itself uses much of what I'm talking about here today um and it also helps drive the development forward on Pinac and the in the API work. But enough about me. I'm here to talk about Pinx API. Pinx API was built out of some requirements we had for a development project we had at Alberian. Um the goal of of Pinax API uh was to is to provide a few primitives for building uh good RESTful APIs Using Django and Pinax that will be implemented. Hopefully these sorts of APIs that we built for these apps are going to be used without throughout the whole Pinax ecosystem Before I get too far into Pinax API, I'm going to step back and talk a little bit about Pinax itself.

Speaker 2: So Pinax is an open source platform built on Django. It has a standard project layout which Which if you're familiar with Django, as we all are, you would do a Django admin star project and you get a project. And that has some conventions that are already baked into it So Pinax kind of builds upon that and provides even more convention around things that are more specific, such as through our starter projects, we have various starter projects. uh like account social project that kind of are more tailored towards a specific set of of uh requirements you may have for building your site And also these starter projects are made up of reusable apps. There's a lot of reusable apps that we have in the Pinax ecosystem, and they're all used as the building blocks of these starter projects to actually implement the function.

Speaker 2: That are in there. And then default templates. There are a lot of default templates that we provide out of the box with, and actually you don't have to worry about spending time building out various views and the way that they look. You can rely on some like Bootstrap or other uh CSS frameworks to actually do that stuff for you and you can just focus on on uh the stuff that makes your site different. And so, like I mentioned earlier, uh the reasoning for Pinux API was due to requirements of an Eldarian development project. Um, during our discovery and research phase, we came across uh the JSON API specification This specification provided a lot of clarity on building APIs, and specifically RESTful APIs, and eliminated all the bike shedding that was that uh that came about with

Speaker 2: how is the JSON payload structured because you can go on and on with days about whether or not the key is here or there or how is the ultimately structured and this just takes care of of of all of that. Let's see here. Yep. Jason Ace JSON API itself was a big reason why we decided to build Pinax API. Job Rest framework is the obvious obvious choice. It's a very well-structured, very well architected uh app. It does it didn't quite work well when I looked at a lot of the different resources that are available to to connect JSON API to uh Django REST framework and and be able to take advantage of all of the the components of the Django of of JSON API, such as the

Speaker 2: such as relationships and stuff, and I'll go a little more into that. And I I do need to admit there's a little there's a little bit a hint of of uh non-invented hair syndrome. But I justify it because we wanted to take uh full advantage of the spec um and and mirror it with pin acts in some very specific ways that really only the built for ground-up approach is going to be able to do So this is an actual flowchart that we had with the development client at Aldarion. I say this is an increasingly common modern application architecture, largely through the experience that I've had at Aldarian with with regard to uh the different sorts of architectures we've been building out for for our customers. At the top there you have the you have the database where all your data lives um and it is it can

Speaker 2: it's being exposed as an API. And traditionally the the view of the application is that a lot of this time, a lot of times this is all kind of in one in one code base, in one process. But but that's that's changing quite a bit because a lot of times you end up needing to actually split off these front ends as they're they're into their own project because you need to implement it in a different language. And in this particular case, it was we because we need to implement these front ends, or specifically the shop front end, in uh Node, React. js, and Reedux. So looking towards the future of a project like this, it isn't going to be a big surprise when the client comes up and says, we need to add an iOS, an Android app. It just becomes a new front end that's able to communicate with the API.

Speaker 2: So a little bit more about JSON API itself. Um there's the the the JSON API. org is the URL that actually talks about the specification itself. There's gonna be a lot more in-depth details. um available there um that I'm not gonna be able to get into here. The way that I look at the specification is as is is a as an object graph. The object graph is naturally as a developer you're going to be creating these object graphs in your applications through through the creation of your models that you create in your Django project, etc. My specification is optimized. It's optimized in a way that it creates efficient rights, and you can also design it in a way to have really efficient I'm sorry, send the backwards. It it enables efficient reads, but you can design it in a way that makes it uh great for efficient writes as well.

Speaker 2: Uh for example, let's say you have uh you have you have a blog post. And the blog post uh is being requested by your front end and your front end actually needs to uh display the comments as well. Well, that could actually be turned into two different HTTP requests, but That's gonna increase uh latency for for your front ends. So the way that JCN MPI works is that you can actually uh use uh relationships and compound documents to say, I want the blog post, but I also want its comments or I also want its author all in the same payload so you don't have to go ask for this information over and over again. And uh also uh JSON API is it's registered with the the Internet assigned numbers authority, so it's actually available as a specific content type application

Speaker 2: via the API plus JSON. So some of the design influences that I had when I was building Pinax API uh it was uh was Kubernetes. Kubernetes wasn't something that I pulled something specifically from. But it has it has a really great HTTP API that I used as the influence, the basis of kind of the thinking that I had with regards to APIs in general. Django is is obviously a big influence because I wanted to make sure that Pinx API was something that that worked really well Django, because that's I think that's really important. Um and uh JOS framework also was a big influence in the way that that that uh Pin X API came about because some of the architectural bits that are available

Speaker 2: the way that it's designed, I you know, in this case it's it's uh The top of the flattery is to copy. So this is I think a perfect example of that and perhaps as well with the ecosystem because these applications are changing so drastically with requirements. We need to implement. Um I wanted to make sure that this is something that works well for that because that's what Panax is trying to solve as well. So I'm gonna get into a little bit more details of the specific API primitives that are available in Pinax API. You have the API resource. And the resource is the representation of the data to the outside world. Relationships which link data together, for example, an author to a blog post. An endpoint set, which is a derivative of a Django view, uh

Speaker 2: Django view class, different HTTP methods are mapped to different instance methods uh forming a RESTful RESTful interface. So talk a little bit more about an API resource in particular. A PIDEX API resource, you can think of it as kind of like a Django form, but without any validation. This is really important to think about in this way, because do you think, well, if the data is there, that's where we need to validate it because you're dealing with untrusted input The way that this is designed is that the API resource is actually completely agnostic to where the data is sourced. And then there's ultimately helpers that are used in the endpoints, which I'll go over here in a little bit, that link it to the datasets.

Speaker 2: So the reason why this is done this way is for affordability. So for example, so for example, this here doesn't necessarily need to be tied specifically to a data source. It's tied to the representation of the data, which can make which allows it to be portable and used inside the at the client layer as well. And some particular design decisions that were made around this was that you'd be able to encapsulate this in its own project that's separated from the project that actually implements how to get this data. And then that can also be used as your client that talks back to your API. And that keeps it all together nicely. The model layer is what implements your validation. So in this case we have an author, which is just a Django, Django model that you would build just in Django.

Speaker 2: The validation lives there using Django model validation. So all your validation would already be encapsulated through your model and then the Pinux API will call right into that. The model could actually even be a dent. It doesn't even need to be a Django model. It could be anything that you can pull data from. Pax API uses uh relationship uh relationship information to um to serialize linkage So a relationship defines how you link data together. We have here the same resource, but actually adds in the relationship. information here. So a relationship can either be a non-collection, which would be a many to one, maybe being the many many authors to one

Speaker 2: uh publisher or it can be a collection which in this case would be an author has posts um and the post being defined here um Using so the string values that are there as the first argument are actually the uh API type that are all being registered as you create your resources through the API. register. And then it's able to kind of build up this graph and it will know how to serialize it once it comes time to serialize the data. So an API endpoint set. API an API endpoint set um is kind of what binds the resources to the HTTP method. So when you perform a Git, it's going to get translated into in in X API into a retrieve method where you would actually implement the logic

Speaker 2: and I'll show it here in a second. An API resource endpoint set is actually derived from the endpoint set itself, and that's largely what the U use to create the endpoint set that linked the uh the resource to the to the endpoint set. All of the uh URLs are automatically generated. Um and it provides a simple validation pr a very simple validation primitive to actually connect the data to the resource. And I'll show here in the next slide. This is the first part of it because this can get really long once you've implemented your actual uh resource handling of a of an author in this case. So here we actually are importing the resource. We are binding the endpoint set to the

Speaker 2: To the resource, the author resource. And this is where you define the kind of the basic uh bits of the of the author and how it's gonna hook up into your URL. So there's a a base name which is just largely it's just the singular name of the of the model um or whatever the data model is. Um and then the base red regex which actually defines kind of where it's going to live in your AP in your URL namespace. But it's done at the collection level. And then the lookup actually defines how it will look for the individual object that's inside that inside that collection. So in this case, the field is a PK. So if you actually kind of think of a URL conf, it kind of all just This all kind of gets compiled down into to single regex, which would be authors with the pk v mapped out of it And then these are the methods that you would define in your endpoint set to actually link everything all together.

Speaker 2: So you have a list, retrieve, create, update, destroy, which roughly mapped into a Git on the collection, a get on the on the individual endpoint or the individual object, a A post on the collection and then a put on the individual object and ultimately a delete on the individual object. And the validation primitive inside of create and update is through is through the validate method, which actually just says on this particular resource class, I'm going to validate this data and it passes and the resource actually defers it off to whatever the model is. And you can actually write whatever validation logic you want in that with statement and it'll handle all the API errors that come right back out of that. So for example, if the name needed to have uh the letter A in it or whatever the requirements may be, it can raise the validation error in in the

Speaker 2: in the the context. Manager will take care of turning that into your API errors for you. You don't have to write any of that logic at all. And the same applies for the update. There's a lot of other features that are available in in the Pinax API that I'm not going to be able to get into a lot of the details of. Authentication. is one, you know, are you who you say you are? Permissions, can you access a given resource? Those are all built into the independent API that can help those individual methods ensure that the right person and the person that you can trust is actually accessing them. And then the other big bit of Pinux API that is really helpful is using API Blueprint, which is actually a specification

Speaker 2: for Writing documentation that's machine readable is automatically generated through all of your endpoint sets, and then you can actually define specific uh Customizations to that documentation. And what's nice about using leveraging API Blueprint is that you have the ability to leverage all the tooling that's available in its ecosystem. One really neat thing is that you can actually write out your Pinax AP you can write your API using Pinax API and then use one of the tooling that enables you to actually generate a client without having to actually write any client code and it can be in any language. Because all the the tooling in their ecosystem supports a ton of different languages And that's just all available at us at an individual endpoint

Speaker 2: that you can hook up. And that is everything that I have. You can actually look at Pinax API's code on GitHub, github. com slash pinax, slash pinx API. There's a ton of documentation there that goes into really great detail on how to actually do a lot of the stuff that I've described here with a lot of examples. um and my website and my Twitter and uh if you need any help on any of this stuff you can check out um Pinax's Slack and YouTube or even on GitHub uh right on the pinaxproject. com which I guess I didn't put that you can get there through the GitHub. If there's any questions, be happy to answer.

Speaker 3: Yeah, we have time for a few questions. I ask you to wait for the microphone so we can pick it up

Speaker 4: Uh hi, so I'm not completely sure where if this question is valid, but I'm wondering w how does this um complement or differentiate from uh JSON RPC? So JSON API versus JSON JSON RPC.

Speaker 2: I haven't seen JSON RPC, but it would that in that in that particular case it'd be um an RPC, a remote a remote procedure call that would be done more about actually serializing the the actual method calls that would translate into something specifically that would already be written in in your Python code. Um this is actually kind of providing a layer between how you actually call into into this versus the code that you're writing. So something that would work really well that would fit inside the Django, inside of a Django project. Okay.

Speaker 3: Any questions?

Speaker 5: Um, hi. Um, so um my question is Can you provide concrete examples or point me to a link or something that I can clearly compare Pinax API against Genkor Respraer? or provide uh real real cases when you when you see Django Respray were this doesn't work for me

Speaker 2: Yeah, so our our documentation actually has a lot of examples of how you might integrate with with real Django with real Django models like blog posts and stuff. I'm not sure if um how how it look compared to Django Rust framework, but you can see a lot of the things like I need to accomplish this one thing, let's say for example, I need to expose uh blog posts, you could you could see how that would look and how it how it all connects and and And then you can easily hit it using just some curl or whatever to kind of see what it looks like when the data comes back. The JSON API Uh the specification, their website has a ton of information about exactly the way that data is going to come back and how you how you would expect it and how you might implement a client that actually does interact with that API. And there's a lot of

Speaker 2: because it's built off that specification, there's a lot of other clients already built that know how to speak that through the that specification. Does that answer your question?

Speaker 5: Um Briefly. I mean the JcorS framework actually added uh JSON API support in the last version 3. 4 So that's why I'm I'm asking about the concrete difference.

Speaker 2: Yes, yeah I wasn't f I wasn't yeah we started this project back in January so at the time there wasn't much other than some Formatters that were that were third party that you can install with Django Rest framework that will actually take the already serialized format of Django Rest framework and translate that into JSON API so you can use any clients that already have it. Whereas the the the The point of this one was to hopefully get something that was actually native to JSON API that was able to take advantage of all of the bits of the specification, such as relationships and the compound documents and All that kind of stuff. So I'm not sure I haven't seen the stuff that they put into Jingle Rest framework recently, so I I'd have to look at that.

Speaker 5: It's pretty much what like last week.

Speaker 2: Oh yeah, see that's that's way too new for me. I was working on this, he said.