An approach to lightweight tenancy management using Django Rest Framework

Okay, hello everyone. Can you hear me? Yeah, okay. So I'll be presenting an approach to lightweight tenancy management using Django Rest framework. And if you don't know what that means, I'll explain it in a bit, so don't worry A little bit about me. I'm a full-stack developer at Octobot. Octobot is a software development company that specializes in designing and developing digital products. And I've been working there with Django and React and some other technologies for close to four years now. I'm from Montevideo in Uruguay. If you don't know where Uruguay is, it's a small country in South America between Argentina and Brazil. And there you have some things that I like, like reading, cats, uh watching my soccer team on the weekends and some other stuff. And this is my first time attending Django

Con, so I'm really happy to be here Okay, so the agenda for the presentation today, uh first we'll go over what multitenancy is and a particular use case Then we'll go over how we implemented our solution, including nesting routes with a library called DRF nested routers, and using a custom USet and a model manager. We'll go over some limitations that we found, and then we'll go over a different approach using a different library So, if you look up multi-tenancy, there are like a billion definitions out there. This is just one that I liked. So I put it on the slide. It's from Red Hat It reads: multi-tenancy is a software architecture where a single software instance can serve multiple distinct user groups.

So the key part here is the multiple distinct user groups. And these user groups are called tenants. So we have many tenants, and these are the user groups and the resources. And the key point is that each tenant is isolated from the rest. And just as a side note, not all applications will have this tenant concept. It's more common in software as a service architectures or things where you market your product not directly to the end user, but rather, for example, an entity or a company That will use your product and then they will have their own users. So there are like two main approaches to this problem. First, you have a single-tenant architecture. where each tenant will have their own application instance and their own database. And so you have all your tenant stuff isolated, basically like physically isolated.

And then you have a multi-tenant architecture where basically you have all your tenants and they all share your application instance and your same database. And you have to implement the isolation kind of in a logical level. And that's what we'll be focusing on today. Uh yeah, so an example that uh maybe you don't realize it, but you're probably already using uh multi-tenant software. So for example, in Slack you have multiple workspaces and you can see all your workspaces, but you don't really cross information from what workspace from one to another. They are isolated and so a workspace could be considered a tenant. And here's a disclaimer, I don't know how Slack is implemented. I don't know if this is what they do, but I thought it was a good example. So

So if you remember the name of the talk, it said lightweight of the tendency. And here are like some disclaimers. This is basically a real life scenario that I've run across. In different projects, and they were different realities, but they could all be abstracted to the same set of requirements, which is this. So we'll have our tenants, we will all share a database and our application instance We'll have resources that will belong to a single tenant. And in particular, we have an API and we want our URLs to look something like what's shown on the screen, where our resources are nested under their tenant So we have tenants slash tenant ID slash some resource, and for example, the resource ID. And finally, users in this case can belong to multiple tenants. So in the same way that a user can access multiple Slack

Workspaces, we want users to access multiple tenants. And the key point here is that we want to restrict access so that users should only access resources from tenants to which they belong. So this is the example. I hope it's readable the code, but I'll talk through it anyways. The example I'll be working with throughout the presentation. And here we have our models, our Django models. We have our tenant model, which in this case is a company. It has a name and it can have any other fields that you want. We have our user model, and like we said, Uh a user will be able to access many tenants or in this case many companies, so we have a many to many relationship between our users and our companies And then we have a resource model. You could have multiple different resources. In this case, I just put one, which is a report of some data.

And the report belongs to a single company, so we have a foreign key. And you may be thinking, well, why would a user want to access multiple companies? Like if the user is the employee of a company, you only work at one place. But a user can be something other than an employee, for example, a person who's lending services to a company, like an accountant. That does the accounting for multiple companies and so they want to access all the companies to which they are providing accounting services. So yeah, that's like an example. And this is again these were real life scenarios that we've seen working in different projects So, yeah, basically the two key points of this implementation will be to effectively nest your API routes so that the resources fall under the specific tenant, and then to consistently restrict access to resources.

And because we're programmers and we want clean code, we wanted to centralize all the checks in a single place so that we avoid code implication and we don't have to check, oh, does the user access these companies that permitted every place in our application? So uh first point the nesting routes and for that we used a library called DRF nested routers It's really simple and easy to use. It's just a couple lines of code. You write your usual URLs file, you import your routers from REST framework. And you also import your nested routers or UMPOR routers from REST framework nested and I renamed it to nested routers to avoid a name clash. You create your default router as usual. And then you can create your nested default router under which you

actually nest all the viewsets that you want nested. So in this case, I create my company 's router with uh uh referencing the original router and then the lookup company and I registered the reports viewset under this router So that's the part for from this library. And so the way you would use this is in your report view set, for example, you can filter the query set uh using the company primary key in the URL. And so, in an example, if this is my reports table that I have here, I have two reports, one for company with IDE 23 and one from for company with ID 5. If I perform a get request to company slash 23 slash report slash one, I'll get a 200 success.

Report number with ID 1 belongs to company 23, so no problem there. But if I perform a request to company slash 5 slash reports slash 1, I'll get a 404 because the report one does not actually belong to company with ID5. So this was the first point that we wanted to achieve. It's done? Yeah, yes? Okay. Next step, and I think is the most interesting part, uh, would be to consistently restrict access to resources. So we did this in the first place by writing a custom view set that will handle all these tenancy checks. This is the viewset. It's just those lines of code, so you can see it's not a very very complicated thing The viewset inherits from DRF's

generic viewset. And the only thing we are doing here is we're overriding the initial method to basically perform this tenancy check for us. So uh first we just check that the user is logged in because we need a user for the check. If not, we can return a not authenticated exception. We get the company primary key from the URL And then if the company primary key is not there, we can raise a not found exception. And then first we get the user from the request. And the interesting bit is this line of code where we look up the company with the primary key, but instead of looking the company up in the company screwy set, like we might do in other scenarios, we looked at company up in the user set of companies. So say I'm looking for company number twenty-three.

If company number twenty-three does not exist at all, this will raise a company that's not exist exception, sorry, which will catch uh in the next line and rays are not found. But if the company with ID23 does exist, it's just that the user does not have access to it, so it's not one of the user's companies. The query will also raise a company does not exist error, and we will still catch it and return the not found. So with those lines of code, we are basically implementing that tenancy check functionality. And then in your resources, in their viewsets, for example, you can simply inherit from the company nested viewset that we just created. And it will perform the initial check. Sorry, I think I forgot to say it, but the initial, if you don't know, I didn't know it before I implemented this.

It runs code before anything else is done with a request. So you can basically put any custom logic that you want there. So here it will run that code and then we can just use our viewset as usual. And uh as a bonus, if uh if if the check was okay and we are in our code We can access the company instance in self. company because we we saved it here. So we don't need to actually go and retrieve the company every time we want to use it. For example, to implement business logic or uh yeah anything that we wanted. Okay, so that was the second point and you might think oh well this was a short talk not quite I have a bonus that I didn't add first. It's not strictly necessary, but we noticed that we were kind of making some mistakes

which led to accidentally leaking information across tenants. So returning information that was not for the specified tenant. And to try to minimize these kinds of errors, we created a custom model manager. And I'll explain the problem a bit more here. So, for example, imagine I'm writing some code and I'm writing kind of a complex query and I forget to filter by company. So here I have my reports and I'm filtering the name that contains some search value that maybe the user input somewhere or whatever. Start date that has to be greater than some date that Someone selected an end date and so on, any filter. And I forget to filter by company and whoever's reviewing my code does not catch my mistake. And so now I accidentally expose information from another tenant

And depending on what the reality is and the scenario that can be oh, that's our kind of bad bag to that's a really, really, really bad bag and we don't want it So yeah. So we thought we found ourselves not all the time making these kinds of mistakes, but every now and then they would pop up. And we found maybe we should try to fix this somehow. So we implemented this company aware manager. It does just two things. First, it overrides the old method of your usual usual Django model manager And it raises a custom exception. In this case, we called it missing company exception. And it is defined there. It does nothing special. But you could implement your own exception logic if you wanted. And so if you try to call all on this manager, you will get the exception

cannot call all method in company aware manager. Company must always be present. So then uh we also overwrote the filter method and we are checking if the company ID or the company are in the keyword arguments. If neither of them are in the keyword arguments, we also raise our missing company exception, telling the user or well the developer in this case, uh you need to pass in either one of these keywords. And if they were there, if either one of them was there, we just use our filter as usual with our arguments and our keyword arguments. And so the way that we used this manager was uh first we created an abstract model uh to replace the default manager. Basically, we overrow the objects. manager to be our company aware manager and we also created a different manager that we called objects or

companies that will be your regular Django default manager This is an abstract model, so we said abstract through true in the meta class, and then we inherit from this model in our resources model. So here in our report model we are inheriting From the company aware model. And yeah, the company aware model is not technically necessary. You could just have your managers directly in the report model. But if you have multiple resources, you don't want to redefine that code every time. So it's good to have the abstract model. And so the way this could work is uh if you have these example queries, for example. Um The first two, so if I try report. objects. all and report. objects. filter and some

imaginary filter query that I want to run, for example, my ID in some list, this will raise our missing company exception. However, if I try to filter and I add either the company, referencing, for example, some specific company, or the company ID, and you don't have to make it be an exact match. I can use, for example, company ID in list of company IDs. This will work as expected. And what happens if I didn't forget to filter by company? I I'm I know what I'm doing, it's not that I'm making a mistake. I want all the reports or all the reports are much sum criteria. That's why we had the other manager, the objects all companies. And so that one I can use as usual. I can call my all method or my filter method without any company ID

And so this what it does is basically explicitly declares your intent to look up resources from all companies. And so it's the person who's reading your code will know, okay, you're not forgetting to filter by company. This is exactly what you meant to do. And if you did forget, you're probably using the objects just because it's what you're used to. Sorry. It's automatic. Uh you will get that exception that will tell you, hey, you're forgetting to filter by company and if you know what you're doing you're like no I'm not then you remember that you need to use the other manager. So uh yeah that was the third point. I'll drink some water sorry Okay, so some limitations to this approach.

First, it does not work if you need different tables or even different schemas for different tenants. This assumes that all the tenants will have the same resources and the same models. If you have like any other custom permission checking, it needs to be implemented separately. So if you have a lot of complex, I don't know, role-based permissions, object level permissions, any kind of permissions. We didn't implement any permission stuff. We are only checking, okay, can this user access this company? Yes or no. And then another limitation is that the checks are only done at the viewset level. So if You have, I don't know, your Django admin, for example, we are not checking anything there. So you would need to implement checks elsewhere for other access to your application. We just did our API because it's what our users were using Um and yeah, and also with the custom

model manager, you prevent some kinds of mistakes, but there are another probably infinite ways to mess up and still leak information if you're not careful. So yeah, that was one fix, but it's surely not a fix all. So with that in mind, especially the middle one, the second bullet point. I have a different uh approach using a library called DRF Access Policy. So, DRF Access Policy is not a multi-tenancy library. It's just an access control library. They are the this is taken from their documentation. They say they are a declarative organized approach to managing access control. And so using this library, each viewset can be assigned an explicit policy for the exposed resources.

And what I think is cool about it is that it allows for a lot more complex permission logic They are like a permissions library. So you can do the multitegenancy stuff, but you can add so much more. So yeah, that I think that's like the main selling point of this. And just a side note, it does not do any URL nesting. So you would still need to implement that either like I just showed you, or if you have your own implementation, I'd be happy to discuss it later as well. So the way this access policy works is that you import access policy from access rest access policy. And you can create your own. In this case I called it resource access policy, which inherits from your base access policy class. And there you can add any, there's a lot of different statements that you can add.

The documentation is pretty thorough and it has a lot of examples. So I encourage you to check them out if you're interested. But here I'm only implementing the tenancy stuff so that this is kind of equivalent to the example with the viewsets that we just saw And so the this access policy has a scope query set method that receives a request and a query set. And the way it works is that basically you get to do whatever you want with that query set and that request. And in this case, what I want to do is that I want to filter The query set by the companies from the user so that the user can only access resources that belong to one of their companies. So in this case, we just get the re the user from the request and we get their companies And we use that to filter our query set, whichever it may be, using the company in user companies.

So if your query set does not have a company, I mean if the model you're querying for doesn't have a company field, this will crash. But I mean it's meant for uh resources that are related to the company, so it's okay. So the way that you use uh this access policy is that in your view set , in this case in a report view set, you reference the access policy and then in your getCoericit method You can simply use the scope query set method that we just implemented, and you pass in your request and your uh reports query set in this case, and that will do the filtering that we just implemented And yeah, additionally, if you had any other permission checking, uh you could add it here as part of your access policy. And also you could define different access policies with like more strict

or more loose requirements if you had some resources that required stricter permissions and some that were like more accessible. Here I just defined the one policy. So, yeah, some conclusions. Implementing multi-tenancy can be easier than you think. And then the disclaimer, depending on your requirements, there are a lot of complex things out there. This is pretty simple. It was just a few lines of code, so yeah. If you need to nest routes either with multi-tenancy or just because I I really like this library, we use it in production. It's working fine. So yeah, DRF nested routers And then if you need a lot of custom permission logic, you might want to consider DRF access policy. I haven't used it. I just checked it out, looked at the examples, laid with it, but it seems pretty promising.

So yeah Uh thank you. That was the talk.