Polyglot Persistence with Django: When One Database Isn't Enough with Abigail Afi Gbadago

This video features Abigail Afi Gbadago at DjangoCon US 2026 in Chicago, Illinois, USA.

Polyglot Persistence with Django: When One Database Isn't Enough with Abigail Afi Gbadago
0:20:12
Published October 26, 2026
64 views

Polyglot persistence is the use of multiple databases in a single application, with each chosen for what it does best. Django ships with multi-database support: define your connections, write a router, and in theory, you're done. In practice, the gap between "Django supports this" and "I have this working" is where most developers get stuck. This talk closes that gap by using a Django MongoDB backend alongside PostgreSQL, with honest coverage of where the ORM abstraction leaks, what migrations look like against a document store, and lessons learned from building with MongoDB's Queryable Encryption in a real Django context. Attendees leave with working patterns, a clear decision framework, and an answer to whether polyglot persistence is the best choice for their project.

This talk was presented by Abigail Afi Gbadago at DjangoCon US 2026 in Chicago, Illinois, USA
https://2026.djangocon.us/talks/polyglot-persistence-with-django-when-one-database-isn-t-enough/

LINKS:
Follow Abigail Afi Gbadago 👇
On Mastodon: https://mastodon.social/@afimaame
Website: https://www.linkedin.com/in/abigail-afi-gbadago/

Transcript

3,335 words · auto-generated · en Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:14

Speaker 1: Hi everyone, welcome again. I know this is right after lunch and we're a bit tired, but If your eyes are getting heavy, you could you could do an icebreaker and it's definitely not the database talking, it's the lunch So my name is Afi. Most people call me Afi. I'm a senior software engineer and developer advocate at MongoDB for Django. I'm also a Django Not and I currently serve as Vice President on the Django Software Foundation board. So what today is about. So backstory. I'll talk about how I tried combining SQL and NoSQL. This doesn't mean I have too much time on my hands. It was just and exploration because I've always dealt with the same databases between in the same Django

1:00

Speaker 1: project. We'll also look at why teams reach out for two databases, we'll look at routers briefly, configuration steps applying the config migrations which is very special to Django, why it failed, obviously what was I expecting, and what I learned. And before you try this things to note and was it even worth it to even begin with. So let's get into it. So what is referential integrity? Before we start, this is a word that's normally used in databases. And it's a rule in DBs that keeps links between databases correct. It makes sure that foreign keys in one table always points to primary keys in another table. So this is to prevent what we call dead links or fake links that creates often

1:45

Speaker 1: records. With referential integrity, there's always a unique ID for a role in a parent table. For example, a user ID. You also have a foreign key, which is a field in a child table that will point to a parent ID. The rule is you cannot add a child record if there is no parent ID. So now back to the backstory. In my seven years of building with Django, I've never combined two different types of databases, as I was saying, in the same app. So to satisfy that sort of curiosity, I decided to try it out and do that within a single Django project. So I had different databases where I had a c a crypt an encrypted database and an unencrypted one. And it was due to a project I had to work on at work.

2:33

Speaker 1: So due to the fact that I had to use the same database, I decided to explore using a NoSico and NoSico database. So doing that here reminded me that Django has a database setting that isn't limited to a single connection. Because normally when we do Django projects, we always use like single databases from like from my experience. So why teams actually reach out for two databases sometimes in prod, is not because one database is better than the other or something is because sometimes your data may need certain uh things such as read or write operations that a single engine DB engine cannot give at once. There may also be issues like

3:18

Speaker 1: or needs like security and compliance, for example, the QE project I talked about. And that's why you may need an extra database, like an encrypted database for those specific specific fields. Also, keep in mind that a relational database enforces fixed schema and referential integrity, as I was saying. So every record has to fit like some sort of similar shape, and every reference has to point to something. Don't forget the offer records. Also, a document store trades that away for flexibility since you have records that can freely be in different shapes without having a reference to point to. So the real question isn't which database is better, it is does this need reinforcement or enforcement

4:04

Speaker 1: or flexibility? So now a quick show of hands. How many people have already configured more than one database in a Django project? Wow, the hands are really small. I thought a lot of people. Um well I hope if you try out the process is great, but we'll look at what I did. So as I was saying, Django has had this all along. Django has supported multiple database connections since Django 1. 2 through the Django-based settings. Do the Django database setting and then the database routers. Django 1. 2 added the ability to use more than one database in the project, thereby letting queries issue let letting queries be issued at specific databases using the with the user method and then the query set object

4:52

Speaker 1: Now, with a configuration, for those who don't know, it's in three steps. So the first one is to add a second connection to your database. So your default database is still going to be there. It's not anything scary. The second part is to write your router and then the third is to register it in your database routers. That is the actual kick that flips the light switch on. So the first example you see as I said the default database up there where you could configure your um relational or non-relational database. And in the second part, you could have the alternate version. So now we have two databases, one pointing to the default, and then the other one pointing to the documents database. The second part is a configuration for MongoDB. So for that one, I know most of us are from the relational background. For MongoDB, you need the engine, host, and then the name.

5:39

Speaker 1: You can store these credentials in your M file and pick it back in over doing stock since you talked about honeypot and all that. That got me a little bit scared Now the second part is to define your router. So that's step two. So in here you write your router, which is a Python class that tells Django which models belong to which connection. So there are four methods here in which we always add which is a db for read, allow relation, um, and allow migrate, and then there's db for write. The third part is to register it, which I said is the part that turns everything on. Without this line in database routers, Django never calls your router at all. It just silently ignores it.

6:25

Speaker 1: So this is a stuff that switches it on. One thing I didn't realize or I didn't say in the beginning was to guess the the workings of the second database installed. I did a pip install Django MongoDB backend. So that was to for me to have a local instance of MongoDB running in order to use it in order to use it as a second database So now the same question from a few slides back comes up. Does this data that I'm working with need to be locking step with other forms of data, or does it need to vary, or should it be standalone? In this case, I'm going to use a very simple example with like orders or let's say products in a catalog like some sort of e-commerce shop. So orders will need a first an order and then it's

7:11

Speaker 1: like line items updated And then catalog will be the second and it will need like probably laptop arteries and t-shirts arteries that don't share the same shape. That means it's a bit flexible. So now modern the catalog we have these fields. So we have the class product name, SKU prize, and specification We realized in this modeling for order and order item that the product and SKU, the product SKU, and name are plain fields here, and we'll see why we did that in a few slides after. So this is a simple class where we have order and then the order item. Apologies. I remember from Frank and um Calvin's top they said we shouldn't be using the

7:56

Speaker 1: the word clean and simple. So it's a class. Now migrations minus the safety net. Migrations will still work here mostly because this the same mental model applies, but there's a road trap. Because if you just run migrate with no flag, it will only touch the default because that's what it's supposed to do. It has it quietly and you don't really realize there's no error, there's no warning, it just skips it So that's why you see me being explicit by mentioning which exact database I wanted to be applied to that document. The second thing is if you are adopting data that exists everywhere, there's no there's not much of a shortcut here. Normally you reach for maybe Inspect DB to generate models from an existing database or table

8:41

Speaker 1: or migrate fake initial to tell Django that, hey, this already exists, don't recreate it. But doesn't exist for this backend. So you have to write a model by hand and hope that you match the shape correctly. Then this is the one you actually worry about a bit because SQL databases catches you when you forget to migrate. To do a migration when a column does not exist, you get an error thrown at you But with some NoSQL databases, it doesn't have that backstop by default. So your code will just quietly work against documents that doesn't match what your models actually think there is anymore So now for this demo, I tried different types of relational and non-relational databases.

9:27

Speaker 1: For this demo, I use SQLite, and the same enforcement reapplies for other relational databases in a deployment. So if you've built with Django apps before, you see that audits will have a line item that should point to a product as a foreign key. An other item will also point to a product. So we'll see that next So after running that, where it fails is where we do the migrations. Because make migrations doesn't touch the database at all. It only writes a description of the changes it has to make to the file. That step succeeds, no errors because there's nothing there's nothing to check against. Since relational databases can't add a foreign key to an existing table with a simple alter table in this case Django will try and work around that limitation by rebuilding the entire table behind the scenes, copying the existing data out, dropping the old table and creating a new one with the other

10:21

Speaker 1: column and renaming it into place As part of making sure that the rebuild is actually valid, Django will check the referential integrity of the new table, meaning that it needs to confirm that the table being referenced, which is catalog products Exist in the same database file, but guess what? It never did. So the router will keep everything belonging to the catalog app entirely out of default. That's the whole point of the router, because that's his job. So catalog products was never going to be sitting in that database. The check fails and then you see the error, operational error, and then you realize what actually happened. So any relational database enforcing the foreign key constraint needs a reference table to exist in that same database or table.

11:09

Speaker 1: So this is the feedback, or this is the error that I got, and why and what I learned. So here's the actual takeaway. Enforced referential integrity is a same database feature. I mean most of us know this, but we haven't tinkered or found out why it's that way. And that's what I was trying to really find out. Order and products were never a good fit for reference anyway for a Vincent database. Prices would change and products will get discontinued. And then a live reference will let old orders silently show new prices or break entirely. That's why other items stores products SQU and products name as a snapshot, if you remember the plain text field, not a pointer to the current state.

11:54

Speaker 1: So the split didn't enforce a compromise here. It already forced a design that was already accepted So what actually brought me to this? So in the beginning of the backstory I was telling you about how I was working with the project and in that project there were two databases and I was like Why do we even need two databases? But it was because some fields needed to be like hidden or encrypted. So with crybo encryption, I tried that out and I realized that okay, I could use two different databases simultaneously, where I could only hash Is it hash? Probably convert certain text into cipher field cipher text with QE. So the setup was a sim was a financial service style risk assessment API. It was Dora compliant and it meant that PII

12:41

Speaker 1: intensive data needed to be hidden at rest in transit and in use. So this is how I designed the model. Um I draw an ERD uh uh an ERD diagram for this highlighting what needed to be what needed to be hidden, what it needed to be exposed, and so on. So names, titles, descriptions, anything carrying PII, yes needed to be hidden, scores, data, and anything that needed to be filtered or no. So this project actually started the whole talk. So encrypted fields produce ciphertext as I was saying.

13:26

Speaker 1: And every time the same value gets encrypted, the filter and getOcreate method won't work against them. So that's why I had to choose which fields to encrypt. The other thing is performance is going to be a real cost because every encrypted field on every row that you return would have to be decrypted. So if it's a thousand times encryption, it has to be decrypted thousand times. So, what you trade away. So, with everything, there's always going to be some form of trade-off. For the first part, select for updates method, which we normally would reach out for to lock a road during a transaction. Does not really work here. It silently does nothing and the road doesn't get locked. So it's the same shape of shape of problem as we saw earlier.

14:13

Speaker 1: And it doesn't fail, but it also doesn't throw an error, which is sometimes even worse than even having an error. Secondly, no Django backend or the engine gives you transactions that spans across separate or different databases that Coordination is always on you. Now to three questions before you add a second database These are things that you probably need to consider before you add a database, not because it's shiny or it's something that's cool, because that adds a different level of complexity. Does a specific part of your data have an access pattern that your primary or your default database doesn't really serve or does it awkwardly? Do you have a compliance or security requirements mapping onto one defined part of data that you need to fix?

15:02

Speaker 1: Is your team willing to operate, monitor and backup a second database, which will be an ongoing process, not a one-time thing? Now, to this point, was all of this worth it? Because I mean we never ever tried this. We already know referential integrity. We have an idea. of what we should do and what we shouldn't do. So why is Afi wasting her time trying to do something that she knows will fail? Yes, the trade-offs are specific and documented. There's always like an avenue for exploration and an understanding why certain things wouldn't work in play. Honestly, polyglob persistence and complexity when a specific piece of your data, as I was saying, needs something that's that your priming database cannot fulfill.

15:47

Speaker 1: Not because a second engine looks interesting. And do you remember the foreign key value that failed before? That's exactly why it failed. And so you need to plan for what needs to stay together before you decide what needs to go apart. That brings me to the end of my talk. This is my experience, and the QR is the QR to the slides. And I'm open for questions. Any questions? Okay.

16:28

Speaker 2: Okay. So if you had decided to stick with a single database engine instead not do this multi-database split. What trade-offs would that have been in your uh what trade-offs would you have chosen to avoid in that case?

16:38

Speaker 1: If I had chosen to

16:40

Speaker 2: If you hadn't if you had not done the polyglot setup and just done a single database, what kind of trade-offs would you be losing in that? What w what what would have gone wrong?

16:48

Speaker 1: Well, in this case, nothing really would have gone wrong because you have the same database which has the same tables, but you don't need an extra database. So You wouldn't have to even um you wouldn't have like performance issues or performance costs or have to do different read and write operations from different databases. Sometimes the reason why people go for a different database on an additional is to serve a particular purpose. For example, in the QA example where I needed an encrypted database which was different for the from the unencrypted one. So where your de where your data or the access patterns are different, that is why you need a second database or when it serves a different purpose. Because we don't really see it as something that's really done like having two data two different databases existing in one project. We normally just have one and

17:35

Speaker 1: Yeah, have all our tables in it. So I was wondering why we don't have different kinds and why we don't see normally like two different databases in the Django project. It was out of curiosity.

17:53

Speaker 3: Thank you for your talk. Um you mentioned uh foreign key failure. Did you is it something you did expect in the first place or not?

18:02

Speaker 1: Well, um, well, the people who you do your dry runs with are grilling you on stage. Well, um Um no, so I did know it was going to fail. I just didn't understand why it had to fail. I understood referential integrity, I understood like the tables are going to be different, the shapes of the data are different, but I hadn't seen it in action. In previous experiences where I used to work in different organizations, I've seen the use of two databases within a microservice architecture, but I didn't start it. I've never created it from scratch This was my first time using two different databases at work using the creable encryption with encrypted databases and an unencrypted ones. And I was like, why don't we have different databases in one like relational and non-relational and that's how come

18:47

Speaker 1: I got to read it. Django Dogs 1. 2 I saw the new Um PR was from um I think it was a Django Con Europe talk was on transactions, spanning across databases. And that brought me to this point. So no, I didn't. I knew it was going to fail, but I didn't understand why and I needed to satisfy that curiosity. So that was it.

19:10

Speaker 4: Okay, um hello. So my question is if someone is on the fence and the person is trying to get this done in their projects, what is that one thing that you tell them

19:19

Speaker 1: Hmm, yeah, you have to really figure out if you really need it because once you bring an extra database, there's going to be an increase in complexity, and that's one of the things, and performance. So you have to sit down and really figure out if the data you are trying to save or work with is not cannot be handled or there's a particular need that's not being handled by the primary database. You must have a very, very good um case point to make sure that you add in a second database really makes sense and you have your team on board to handle the increase in complexity. And that's also going to increase costs, so that's money.

Conference

DjangoCon US

August 24 - 28, 2026

Chicago, Illinois, USA

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos by Abigail Afi Gbadago

More videos from DjangoCon US