Production Django on Platform.sh: reaching operational maturity in days, not years

Hi everyone, thank you very much for coming. I have 10 minutes for this talk, so I'm gonna jump right into it. Um PlatformSH, if you haven't heard about us, is a in part a platform as a service provider. And the way we like to think about what we're trying to do is this idea of operational maturity. So we are all delivering applications and we have these pretty clearly defined stages of that delivery lifecycle. And ideally, we'd like to keep the whole life cycle secure as they go through each of these stages. And part of the complexity of managing this life cycle comes from people at each stage that own the tools, the tools themselves, and the configuration that passes our software from us writing it to it actually being delivered to our production environment or any other environment.

And the challenge then becomes in managing all the little pieces that make this workflow complex. So we can imagine in our packaging stage of the life cycle, we have a subject matter expert. And one day she gets a job offer that's twice as much as we're currently paying in the organization, and she leaves. And how quickly can any of our internal tooling adapt to the loss of this person? How well and how quickly can we onboard this new person who it's probably a little bit of an expensive new ad to our organization And software works the same way. Things are evolving and things are deprecating outside of our control that we depend on in our organization. And what are the changes that need to happen and how flexible can any of our intro internal tooling

Adapt to Jenkins no longer being supported and having to switch to some other tool at this really critical stage. Um And it won't necessarily be just one application. You know, it's going to pile up and become more complex as we add more apps to our organization, whether that's we have many different clients that have individual instances of our main API. Or we're delivering custom apps for each of the clients in our agency, or we're doing microservices with many interconnected apps You know, how do these life cycles fit together to manage all of the apps in our organization given all of these moving pieces? And so when we talk about operational maturity inside of our company, we're talking about Flexibility, maintaining security, keeping compliance in mind, and uh

what are the tools that we build or the tools that we choose in order to cover this ground. And so obviously that's some standardization to manage the complexity. And for many teams, that's when we start to talk about an internal platform, platform teams. uh connecting all of these tools together into making something where any app that we are trying to deploy can become part of the same system to make this as efficient and repeatable and reliable as possible. And so where platform comes in is saying starting at this packaging, provisioning, and deploying stage of the life cycle. If we can just accept from the outset that everybody's committing infrastructure in YAML and everybody is using Git, how much can we really squeeze out of the Git protocol? to take care of most of this life cycle for us

and then connect other little pieces to it so that we can get the same sort of platform that we would build internally, but get us to that place of operational maturity more quickly. so that we can focus on things like writing the application itself and then some more fun things like optimizing this whole process and uh working towards better performance Rather than making it this huge secondary task of our organization of building and maintaining an internal platform. All right, so that's figures. Let's go to code. So this is what Django looks like on our platform. And so this defines an application container that's using Python 3. 9. It has a build phase to uh install dependencies. and to build the app before it runs migrations in its deploy phase and has a start command to use a GUInicron

server to actually run it. And so from here we can also define explicitly access the other containers in this cluster that make up our production environment uh where that that that access is allowed. So in this case it is a Postgres database where we have another file that looks very similar for services that are built into our platform that avoid the the need for anyone to know how do I provision and put together a Postgres or a Memcache or Elasticsearch container. I can put this line in, provide access in that relationship key, and I start putting together this cluster of containers to define the production environment. So in this case, I add the files to my default branch to build the cluster, push it up to our platform, and then our platform will actually

Notice those config changes, validate them, provision these containers, build a new image based off of what we've defined inside that app config file. uh and set it, deploy it inside of those containers so we get our production environment. So that's that's default branch, that's production stuff. So that I think takes away a big chunk of just the main uh right half of that deployment lifecycle. And what we can do is we can enforce some standards in here if say we can only have right access to the container at build time And at deploy time we can pull credentials to connecting to a database right from the container itself through environment variables, which fits really nicely inside of the settings pie

file, which in this case I have a platform relationships environment variable and from there I can get username, password, anything for any service container I connect in that cluster. And from there, we kind of are setting up what we would consider environmental, environment-independent builds So it makes solving a problem like this relatively simple of in your own internal platform, how long does it take and how many people are involved in upgrading an application from 3. 9 to 3. 10 or any other piece of infrastructure? That's for purpose of checking out new features or improving performance. And on our platform, it's creating a new branch, leveraging Git, and editing this file.

And so why that works is because if I create a branch, there's actually no change in the unique hash that um identifies and character characterize the state of that default branch versus the new branch. And so we've already pushed to production and created that build image. So we can just move it over to this new branch. And actually take production data alongside with it. And as a result, by any new pull request that's open or any new branch that we activate on the platform, we get this exact copy of production in this new space And so we have an isolated experimentation uh area to check out what is this change in infrastructure going to do? Is our production app gonna blow up?

Are we gonna get the features we want? We could throw it away if it doesn't work. And if not, we could start promoting it and continue to leverage Git to actually merge into production. using the same idea. So in this case, we battle test it in the isolated space in the true staging environment with production data where we have another unique build image for Python 310. So now we can in this promotion to uh production. Go ahead and reuse that same build image here. And there's no chance of some unexpected post-merge build error happening in that transition because we've already built the image, we just reuse it. So the goal here then is to say we can take that rule and kind of build everything on top of it to

get to this operational maturity stage. So If we're building an internal platform, all the moving pieces may put us in a position where we kind of have to stick to a dev stage prod model in order to coordinate all these moving pieces. But with this approach, we can just say we're dealing with environment-independent builds, and for every branch and every pull request, we get a true staging server, we get a true staging environment for our application. So that means we could start separating workflows based off of feature sets or external vendor collaborations of somebody working on copy or just the front end. Uh we cuse me, we can get revert in any time, but we can start running a cool experiment with the new piece of technology. So it becomes more adaptive to where an organization may be five years down the line that we might

not anticipate. So now the context would be. Do we want to provide a separate front-end application or do we want to get some new component of the API that's in Go or Ruby? It's easy to just create a new environment and explore that new aspect of where our app could go. And um the basic idea is that this API can be generalized across all of our projects, and it's very easy to do repeatable tasks that are involved with maintaining all of our client sites by saying define custom endpoints since we have these rules at the base of everything to say Update dependencies in a dedicated environment every one week, or pull from the common code base upstream in the same type of dedicated environment every day.

Um or um infrastructure upgrades just like I showed with Python 310 to 9. Done. That's that's ten. Okay. I just wanna close out quick by saying that with that simple rule, we can set up a place where The operations team and everything around our application, where they're spending a ton of time working on the internal platform, could start focusing their energy on optimization. and performance and all these other things that aren't necessarily building and maintaining the tools, but making them work better for the business and have less impact on the environment, hopefully in the process. Thank you all very much for your time. We're at the booth right outside if you want to talk more.