You shipped it, you fix it

Oh Glad you can make it. This is the last talk of the day and I'll be sure to make it short so we can all get to the beers You probably seen this graph when you came in. It's obviously not entirely true, not even remotely true actually, but it serves as a good point of departure for our conversation. Um so my channel is called you ship it, you fix it and it's going to be a whirlwind of advice or experience that we've gathered over the last couple of years. Hopefully you'll be able to avoid some of the pitfalls that we we ran into. So how many of you guys work in

an organization that looks like this, where operations is totally separate from development? See a few ways, yeah. Um so you've probably run into this problem where The locals load some code, they give it to apps people, it blows up. Um it's actually a pretty bad problem. Um And the the problem is sort of this. Developers write the code, they give it to the apps people. If something breaks, the ops people get paged and they have to figure out if they can fix the code or they need to wake up a developer. And it's kind of You know, the problem is accountability here. If you know that someone else is going to be paged when your code breaks, you're just gonna write slightly worse code than you would if you were gonna get paged if the thing breaks.

That's just human nature, I guess. Um so we've employed uh policy of If you ship it, then you fix it. And a prerequisite for this to be useful is that if you build it, then you ship it. So you ship it, uh sorry, you build it, you ship it. You shift it, you fix it. So you're accountable for the whole process, the whole way from building code to actually making sure it runs in production. So is this DevOps? Maybe. Who knows at this point what DevOps is? Uh everybody everybody is

kind of not in agreement about what it is so I Googled it and this is there's a lot of different opinions on what developed this. These are some of the ones that kind of um They're mentioned the most I guess. And they're pretty good. I like them. So empathy empathy makes a lot of sense. Empathy across departments. Trust and respect. um working towards common goals and breaking down silos in the organization. That all makes a lot of sense. So This could be what DevOps looks like in a modern organization. You've outsourced most of the hard corporation stuff to Rackspace, Amazon, Heroku, etc. And then you have developers. You can't see it, but down there says developers, and there's perhaps also an aut

person. And they're working together and they're doing development but also web operations. So this is an example of what DevOps in modern um modern organizations could look like. So let's say you you got this DevOps thing figured out. You bought some DevOps for the team, everything's good. What can you do now to actually improve the situation? You got that ups mailed. One of the key points that we've found is that you need to increase transparency. And that means you need to be able or developers need to be able to figure out what's going on. And one of the important ways to do that, or good ways to do that, is to make sure that the developers have the tools they need. So good tooling is really, really important.

Making sure that developers have access to the best tools they can find and that all developers have access to these tools. Here's an example. It's pretty uh pretty small up there on the screen, but the idea is that an error happened. Um you log into a system, you can see a stack trace, you can see who actually wrote the code. You can see when it went into production. So this is a snapshot of Up Eat. And it's a way to increase transparency. You can see exactly who wrote the code and when it when it went into production. Another example of increasing transparency is this. This is um a screenshot from a Slack room

and what happened here is that a penny who's a great guy he accidentally broke the build And uh Venya tells him to uh start doing push-ups, so he has to do 80 push-ups now because he broke the build, and Benny doesn't like it. Um So that's another way of increasing transparency and making everybody aware of what's currently going on. Here's a screenshot from the Rollock, which is also a great tool. This shows Performance for webcam and making sure that everybody can go in and see how their code is performing right now is extremely important to making sure the developers care about performance So a big part of this is making the information accessible to developers. If it's hard to get to this information, they

will usually just don't go and get it. We're lazy. Here's another um example of a way to increase transparency. Make sure everybody knows when new stuff goes into production. People will be aware of what's going on in production if they actually get a ping that something is now in production. Another screenshot from our cyclone. So that was a little bit about how you can increase transparency and how that affects accountability in the organization. Now I want to talk a little bit about releases and sizes of releases and frequency of releases. So you can either do big releases This isn't even a big release by some standouts, but by our standards, this is a huge release. It took six weeks to prepare this release, and it has 77

commits and a bunch of changed lines. There's only two people involved in this release, so that's a good that's a pretty good uh pretty good metric. This is a small release. Six minutes from the commit was um was made to the hit production. Uh only one commit and only one one person in this uh release So the idea is that if you make small releases, it's much easier to figure out who broke stuff because fewer people are usually involved. And what happens is that If you have one one developer per release and you find yourself in this scenario where there's a release that's suddenly causing errors, you can go in and and uh and see immediately who who knows how to fix this.

Besides that, there's a bunch of good reasons why you should be doing small releases. For example, developers love to put stuff in production. It's a great feeling when you finally put things into production. So you'll have happier developers if you do small releases. But there's it's a huge topic, and uh I encourage you to go and investigate it even more. So You shipped it, you fixed it, you build it, you ship it. Or as Vera likes to say runner from s from uh Amazon, you build it, you run it, which is sort of a graduation of that. And running it here could mean a lot of things. We usually try to make sure we have to run as few things as possible ourselves.

So we like to buy services so other people will run it for us. But still, you know, you get the point. You're responsible for the code you write. Um okay. Now to something a little bit different. Do you know this person? I'm just a developer. Just let me know what I should build and I'll build it. And a variation of that is I just build whatever the spec said. The spec set A, I did A. But is that really how we want our work as developers? Developers are smart people. We know how systems work. If we're really good at our job, we also know how users users think. So here's an example.

Again, this is Benny. He caught this quickly. He's a great guy. There's a nice nice spec here. We're changing how timestamps look in our product. And it's spec out pretty rigorously what it should look like. Now there's this thing that I've highlighted where it says if the time is more than a year ago, it should include the year as well And there are two ways you can interpret this. If it's more than a year ago from the time it should include the year, or if it's the same year. That should not include the year. You know what I mean?

So it's either 365 days ago or just within the same year. So Then he got this really um extensive spec, but he still went to ask the product guy, what exactly do you mean by this? And it turns out that The most obvious um interpretation of this was not the one that the product guy meant. So it's a way to or what I'm getting at is that Developers shouldn't just be these code monkeys who get specs thrown at them and then start to start to code, right? Developers are smart people. We usually have a good idea about how things work. So Why should we just stop at dev and ops?

It's great that we're trying to break down the silos between the development department and the ops department. But why not break down some more silos? So also break down the silos between the developers and the product development or the product team. and break down the the silos between the development developers and the customer support team There's a lot of advantages in these, and you can probably imagine uh where you can go if the developers have more insights into what's going on in the organization in general, what kind of support requests are coming in, what are the thoughts from the product development team on how things are going to look in the future. I just call this the last product that we have to custom product.

Um Yeah, that's it. So installing DevOps new world order, maybe there's a lot of good ideas If you can increase transparency in your development team, you'll have better accountability and in the end you'll have better code and everybody will be able to sleep more, which is great You can use tool to increase transparency and affect the culture on your team. And you should try to break down sidos, not just between development and operations, but between many of the departments in your organization That leads to better products and better solutions. That's it. Some of this was slightly maybe controversial. I'd love to discuss it with you guys,

girls. I'll be right up there, so yeah, come talk to me.