The 750,000-line long pull request: crafting a more resilient open source community

Speaker 1: Hello everyone. So my name is Anne Olivia and I'm on Twitter if you wanna if you want to check that out. But uh other than finding very catchy titles for talks, uh I'm also a Python developer uh and I um I co-host uh the PyLadies meetup in Paris And I've been working on open source projects as a full-time job now for the last two years. And I something um that has been in the back of my mind for those two years. is to see the relationship between tech and community. And I realized that more and more I don't want to separate soft skills from like

Speaker 1: Other skills that would be not soft? I I don't know. But basically this idea that they're completely separate things is bothering me more and more. And I wanted to talk to you uh today about a case Where the fact that we worked on the community helped us to deal with a big technological challenge. So, what's going on? First, I'm going to talk to you about this project called OpenFisca because I need to tell you about a bit about the c the context around the uh 750,000 line pull request Um and then I'm gonna talk about building bridges, uh, which is one of the two things you can do with social capital. One is building bridges and the other one is doing some bonding.

Speaker 1: And we're gonna see when you can use um one and the other to deal with technological challenges. And finally we'll have some closing words and I'll take some questions. Okay, so what is OpenFisca? OpenFisca is an international and contributive open source project. The GitHub is OpenFisca. And the idea is to turn law into code. So the idea is that you take any fiscal or benefit uh law text of law and you turn it into Python. So As one of my friends said, it's like The Sims, but for real life. So this is an example.

Speaker 1: For uh countries starting out we have something called the country template and it's basically when you when you do a uh Django admin and then start project and it just creates the whole um architecture of your project. This is what you do when you want to use OpenFisca. You use our country template. And we have some very simple equations. I don't know if you know, but fiscal law is complicated. So I'm just gonna take that very simple example. But basically what it does here, it says that if you want to have the income tax of someone, then you take that person's salary and you multiply it by something called an income tax rate And that would give you the their uh their tax, uh their personal taxes. Um

Speaker 1: so what do we do with that? Because it's very nice to have Python equations, we all like them. But uh what do we do with that? Uh two things, two projects right now, um mostly in France, but in other countries as well. Um Lex Impact, which is a tool for Parliament So that they can know what effects changes in the tax law will have on certain types of households. And the other one is Miz Ed And uh there is a French version and there is a version for the city of Barcelona now. Um and the idea is that you input all your situation, how many kids you have, how much you earn, and all that information and then it tells you which um benefits you can apply to. So instead of going around and having to apply to one after the other after the other

Speaker 1: You just do one time this one simulation and it tells you everything you you can apply for. Um so how does it work? So OpenFiska is like a game console. Pretty much you have one big engine that we call OpenFiska Core. And then you have several cartridges that are the country packages. So you have one engine and then you can have the Tunisian cartridge or the French cartridge or the one for New Zealand. And you also have local uh cartridges, um, such as the city of Barcelona or the help from the city of Paris. And to do that, so we have the core, and the core is vectorial computing, which is basically you can run simulation on millions of households at a time.

Speaker 1: Which is great for researchers who uh use OpenFisca on anonymized databases of all the um French tax system. Um and they can do that because it's vectorial computing. So we have two types of experts in that community. One is tech experts who do the engine mostly, and the other one is economics experts who understand the law. So the more tech experts you have, the better open source project because uh you will have a code that is reusable. Because you will have tools to help new contributors come in, because you will have like complete documentation. And if you have more econ experts, ooh

Speaker 1: um your systems will be. So you have new use cases and you can use all that code uh to do more simulators and to create more value for citizens. But uh in all open source community, you have the issue of sometimes interpersonal conflicts. And when there is no social capital left, well sometimes these people fork. A note about forking. What is it? The idea of forking is that you have a community that has a product and then suddenly a part of the community wants to change and the other doesn't And so they just decide to move away. The problem with forking is that usually after a while

Speaker 1: you cannot really reconcile uh the two projects anymore So it's a big loss for a project when someone leaves because it's with time as time passes it's harder and harder for those uh contributors to come back And when I arrived on the project, one of the contributors had just left. And so we wanted to prevent this. So we worked a lot on as I mean as well as working on the engine and working on making the documentation better and working on creating um new content. uh we actually worked on uh this social capital, which it's like having a newsletter that asks you not a lot of work. I don't know how many of you started a newsletter and it kind of disappeared after a while.

Speaker 1: It happened to me a lot. Like It always seems like a good idea, but after what the while you need to um you need to keep on doing it. So we find a way to uh have a newsletter every two weeks And what we did is that we just uh took the name of the PRs we merged and we just list them. And it's good enough. It's good enough to have um a way to talk to your audience uh to show them that things are moving along and to show their work because as they receive the newsletter they tell you about their news and you could put it in the newsletter. We started having monthly after work events and co-working sessions where new contributors will come to our offices and can get bootstrapped and ask questions in real life.

Speaker 1: And for people who are not used to working open source, being able to talk to someone instead of writing down an issue can be something very useful. And finally, we started having road mapping workshops where everybody in the community could come in and we could agree together where that core component should go toward, what it should go towards too. Good towards. And so this is where we were. We were working on knowing more about our contributors, and suddenly this happened. One fine morning. Um so this is the uh error message GitHub gives you when uh you have more than 3,000 change

Speaker 1: files. So I never saw it before. And the first thing to do is not panic because uh things are not as dire as they seem. Um we panicked a little bit at first. Uh, but then we realized we know this person. We've had uh you know, coffee with them and we've worked with them on other projects. So we just give them a call and try to understand their context. And s what that's one of the first thing you need to do when you work on that social capital inside your open source project is to understand that your context as a developer might not be the context of other people who have other kind of work environment, other experiences and other

Speaker 1: uh way of working and especially in the economics world, uh for example, one of the things that we discovered is that their habit was to finish a project until the end before they would show it to the world. So the idea was this person, she worked for three months and she updated the whole French um tax system. So this PR would like update everything tax-wise on OpenFisca, France, which is a huge deal for us. Um and she just waited until everything was perfect to open the PR, which is a good intention. So we analyze it with her and we discover that she uh actually wrote a script to um to generate

Speaker 1: automatically tests Which was great. I mean she understood that tests were really important, but they were randomly automatically generated. So once we took that out, we went from 752,000 lines to a mere 215. So that was a big you know improvement on the uh work we had to do now. Um but still it's a it's a it's a large amount to l of lines to uh read. So with my colleagues there were three different strategies. One was just push the merge button and let's go to lunch strategy. We'll deal with it later. Another one was just reject it, just close that PR, say no. And but that would mean

Speaker 1: maybe our uh contributors who do such great work would just fork and Maybe maybe not come back. And the other one was to find a compromise. And that's the option we chose because we thought this is just too much work. She worked for three months. If we close it now, if we say no now, like in in human terms, it's an it's not something you come back from very easily. So we met. We met for half a day and we talked. We talked about their context. We talked about what they could give in and what we could give in so that we could move forward together. So in the end, we agreed on 15 smaller PRs that would amount to the 200,000 lines of code.

Speaker 1: We agreed on review guidelines which part of the PEP 8 we would apply, which part of the testing we would apply. And especially uh the point that was the most important is that they needed the tools to s to to succeed. Because telling someone Just you know just do just do cher uh cherry pick and then a rebase and I'll come back in a few days. Uh this doesn't fly. You need to be with people and you need to hold their hands at first So that they know how to do this and then they can teach others how to do this. A quick point, what is Git Rebase? Git Rebase allows you to uh take a branch from uh your project and then

Speaker 1: Actually base it on another comet. And cherry picking, it allows you to take a comet from a branch and put it on another branch. Which is quite useful. But those are technical tools. The real thing that we developed Is social capital. Because by create by doing those 15, it took us two months, 65 days to merge those 215 lines of code. And by doing this, we created social capital. We created trust in our project. So what is social capital? Social capital has many ways of being defined. It's quite elusive, so I'm gonna define as it as the quality and momentum of

Speaker 1: relationship within a community. And there are ways you can create that momentum and that positive quality in your project. And the first thing we want to do is put energy and enthusiasm in what you do, but also what others do in your community. Not only should you should, I mean if you want to create that social capital, something that is great is organizing things, organizing conference, organizing events around what you do Um ease cooperation, make it easier for people to work with you and find things that are kind of hard. Let's say a documentation that is not quite right. It might be doing our country template

Speaker 1: was something that helped a lot of new contributors work with us And finally, work on trust and reciprocity. Having guidelines that you never budge on will not create that trust. Trust means taking some risks And people can then take risks with you. So it's important to find those opportunities to build trust. So once you have capital, what do you do with it? There are two things you can do with it, and I think I mean there are many things you can do with it. But two of the main things are called bridging, which means that you're gonna create external ties For example, for us, we used our social capital to bridge the gap between our developers' culture and the economic

Speaker 1: culture. Um but the other thing you can do is called bonding, which means like strength strengthening the ties within your community And uh I have another example of a open source community who use social capital to do something great, but more into the bonding than briding. I don't know if you heard of OpenStreetMap, but it's a contributive open source project. And they had this project of delimiting city limits in France. There are 30 more than 36,000 cities in France. It took them six years to do it. So it's an open source project doing a six-year-long technical challenge.

Speaker 1: And from their own account, having some community meetings, having their yearly conference was one of the determining factors in their success So finally, what does it mean for open source? Well, we often think about community uh work uh as strengthening the bond between us And I think it's great. But if you want more diversity, I think we should use that social capital we build to actually build bridges towards other kind of skills that we have, but also other kind of people that we usually see in open source. Because now that if we can create the tools they need and the paths they they need to become part of our communities, I think will be richer.

Speaker 1: Uh and we can have those like open source projects that are well designed and well marketed. and uh who can be you which can be used by other types of people. So thank you very much. And if you have any questions

Speaker 2: Thank you. Are there any questions? There's always a question, Russell?

Speaker 3: Sorry, yes, there is always a question. Um the the contribute the contributor who submitted this uh uh hundred thousand line patch or where however many it was The question that comes to my mind is how did we get into a situation where they got so far down that path before they came to you where we even knew they were working on that? uh preempting a problem is always better than than solving it. How much do you think the role of setting up expectations about how people are going to engage before they start engaging is is part of this process?

Speaker 1: Um so if I understand the question well, uh you are asking uh first um was there any way for them to kind of ping us beforehand before we got to this part um and uh than if we could have some kind of documentation for it like a a ritual Um so what I understood from my conversation with them is that for them you don't sh well for her beforehand, you wouldn't show your work before it was perfect. There's so if you have that in mind, there's no reason for you to ping anyone until it's perfect. And I mean if you want information you have to look for it and that's the thing. If you if you think you know you're not gonna look for that information.

Speaker 1: Um but now sh now now she knows so now she tells her colleagues and so we have a better community for it So having gone through this experience and having been on the other side of this experience now, uh it really helped the community as a whole learning this together. Uh but it's still an issue and I think it's it can be an issue for any community working with people who are not used to just opening a PR as soon as it works. There's this idea of pride, there's the fear of being judged on your work, and all of that I think we need to have also , we need to behave ourselves better when we answer and when we talk to new contributors. uh to show them that any work is good work.

Speaker 1: And I mean at least that's what I believe. And I believe we can have better open source communities if we can convey that this idea.

Speaker 2: All right. Are there more questions? Are there questions from the internet audience? No. Okay, thank you again. Thank you very much.