Lightning Talks

Speaker 1: Alright, so my name is Ricardo. I'm sort of an hybrid. I'm like 50% scientist, 50% software engineer. So I'm going to talk about neutrons today. So I work for the spellation neutron source uh at Oakridge National Lab. So this is a picture of uh of uh the spallation neutron source For those who don't know, this is in Oakridge. Okridge was uh one of the secret cities that was built for the Manhattan Project. So uranium for the the the little boy, the bomb that was dropped in uh over over Japan um it was was enriched over there in Oakridge. Things changed, it's not secret as anymore. And today for example we have the the most powerful computer in the world

Speaker 1: We also have huge programs in biology, in material science, we have environmental science We also have bokats and bears uh in the parking lot. And this is really true because I've seen them. Okay. Um So just a quick introduction about spalation neutron source. So we put we we work with neutrons. Neutrons they have properties that allow us to study matter. They have uh um they have a size that is uh uh a wavelength that is similar to the distance between atoms. So we can study matter with neutrons. So to produce neutrons basically what we have is we produce hydrogen ions. So an atom of hydrogen with two electrons. We speed this to the in a linear accelerator to roughly 90%

Speaker 1: of the speed of light We then chop off the two electrons because you don't need them. And then we accumulate them in this ring. And then we fire this. to uh a mercury a mercury uh target and we obtain neutrons like off and uh those neutrons they are then uh directed to instruments Okay, this is an example of an instrument that we have here. This is pretty huge, as you can see, we can see like a little guy over there. Um So basically what we have is we get those neutrons. We have a sample. I don't know, can you see the mall? Yeah, cool. Okay, there's a sample over there. So we shoot our sample with neutrons. Those neutrons are scattered And we get them on detectors.

Speaker 1: I'm not going through the details how we get the neutrons and all that. So basically, how we use Django. So Django is used everywhere along all this process. So all the instruments we have I think we have around twelve, fifteen instruments. They all have Django somehow. So we use Django to monitor everything. And this is an example of how we use. So you use Django to monitor all the process. We use a lot of JavaScript, fancy plots, fancy Fancy stuff. Alright, and then there's another thing that we also do. So we do data reduction. Because our data is huge, we have to submit our data to a cluster. This is part of the project that I'm working on. So The idea is something like this. We have a reduction and every reduction is composed by several scans. We want to minimize errors. So what's

Speaker 1: what the users or scientists usually do they measure the same thing several times. So for example for a small angle scattering experiment which is basically measuring the deviations of of our of the scattered neutrons We have the same sample, a protein, for example, and we dissolve it in several concentrations. So we measure everything, we average, and we have some data. Okay, so um then this is submitted to us to uh to uh to a cluster. We can either submit a reduction composed of several scans as a simple MPI job or a single MPI job or multiple jobs. And now this is what I'm working. So I I don't have much experience with Django and I've got a lot of questions. So I'm taking advantage of these five minutes to get response from you Okay, so what we have what I have basically is I have my reduction, my reduction is composed by several scans, and I have this job.

Speaker 1: What I submit either a reduction or I submit several scans So I have this polymorphic association. I have no clue how to do this in Django. Okay? Uh another thing that I have, so I have my results, and my results It depends on the inputs of the of that the scientists give me. And they they can be the ideal for this would be JSON. But JSON, so far as we heard after lunch We don't we only have JSON with a key and a value. There's no nested JSON still supported in uh in uh in Django So whoops, I'm summarizing everything. So basically I need something like this where I get all my objects and I get the type associated with the reduction on a scan. I also have something that I didn't tell. So I have Uh my my

Speaker 1: so the forms that you saw that you saw before just to submit data, they are cluttered with a lot of so I have like one hundred entries in my form Even more sometimes. They're all conditional. Like I have default values. Sometimes the users tick a button and I have like a jQuery very fancy thing that show up to be parameters that I have to change. And my templates, they are cluttered, they are huge. And I wonder if I can have this if there's a way to have this better. I saw this package, crispy forms, I don't know if it's good or not I have seen requests I talked about with some people, some like some some don't. Also about about uh JSON. I saw well so the JSON, the proper JSON is not still supported in Django I wonder if it's going to be support in 1. 185 or only in

Speaker 1: 1. 9. Well, and that's all what I have for you. So please come and help me. Alright? Thank you very much. So

Speaker 2: this talk is about sorted containers. It's a Python module I've been working on. This is going to be sorted containers in five minutes My name is Grant Jenks. I've been working on Python since about 2011. Like many of you, I was pleasantly surprised and impressed by Python's batteries included slogan. Not long into Python development, I started asking questions like these. How do you sort a Python dictionary by key? How do you sort it by value? How do I get the maximum value out of a dictionary? Could I somehow search a dictionary's keys? These questions get viewed millions of times for every priority queue or cache or in-memory index or even indexable set that somebody needs.

Speaker 2: In response, I got some great answers. The first is ordered dict. So you might have actually also used Django's sorted dict at one point. You quickly learn though that ordered doesn't stay sorted, it just stays ordered. There's also counter, which is pretty clever, another uh Raymond-Hedinger contraption. It's combined with another module called HeapQue, which helps you maintain a binary heap. And all of these were wonderfully fast except when they weren't. So they kind of assume that updates can be batched and sorted methods can work lazily. And that gets us really far actually, like really far. But sometimes you want even more. So if C<unk> and Java and. NET all have sorted map-like data types, why doesn't Python? I found

Speaker 2: the answer from Nick Coughlin paraphrasing Guido. If a user is sophisticated enough to realize that the built-in types aren't the right solution for their problem, then they're also up to the task of finding an appropriate third-party library. To that, I say good luck. Uh if you go out there, you'll find a dozen different solutions. They'll all have different APIs, they'll all have varying levels of compatibility and performance. And if we decided only by most downloaded, then we'd end up going with B List. So let's try that. BList implements a B tree data type in C. With a node size of 128. And I was pretty happy with it until one day I made this discovery. I'm surprised here because B list should shuffle 128 pointers, while bisect should shuffle a thousand

Speaker 2: pointers. Like how can bisect be faster here? So I went back and I uh tried trying tried to write something in Python. I tried evaluating a lot of different kinds of sorted tree data types. There's red, black, there's skip list, there's splay, there's randomize, there's treep. A lot of these are all written in C or C. There's one that's a skipless implementation written in Python, but it was kind of slow. All I can say in five minutes is that sorted containers runs with this idea. It's kind of like a B tree, but it's it's only half-heartedly so. It relies entirely on the bisect module and While it's slow to program in Python, the interpreter, the Python interpreter is written in C. So if you think of it as programming the interpreter,

Speaker 2: You actually are writing C code. And it turns out lists are really fast. Like really fast. And trees, not so much. So listen to what some smart people have to say about it. Alex Martelli writes, good stuff. I like the simple, effective implementation idea of splitting the sorted containers into smaller fragments to avoid the order and insertion costs. Jeff Nupp writes, uh that last part, fast as C extensions, was difficult to believe. I would need some sort of performance comparison to be convinced this is true. The author includes this in the docs, it is. And Kevin Samuel says, I'm quite amazed not just by the code quality but the actual amount of work you put it stuff that is not code. Documentation, benchmarking, implementation explanations. And it was a true inspiration actually from Django

Speaker 2: to be this driven to create that much in terms of docs. Lastly, uh Mark Summerfield makes a short plea. He writes, Python's batteries-included standard library seems to have a battery missing. And the argument that we've never had it before has worn thin. It's time that Python offered a full range of collection classes out of the box, including sorted ones. So I submit sorted containers. Thank you.

Speaker 3: So I hope you all forgive me. Forgive me for doing a pre purely promotional talk here. Marketing, marketingness. I'm Tracy Osborne. You might have seen the book I've written, and I'm really, really proud of it. And if you haven't seen it yet, come find me. Hello Web App is an introductory into Django, uh, and I aimed it at non-programmers and designers and people who are used to using the websites and I start out with templates first and doing static files and check out the website. So this is the original book, and it's awesome. You can buy it off of me here. And it's on Amazon as well for later. And I wanted to point out the most helpful critical review, which I think is really funny, where it says

Speaker 3: overpriced book. Hey, I self-published. Very introductory, certainly not worth the $20. At least it's on luxurious paper. So I'll take it. I'm a designer, I dri I ri I did everything. By content is well written. Thank you, Mr. Critical Review, or Mr. or Mrs. Um, but I'm really up here, I really want to expand this. There we go. This is why I'm really up here because I wrote this book and I am fundraising for the second book, which is Hello Web App Int Introductory Concepts. So anyone who has gone through a Hello Web App and has a basic web app, or the Awesome Jingle Girls tutorial, or any other tutorial that launched them into their first basic web app, this new book We'll cover intermediate concepts like adding Stripe, adding uh

Speaker 3: an API, working of sessions, adding user-uploaded images. So all these like Things that you would build upon a basic web app. And it's going awesome. I just launched it five days ago. I'm almost at the minimum goal, which is fantastic. But um to Kickstarters of to very least at Kickstarters, if you back the Kickstarter campaign, you'll get the new book by December. And if you back the first campaign, you would know I'm a year late. from that one. That will not happen this time. I promise the book will be in your hands by December. So yeah, Hello Web App Intermediate Concepts. That is fundraising on Kickstarter. Watch the video because you get to see faces like this in the video. And thank you to everyone who backed the last Kickstarter and has been so supportive of my journey with my book.

Speaker 4: Wow, Django Khan. Alright, my name is Dan Dietz, and I'm going to be giving you a quick overview of something that we built called Fabric Bolt. And the title of this talk is Look Ma No Command Line. So go ahead and wait for the presentation to pull up here. Okay, okay, perfect. Alright, a little bit of a false start. Uh first of all, we've got a company, Boltifekt. uh that supports this project. So I'm just gonna give a shameless plug for our company.

Speaker 4: Django Dash. Uh 2013 was where we originally developed uh this piece of software. Uh so I don't know if you're familiar with the competition that hasn't run for uh a couple years now, but we uh I'd like to say we take we took first place, but We actually tied for first. So what we tried to do was solve some problems that we saw. Built a tool that's designed to help with deployments. Okay, so uh there's lots of people that use fabric. I'm not gonna get into that, but when you're using fabric to manage deployment systems, uh you need programmers in order to execute uh those deployment tasks. Um you also need your programmers to have permissions to connect to appropriate machines so they have to have

Speaker 4: the elevated permissions to you know go out to your production servers and all that good stuff. And then you also have to keep track of what's actually happening. So when you run those deployments you want logging, you want tracking, know what's being released Okay, so uh what we wanted to do was be able to take something uh safely uh to production uh without you. So a normal workflow could look like this. All kinds of organizations have different workflows, but you could be doing programming, send something to QA, you have then client review, potentially in a staging environment, something like that Project manager is working with a customer saying, hey, great, uh, this is all good to go. We got it approved to go on Friday, right? So it's Monday, it's approved to go out at some point in the future. So then Friday rolls around, they stop you from working, uh, because you're happily working on the next feature, uh, and then you run that deployment.

Speaker 4: So um Wouldn't it be great if uh that project manager or some other non-technical person could go ahead and run that deployment Uh it's kind when when we have a workflow that's constantly interrupting a developer to run those deployments and manage all of that stuff, it's kind of like the driver here having to get out and change his own tires. Uh so that's at least from our perspective Uh access control. Um you can have a system like this where you have developers that have machines and uh they have to have their SSH keys installed and then those SSH keys configured on all the those other machines out there. Wouldn't it be great if we had a system where uh we could have a central place uh where those keys are stored, it has access, and then we hand out permissions uh in a nice web interface to different types of folks to give them different levels of access.

Speaker 4: Being able to track releases is great. So we had a fantastic talk from Lincoln Loop earlier today where they were talking about doing um releases and best practices and uh you don't want to be able to answer a question like oh I think we released this you know sometime last week. We want to be able to say yeah you know 342 um on Monday is when it went out So what uh we did was we built a web interface. It's a Django project to run your fabric tasks. Uh and so you can create users in the system, you can hand out permissions, you can manage all that stuff And then you can actually track what happened when those deployments run. So it's really easy to get started, pip install fabric bolt, you do your init, you migrate it, you run server, and you're up and running.

Speaker 4: And then you have a beautiful interface that looks like this. Welcome to Fabric Bolt. You would go in and configure your hosts. You would configure stages. So you'd say, okay, these hosts are what I use for doing um you know my testing staging whatever and uh I've got a stage over here for production. Um Once you actually get to the point where you want to run a deployment, you would select a stage, and then uh you actually can see over here on the right hand side, there's a drop list to select from the task you'd like to deploy. Once you go through that process, you click deploy, you actually see what happened. So we have a status that shows, yep, everything was great, and here's the full log of everything that happened. So all of that beautiful log that you would normally get working on the command line We capture, store it, we keep it for later.

Speaker 4: So in the event that something bad would happen, we go back and we review, okay, here's all of my deployments. They look wonderful, but I did have a failure. You could actually go back and say, okay, what happened during that failure? You know, what was the issue Uh what I really wanted to do was invite people to help us on the sprints. So uh we've been taking feedback from people. So uh last night at the party We ran into Mike, asked him some questions. He's like, man, it would be really great if we had a feature like that. So we would love your input. Uh come help us, give us ideas for features, how to make things awesome. Our uh core developers that maintain the project are here. So uh Thursday and Friday we're gonna be sprinting. We've got Nathaniel, myself, uh Beth unfortunately uh

Speaker 4: didn't want to come. She's like not a programmer and gonna be bored to tears so didn't want to come. And then uh we've got uh Dallas resident the Texan, aka Jared. Um in fact I think his picture's a little bit bigger there. I think yep, everything really is bigger in Texas You can check us out on uh fabricbolt. io and then of course the GitHub project MIT license, pull it down, test it out, try it. Um and uh that's all I've got

Speaker 5: Okay. Hello everyone. I'm Andrew Goblin again. And I am here to talk not about databases or migrations this time, but instead about a project I'm working on for the last couple of months I'd say. So the background to this is I've always been annoyed by the lack of support for things like WebSockets and Django. I love WebSockets, they're pretty great. They do a lot of interesting stuff. And so, oh, ah , we're having interesting. Let me try plugging in again. Hang on. Perfect. Okay. So, um so channels is a sort of idea that I I came up with a while back and I've sort of started implementing now. And so what it is is it takes one of Django's most core concepts and changes it around. So Django is a web framework that's tied to request response, right?

Speaker 5: Everything you do in Django, the middleware, the URL routing, the views are all part of this request response cycle. Projects like Celery try and break that out by having tasks, but the core part of Django is still that. And so what I said like, you know, what if that wasn't it? What if there's a different abstraction we can take here that still preserves the simplicity of Django, but that should do stuff with WebSockets with HTTP2 push. with background tasks, all this other stuff that happens kind of outside that old HTTP1 request response world. And so I took the core idea of a view that takes a request and returns a response and took a new abstraction, a sort of slightly lower one that is called a consumer. And a consumer listens on a channel and it takes a message at a time and it returns zero to more messages. Now, this seems very simplistic and it makes no sense

Speaker 5: out of context, so let's see some context. So first of all, what this means is that you can implement a view as a consumer because what channels does is it makes a channel for incoming whiskey requests So you have a thing that listens on that channel, it takes the requests, it does stuff with them, and it sends back responses on the response channels. Now, this is just left plugged into Django's URL routing system by default, but If you want to, you could do low-level stuff here. But that's not the fun part. The fun part is that as well as all the normal HTTP Whiskey stuff, you get lots of other things as well. So for example, channels comes with WebSocket support. So you run a WebSocket server and then you get WebSocket channels you can connect to or you can listen on. And so you can say, when someone connects you can say to you can just have a view that talks to a model. when someone

Speaker 5: you can have custom channels. So you can say, okay, whenever this thing happens on the site, send a message to channel name X. And whenever that m channel gets a message, you can send stuff back to WebSockets and so on. And then That's kind of brought together in a concept called a group. So one of the things you have with WebSockets in particular is lots of clients connect and you want to sort of group them by that way These people are looking at the live blog, these people are looking at our front page, they're looking at the chat over here, and you want to be able to send messages to all of them at once. And so there's also a group abstraction channels where you go, okay We're going to have whenever somebody connects, we have a thing listening on the connect channel for WebSockets, and it puts them into the group as they connect and handles that stuff. And you can do more, so

Speaker 5: because you have these custom channels, you can do things like, okay, whenever someone uploads a new avatar, in the save method, we put a message onto a different channel, and then that channel's consumer does thumb mailing. And that's a very brief explanation, and I'm sure none of you get what's going on here, so let's go a bit further. Um so what what what's basically happening is I'm I'm splitting Django apart. I've taken Django, which traditionally runs as a single instance process and making it three different layers. What I call an interface layer, which is the thing that talks to the outside world. So you have your normal WISGI interface, you have a WebSocket interface, you could have a HTTP2 interface in the future and other ones There is a channel layer, and the channel layer is responsible for routing between the two ones either side of it, and there is a worker layer. And what happens is

Speaker 5: rather than Django running as a single process that Takes a thing, does one thing at a time and returns the thing. The interface layers sort of handle all the negotiating negotiation with the outside world and put those messages onto the channel layer. And then the workers, much like celery workers do, for example, just take one one message at a time, run the appropriate consumer, and do the right stuff. So The kind of point here is that this is approaching some kind of asynchronous programming, right? I think asynchronour is great. It's really tricky. And so my goal is to give the people of Django a way to do the useful parts of async without having to know things about like race conditions and deadlocks and live locks and all the other stuff that

Speaker 5: even with good frameworks you're gonna run into some really quite nasty problems with this stuff. So channels is this project. I have documentation that's far more verbose like I could do in an hour on the stage here, even at my pace. So um if you want to go and read read the documentation it shows you an example and the idea of this project is it's meant to be a way to Prototype something that could be in Django itself in the future. So it's currently a third-party app, you install it, plug it in, it will just take over and do stuff. There's some worked examples in there as well. And then As a final thing, I've put up an example project called Fiath, which is kind of like a link aggregator Reddit-ish kind of thing, but with live comments and chat and stuff like that. That 's also up for you to use.

Speaker 5: There's a chat example you can go and play with now if you like. And the source code for that's up too, so you can see how that works. And that's that in a nutshell. Thank you very much.