Lightning Talks

This video is from DjangoCon US 2014 in Portland, Oregon, USA.

Lightning Talks
1:00:02
Published September 24, 2014
475 views

By, Various Presenters

Help us caption & translate this video!

http://amara.org/v/FQm0/

Summary

The presenters show practical Django tools and patterns: Widji uses a heterogeneous tree and drag-and-drop editor to build extensible forms containing different field types and non-form content; Toga provides Python-native, system-native widgets across desktop and iOS; and a Django library exposes PostgreSQL pgcrypto encryption transparently, including encrypted-field lookups. Other lightning talks cover hierarchical multilingual Haystack/Solr facets, reusable Ansible roles and deployment practices, Hendrix for running Django with Twisted and deferred work, and a clean separation between Django REST Framework APIs and AngularJS single-page apps. The final talks argue for removing Django’s global settings dependency through Unsettings and introduce Form Factor, a Django-backed API intended to render the same data-collection forms as native mobile interfaces; the transcript ends during its implementation discussion.

Key takeaways

  • Widji’s heterogeneous tree model supports reorderable, extensible forms that can mix field models with images, videos, maps, and other content.
  • Toga aims to provide cross-platform, system-native GUI widgets from Python, with desktop and preliminary mobile support.
  • PostgreSQL pgcrypto can be integrated with Django models so encrypted values are transparently decrypted and queried in the database.
  • Ansible roles, variables, templates, and gathered server facts help make Django environments reproducible across development, staging, and production.
  • Hendrix combines Django with Twisted to support WebSockets and deferred or asynchronous work without relying on Celery for every background task.
  • The speakers recommend keeping AngularJS single-page applications separate from Django and using Django REST Framework as their API boundary.

Summarised automatically from the transcript.

Transcript

10,059 words · auto-generated Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:22

Speaker 1: I'm Gavin Wall. And we're here to talk to you about Form Builder or hey, look what you can do with Widji in the tree model.

0:30

Speaker 2: Uh we work at Fusion Box. Fusion Box is a Django web development agency in Denver.

0:35

Speaker 1: And at Fusion Box, one of our open source projects is Widji. Uh Wigi is a content editor that we made in response to some of the problems that our clients had, that we've been seeing with our clients. It's a drag and drop editor for tree structures. Um

0:54

Speaker 2: so a problem that we've encountered frequently is a client is asking us for the ability to dynamically create a form on their website um without any programming. So they want to specify what fields they want, they want to specify validation for those fields, and they want to control what happens when the form submits.

1:11

Speaker 1: So um we've actually built a bunch of these form builders for clients before, and every time we do it we would run into some problems. So like If you wanted to have different models for different types of fields, um it doesn't really work really well with just what Django gives you. Um if you want to have like an input and a drop-down Uh those have different fields, but you have to put all the data in one model. And then and it's kind of gross, um ordering. Uh you can do that with something like the inlines and Django Admin sortable, but Um it limits you to just having one model. And then on top of that, for example, one time we had a client that had a field that referenced an image and they needed the image to be inside of the form

1:57

Speaker 1: so that they could refer to the image. Um so it's it's kind of hard to have non-form content inside your form when you can only deal with one model. And on top of that, when you're done when you've made it, um you can't really package it up and put it on PyPI because It's not extensible. If anybody wants to add a field type, they have to edit your original field model. But that was before widgets.

2:23

Speaker 2: Widji's data model is a heterogeneous tree. What that means is it's a tree where the nodes can be different types. That's implemented as a tree-beard tree of nodes, each with a generic foreign key to their content. In addition to the data model, which also provides the UI for manipulating that tree, we built a drag-and-drop editor to edit the tree data.

2:48

Speaker 1: So how does Wuji help with this? Um let's look back. Here's a here's the reasons why it was kind of hard, and a good form builder would overcome these. ordering forms, mixing different types of fields, having non-form content, and being sensible. So if we look at ordering forms Django TreeBear gives us this and Widji gives us an editor for it. So it's already done. We don't have to think about it.

3:14

Speaker 2: The generic foreign key allows us to reference any type of content, so we're now able to use a different model for each of our fields. Um

3:22

Speaker 1: interspersing non-form content, just like Gavin said, we can use any model. So we can add images and videos and Google Maps and we just do it.

3:34

Speaker 2: Since the generic foreign key is able to reference any model, sometimes you want to limit that. So Widji provides a compatibility system that allows you to specify which tree relationships are possible. So if you want to add a new field type, you can write a model and then specify that it goes in a form, and now Wigi is able to add that field to a form.

3:54

Speaker 1: So we went ahead and built a built a form builder with widgets and you can see it right there. You've got different types of um form fields like choice field, form input, file upload, and you can just drag them and drop them, interspersed non-form content. It's got a form wrapper widget, and that form wrapper widget's got a cool method called getForm. Which returns a Django form class. So you don't have you everybody's familiar with Django form classes. You can just use that, but the user built that form instead of a programmer.

4:29

Speaker 2: And here's what it looks like on the front end. So the user has complete control over this form.

4:37

Speaker 1: Thanks guys. Links they should look at widg wid. g. There's a demo and it's on GitHub, it's open source. Uh it's on PyPI. Thanks, guys.

4:50

Speaker 2: Come talk to us if you have any questions

4:56

Speaker 3: Okay, so um at DjangoCon US last year, uh I launched the BWare project. Uh BWare is uh an umbrella project I've been running in my what I laughingly refer to as my spare time. in which I'm hoping to collect a bunch of useful graphical development tools for testing, coverage, debugging, the sort of things that we do in a day-to-day life as uh as Python developers. The cornerstone of BeWare at the moment is Cricut, a graphical tool for running your test suites, including Django test suites, and enable you to navigate the whole testing process. But last year, or since last year, I've been a little bit quiet. Well, what happened? I'm still passionate about graphical tools and having good tools for development to support development, and I've hit a few road bumps. Most notably, uh I was using TKINTER to develop these tools, and TKInter doesn't have a web widget or you know a web

5:43

Speaker 3: view widget, which is kind of a big thing to be missing in 2014. So I started looking around at my options. There are lots of options out there, but none of them really work out of the box really well with Python. QT works nicely, but it's huge, and the Python libraries have this really weird limbo-state licensing problem WX Windows is really annoying to install. GTK's support for OS X is Run X Windows. Kivi does cross-platform but it uses themes and I really hate themes. So this was very rapidly becoming a yak for me. I've got things that I want to do, but I I need a widget toolkit that meets my requirements. So about the start of this year I started shaving that yak and uh about a month ago I publicly announced Togo. Toga is a cross-platform, 100% system-native, Python-native

6:30

Speaker 3: widget toolkit that's installable using PipInstall Togo. What does all that mean? Well, uh it's cross-platform. I've got it running on OS X, I've got it running on uh on Linux, on Ubuntu, and I've got some preliminary support for Windows as well. It's 100% system native. It doesn't use a theme to pretend it's rendering system native widgets. It uses system native APIs to render system native widgets. So a button on OS X is an NS button. It looks like, behaves like, acts like an actual NS button. On Ubuntu, it's a GTK. button These are accessed either using the native Python APIs that exist or a C Type's binding to the system binaries. But because the system binaries are always there, you don't have to compile anything to get them. It's also Python native, Python first and Python native.

7:15

Speaker 3: Because I'm developing it in and for Python, it means the APIs feel Python native. So you can do things like have context uh context managers. And for example, if you've got a long-running uh graphic or a long -running action that is stimulated in response to a button press, most widget toolkits coming from a C heritage uh will tell you to spawn a thread or create a callable that's then in put into a timer method that you inject into an event loop. In toga, you just yield Python gives us the ability to yield and a long-running event handler can yield back to the main event loop. There's no timers or callbacks, uh callback threads required. It's a pure Python app, so there's no binaries to ship with it, nothing to compile, no compiler required. With one small caveat on Ubuntu and virtual environments, the installation process is pip install toga and off you run.

8:02

Speaker 3: So how mature is it? Well it's only had six months of spare-time development. Um but uh here is a simple Fahrenheit to Celsius calculator running under OS X. And here is the same user interface running on the same code running under Ubuntu I've done the most work on OS X, so the widget support there is at the point where I can mock up the full Cricut user interface using Togo. And the original source of my Yak here's a really dumb web browser written in 35 lines of code. It's got a little bit of documentation. The code for this example is part of that documentation. There's obviously a lot more to go, and there is a basic tutorial. Oh, and uh one more thing. Uh writing a new GUI toolkit is all fine and dandy, but it really is a bit pointless if it doesn't support all the hot platforms that everyone's using today. Uh and that includes mobile platforms. So I ported it.

8:48

Speaker 3: Here is the same Fahrenheit to Celsius calculator running off exactly the same source code, running natively on iOS. To be completely clear, this is a completely native iOS application running using native iOS widgets on an iPhone from Python source, uh using exactly the same Python source as the desktop application used. Unfortunately it doesn't work on Android yet, but I am working on it. I'd love to talk to anyone with Python on Android experience about what might be possible There's also a template project out there if you happen to want to develop a Python app and not use Toga or use some other project out there. So if you want details or anything else about Toga, go see pyb. org slash toger or jump on Twitter. I'm also potentially interested in doing a Kickstarter to fund some further development on this, with the intention of turning it into something you can actually use for serious professional enterprise development.

9:33

Speaker 3: If you or your company think you might be interested in helping to fund that development, paying for commercial support maybe or otherwise getting involved, please get in touch. Thanks very much. Uh

9:48

Speaker 4: I have the unenviable position of following Russell. It's very charismatic. Um So my name is Dan Watson. I'm going to be talking about a little library I wrote for interfacing with PG Crypto. PG Crypto is a Postgres extension that basically lets you encrypt data inside your database using AES or blowfish ciphers. So why would you want to do this? Um I mean the most common reason is you have some sensitive information that you want to encrypt in your database but still want to be able to search on in your ORM in Django. I also have a couple functions that might be useful if you need to deal with ASCII armor, which is a open PGP message format or padding functions for dealing with block ciphers. Uh the low-level API, which is basically a collection of Python functions for dealing with PG crypto.

10:38

Speaker 4: You can see if you're using PG crypto, the SQL. You just encrypt data using a key file or a key string and a cipher and that's what you get back. And basically doing the same thing in Python using PyCrypto and a couple utility functions in my library, you basically get the same data back. So you could also do the same thing with ASCII armor, which is basically a fancy name for again open PGP uh message format. Um You can take that same encrypted data and call armor on it and you get back this text string where you can actually store basically the binary data in a text field instead of a byte A or whatever you want to store it in. Um So using the pgcrypto. armor function, it's basically the same, you get the same message back.

11:28

Speaker 4: Uh I guess the real meat of this is doing it in Django. Um you might have a Django model For an employee, say you want to store social security number, secure um salary, date hired, and you don't want these things to be human readable and you don't want to say transmit your key over the network when calling into Postgres. So here's an example model with some example data. And you can see when you get back the model instance and uh try to access an encrypted field, it's decrypted kind of transparently. With Django 17, the custom lookups are awesome. If you haven't played with them, I highly recommend it. You can now query encrypted fields in the database as you would normal fields without having to pull them into Python first.

12:19

Speaker 4: So you can filter on social security number. or you can do date hired less than and salary is greater than and you get back um query sets. And you can see the SQL for that actually looks like All the decryption and conversion to ASCII armor is done on the database side, so you don't need to do Python processing for it. That's pretty much all I have. If you're interested in talking about it, find me later. Here's a couple links for um Padding information, OpenPGP, and PG Crypto. Thanks.

13:00

Speaker 5: My name is Dier Ucha. I am from the Son Paul Research Foundation and the lead developer of the Virtual Library over there. And here with my colleague Pier, which is. right on over there. And I have to talk about the moot level, root select, and moot lingual source that's good hastack So all the research foundation is like the NSF over here in the US, the National Science Foundation. The mission is to water research and the development of the state of Saint Paulo. In this case NSF is from the whole US and São Paulo is the foundation zone for the state of São Paulo, like the state of Oregon, like the state of São Paulo

13:48

Speaker 5: in Brazil, in the southeast part of the country. So here we have uh it's cut but it's okay. Here we have the search result of a search of uh for scholarships and research grants in our before library. The user is searching for Alzheimer disease and it has returned more than 200 results of scholarships and research grants And on the left we have the facet, and in the facet we have the boot level, so each boat one is a category, and inside of each one there is a like For instance, in that case the future's knowledge, and we have four items of agronomical

14:35

Speaker 5: science and so on, and each of these have one hierarchy. For instance, we can have on the left the health science, and inside of health science we have collective health, dentistry, medicine, nursing and so on. And this is the same search result, but in Portuguese everything is translated. Actually, we have in Portuguese and everything is translated to English. And we have the same hierarchy and this is the same idea, the user can select many of them, many of the facets at the same time which was a problem. And what I'm going to show you guys is the main changes you can do to your hack

15:22

Speaker 5: implementation in order to accomplish this kind of thing. So for which level of assets you can choose which one you want. If you want all of them, you can get a free flight that I'm showing you. guys and implemented but for level first we have like a facet A which is like the top of the category and inside of that we have B, C and D And for that, we have a map, my search app, and the search index is . py which is the common file for haystack And we are creating the main one is the area which is the multi-valuable field and in the schema XML solar declaration we are using the type descendant

16:07

Speaker 5: path Which is like for the hierarchy, usually in the solar documentation they explain this like this, but we are using in a different context. Which it is not a problem. We are using that slash uh in the prepared area uh to make the hierarchy And we have only the few area search for query, for filtering the user what he has selected. For ModeSelect, we are using the tag EX for exclusion insolar that they have also in their documentation. Probably they don't have these in Haystack because these only would

16:54

Speaker 5: only work for solar and not for the other ones that the Haystack works with. And for that we use the EX tag for declaring the facet. And when you're going to narrow it down, you use the tag, the tag W with the same name in this case EYS and then narrowing down on this case I have hard coded the search with Arias search is XP or Arias XP, and then you get the facet counts which would show exactly like the last part of the slide. you would get the seven results also the other counts for the other

17:42

Speaker 5: categories in that facet And for lingual you do out all these but using uh A suffix in each view, determine which one is Portuguese and which one is English. For us it's good because it's only English and Portuguese, but if you are for UN or something like that, things could be a mess. So if you have a better idea, just come to me and we can talk more about it. Thank you.

18:20

Speaker 6: Hi, I'm the I'm Samith. I work at Tivix, uh small uh Django consulting company based out of San Francisco Uh Jeff gave a really good talk I think uh uh on Tuesday uh on Anzalot, so I just wanted to sort of build on that and share some of my uh findings and best practices that I've been using. So some background, what's the need? You know, as a consulting company we get projects in, we want to get up and running very fast, as fast as possible anyway. Uh you know the project goes into production after months of development. Well the staging environment's out of sync. Well someone knows which packages to install, etc. etc. So you know that would be nice to not you know panic. Uh and then you know sometimes you have a code branch, you just want to set up an entirely separate environment where you can run a branch of uh your code base and show it to the CEO or the client or whoever you know it is.

19:05

Speaker 6: Uh and I know Heroku supports it really well, but you know if you're in AWS or Linode or Rackspace, um you know it's somewhat of a you know pain. So Y Ansible, we really like the philosophy behind it. It's agent less. So if you like fabric, you'll definitely, you know, or you might like it. Batteries included, you know, tons of modules So you don't have to scour GitHub to find the right module you need or may need, etc. So they have a certain list of approved ones you can write your own, obviously. Lots of releases, 7,000 followers on GitHub. You know, that's my unscientific way of finding out what's good or not Um Chef has 3,000 by comparison. The company behind Ansibles raised $6 million. So that, you know, it's only going to get better and better over time. Technically speaking, why would you want to use it? Configuration management. uh you know what my what packages my server needs, things like those, orchestration orchestration and deployment.

19:51

Speaker 6: You could potentially just ignore orchestration and deployment, just use it for configuration management. We're leaning towards you know using it for everything along with fabric and I'll show you examples The core concepts, there are lots of them. So I'll just kind of skip over. In a nutshell, the three one three are playbook, play, and tasks. At a very low level, a task is the sort of granular thing that you see in yellow. This is off from their video So some gotchas and best practices. You know, where do I put my Ansible code? So we've gone with uh you know the directory structure on the left, which you see you know high level there's a Vagrant file author 's license, Ansible client and Django project name. Client is basically a yeoman sort of Angular client, all front-end code. Django is the Your Django project, and then there's an Ansible directory. Underneath Ansible you'll see uh you know there's dev and and staging and production. Those are basically the inventory files. The big directory to note there is the roles

20:38

Speaker 6: Uh and that's the key. You don't want one giant Anzibel file. You want to basically follow this kind of structure and have things in roles. What's inside roles? Uh so yep, so that's the directory structure. The middle one is basically the role. So Django, Python, USG, Virtual Rem, these are very composable roles. So you could swap out USG for G Unicorn, not a problem. So just get the G Unicorn role. You can swap out Nginx for Apache, swap it up. Virtual England is its own role. Anything under Django will basically be very Django specific. Anything under virtual end will be virtual and specific. If you're using Flask, you can ignore the Django role, et cetera, et cetera. So you can basically build these these roles and within these roles, as you can see on the right, under the UVSG role, there are handlers, tasks, templates, and wires. I can't cover all of them, but the main one is tasks. That's where all your tasks go Uh the templates are basically very interesting.

21:24

Speaker 6: There's the USG in dot INI template. Instead of having different templates in your code base for staging production, etc. You can have one template. It's a ginger template as you can see, and then you know use variables So what are some of the best practices? User modules. You know, they have lots of modules. I've mentioned some of them here. Pip and Django Manage being the two ones. Uh what are the alternatives? You could use the shell and command module and then you know go raw, but then you will end up doing basically what you're doing with fabric, right? Uh so you might as well leverage you know the modules that they have. They're pretty flexible Here's the Django managed modules, for example. You can specify the app path, the virtual env, the command, and also you know the settings you want to use. And as you can see, I'm using variables here. So there's almost nothing here that's specific to dev staging production or any other environment. Use variables. They're group variables, they're environment variables, they have lots of variables.

22:12

Speaker 6: It'll save you a lot of time. So if you're actually changing your code or anything or you're adding another environment, all you have to do is add another file or two and then you're good to go. So as you can see I've defined where my virtual end was. It might be different from dev versus production. So you can accommodate for that. Use templates. Use templates and these are ginger templates uh and they get compiled and put in certain places on runtime. Uh use fax. This is a great uh you know Anzible uh thing it basically when it connects to the server it uh you know it gathers facts. Facts could be you know number of processor cores. So for example in your Nginx settings you don't want to hardcode the number of processors or listeners. It's just based on the server. So if you're upgrading your server these will automatically change

22:58

Speaker 6: You can still leverage fabric. And you know, so you know, for example, we want to have a fab bootstrap command where we bootstrap our local development system. So you could still use fabric, but underneath it's calling that big Ansible command which you never want to call by hand The same thing you can deploy by fabric. Again, a big Ansible command, but it's hidden in your fabric file. You still continue staying fab staging launch and it'll launch through fabric uh launched to production. Uh this GitHub repository there is really nice. Some other articles, there's analysis by Lyft , why they worked with Stall Stack sadly. And that is the end and that's where the presentation is. Thank you very much

23:36

Speaker 7: Alright, so um I'm gonna tell you about this thing called Hendrix. It's a framework for deploying Django apps that was developed by these people right here at this company called Relio. com, where it is currently being used in production. I'm here because they hired me to implement some asynchronous stuff and celery. And when I got to work That guy Justin over there basically like threw himself in front of me and was like, don't do celery, try this instead. So and I was like, are you crazy? But so I did, and now I'm converted and I think everybody should know about this thing. So Hendrix. It's basically Django

24:21

Speaker 7: plus Twisted. That's the idea with the name, or so I've been told. What is it? It is essentially um it goes in the pipeline where you normally use Unicorn or UWSC, however you say that. Um And it is a twisted uh wrapper for that process. So instead of using green green lits or um Whatever it is that UWISKI does, it uses twisted to handle that uh whole p thread pool, et cetera. If you don't know Twisted is from their web page, it is an event-driven networking engine written in Python and it's really amazing. The more I learn about it, the more impressed I am. So

25:06

Speaker 7: easier Django development and deployment with benefits. Why is it easier? Um a couple of little things like um when you're running the dev server, you don't have to do that if debug. Then serve my statics from here. Hendrix has like a nice little way of doing that. You also don't need simlinks for your st admin anymore, it just works now, which is kind of nice. And another bonus is that you get you can just make a self-signed certificate and run SSL right there in the Django dev server, which is very handy sometimes. Um in production because twisted has its own thread management, you don't need to do your like um

25:52

Speaker 7: unicorn how many workers, conf file, stuff like that. So we have a little upstart. config that you can just it's actually made for Ansible so it has like the template brackets and you just use it in Ansible pops it in the right place and uh you can just this is actually a typo right here it should be I think it's Hendrix Start. I don't remember. It's basically an Ubuntu service thing. There should be a sudo in there too. So um but the good stuff is really what Twisted brings to the table. By running a Django app in Twisted At DjangoCon, I've seen a lot of people with a lot of different ways for dealing with some of these problems that Hendricks really kind of has for me solved. Web sockets, deferred threads that normally like stuff that you don't want to run in the request cycle, you can just defer it, do it over here.

26:46

Speaker 7: Don't have to use Celery all the time. Asynchronous I. O. for doing well let me just go through I have some examples. WebSockets, real-time communication You can do chat clients or chat servers, you can do I did some games, like it's super fun. You can write your ORM right there in in a socket uh call. And you don't have to run node. Um which is kind of nice sometimes. Here's a little example of doing a deferred thread. Like this is a pattern I see constantly where you want to send an email. The right way to do that is usually to have Celery do it so that the user doesn't have to wait for it. But now you can just do defer it to a separate thread. Update, do something that takes 10 seconds in a separate thread, go back to the user instantly.

27:37

Speaker 7: Um yeah, so I still haven't installed Celery at Relio, strangely. Um This this use case is um where you have like you have to go hit some API, it takes a second or two. You can basically send them set them all up instantly, and then you're back to doing something else. Um it's a little callbacky, but it's something that you have to get used to once in a while. Uh and The basic functionality, which is serving your web app, it's a little bit scary doing something totally that nobody else seems to be doing, but so far so good. And uh one of the reasons that we wanted to show everyone is hopefully someone will find some problem with it before the wrong time for finding problems with things.

28:30

Speaker 7: In the future, it's still in development. We're kind of trying to solidify some conventions, get some tests going. Prove that it really is a viable way to do things because it really does seem too good to be true, and uh some little details are still being refined. We'll have the readme tidied up by end of day today, I promise. And um oh good, I'm almost done. Pip install Hendrix. And that's it? Thank you. Thanks a lot. Um

29:03

Speaker 8: So I'm also from Tivix, and uh at Tivix uh we've had a number of projects over the past uh year and a half where we've been using Django and Angular. js together to come up with uh single-page apps. And so I wanted to take a few minutes to just sort of uh share some of our insights and what our experience was in putting that all together. So uh just wanted to say thanks to Nina for her talks on Django Rest framework and uh Angular. js earlier this week. And uh We'll move on. If you want to follow along, uh it's public uh slides, you can go and visit them now. So, um AngularJS and Django work really, really well together. Really like it.

29:49

Speaker 8: But you really also need to keep their roles well defined and separated. We had started with a sort of mixed uh configuration where we had Django uh templates and pages actually driving the uh Angular JS implementation. And that was kind of messy and difficult to really organize and have working productively. We then progressed to a website where it primarily just consumed uh JSON P content from a TastyPie uh API. And that was better, but as I'm sure you're all well aware, JSON P has some rather significant limitations. So

30:34

Speaker 8: step in the right direction. But what we ultimately settled on was actually using Django REST Framework to uh provide all the content and interactions between the website and Django. So you need to give them both their space. They will work together well. But don't uh don't mix those up too much or you're going to end up with strangular JS, which is what we like to call it sometimes. So yeah, uh going forward, we're only using Django Rest framework and AngularJS in that configuration. They are completely separate. We just have the calls back and forth. We do not have Django defining any templates up front. The single page app is just a a collection of HTML files and assets that have been optimized.

31:23

Speaker 8: So how did we get there? We've been treating Angular. js as a platform. So think of it as like you're developing an iOS app. You don't go into developing a iOS app by saying, now how am I going to have Django generate my views and controllers and all that? No. Think of them being completely separate. As a result, you need to utilize separate tools from Django and Python. So for us, we've been using uh Grunt quite extensively as well as Yeoman. Furthermore, if you're going to be building an API for mobile apps or for your website, just do that all up front. When you're going to put together your single page app, don't put it off. Don't put off aspects of it.

32:09

Speaker 8: Make sure your API is all fleshed out. You've got validation, and everything can be done from there. You'll really uh reap the benefits later when you do go to mobile and it's all ready to go. So this is a quick diagram of our stack that we utilize to develop and deploy. Uh Django Rest framework. We use uh Nginx in combination with uh Angular JS. Yeoman, Bauer, uh, that all goes towards the end result, which is this nice and tidy set of assets. So during development, what we'll do is uh we'll use Yeoman to actually scaffold out the the project. This is nice as a good way to avoid strangular JS, which is to let yeoman

32:59

Speaker 8: Uh make opinions about how this should work up front. Defer to its directory structure, uh utilize the the grunt job that it has set up. And um Just once again avoid mixing Django and Angular. js. Just keep them separate. Uh we moved on to using uh GruntServe to actually try uh manage them in parallel and that got messy because we had cores and we also um had to write code that accommodated that. That wasn't great. So we're just serving it as static content on the same domain. And that works nicely. There are some caveats, but not a big deal. Deployment, once again, don't use Django to deploy it. Use Grunt and Gulp to actually build and deploy the assets.

33:46

Speaker 8: and uh utilize those tools as much as possible to crunch it down. So if you want to check out some code we've written, uh we're at github. com slash Tivix. We've written Django Restouth which integrates uh the uh authentication tools with Django in a RESTful interface as well as an AngularJS module that uh interfaces with all that nicely. So once again, there's the URL for the slides and thank you for great conference

34:16

Speaker 1: Five years after Eric Flowe's talk, why Django sucks, Django Still Sucks. To my way of thinking, it sucks primarily because it still thinks of itself as an Apache plugin. rather than a Python library. This is evidenced most clearly by the configure method of the lazy settings object. So what is the UnSettings project? It's an effort to change project-wide settings from a globally incorporated single entity, you know, Django. comp. settings to a set of individually injected objects configurable with a diverse and flexible variety of patterns. Here's a quick example. This is how you send mail with Django, right? So an exceedingly simple task that Django happens to handle beautifully. Here's the documented approach.

34:53

Speaker 8: Import sendmail and call it with its common sense dependencies injected Of course, when we try to do that, we get an error because we haven't supplied the backend as a quad to the function. Now in Python this is called type error, or maybe in some cases attribute error, but in this other language called Django, it's improperly configured This already betrays the underlying thinking on the matter because Django doesn't and can't know whether I've actually configured an email backend. It only knows whether I've supplied it to the function. Fortunately, Django is Pythonic enough to allow me to specify my own backend as a Quarg, so I do. You can see here that I also specify a file path. to avoid getting another improperly configured. This isn't documented, but it actually works in the sense that it allows me to instantiate and obtain the back-end object. However, even with a properly instantiated back-end object, watch what happens.

35:39

Speaker 8: This is the end of the road. I can't tell sendmail about default char set, and since I haven't run settings. configure, my logic is essentially being rejected at this point. I can go no further. Now I don't know about you, but I run into this problem constantly. It's especially burdensome for a project like Hendrix, which Jamin just talked about, because it Hendrix tries to import Django and feed it to its own APIs, which is a perfectly reasonable thing for a Python library to do Now it might be an apt metaphor to think of on settings as the third and final phase of Django's puberty. Prior to magic removal, Django was a child. Starting at about 1.

36:14

Speaker 6: 4, it experienced a sort of adolescence, and now it's truly coming of age. Right? The first step was the custom off -bot user model that shows that Django cares about its people. Then migrations shows that Django cares about its people's right to change our mind. Unsetting shows that the Django tribe does not treat its project as an Apache plugin, but as a feature-rich Python subculture. So what's the current state of unsettings?

36:35

Speaker 1: In addition to myself, the project has two other active contributors, both of whom are very talented and thoughtful Pythonistas, Skylar Duvine of WNYC and James Farrington at Slash Root. We've developed a decorator called uses settings. When applied to a function that uses the settings singleton, it allows injection of that setting as a quarg instead. Simple. Now how many of you have written parts of Django's internal APIs that use the settings singleton? A small number of us, right? Those of you that aren't raising your hands will experience no change of behavior whatsoever from the implementation of our decorator Now for those of us that are raising our hands, the Use of Settings Decorator is actually a relief of many of the pains of maintaining and testing APIs that require settings. Together, we crafted a pull request of 78 commits that indicate our vision for the implementation of this decorator. Of these, we think eight are ready for review today.

37:22

Speaker 1: If these eight are merged, you'll be able to call, send mail, just as I tried to do, without raising an error. Now, the reasons that I think that this is really important is of course it removes burdensome import logic, allowing you to send mail. Great. But it's also a signal to the Python community that says, we are you. We're a Python library.

37:41

Speaker 8: We're a Python project and we appreciate the norms of the larger Python tribe. More importantly than that, it's a signal to new Django contributors that says your work belongs here, and it won't be locked away behind our configuration drama. There's a lot of Python developers with incredible logic to contribute to our project. And if they know that contributing it to our project means that anytime they want to use it, they have to negotiate with the settings singleton, they might be less inclined to. And there's plenty of other projects where their logic can go. Now the there are a bunch of things that you all can do. I was on Elena's Django news podcast about a m at PyCon. Um give that a listen and you can learn more that obviously that I can tell you in five minutes. Look at our pull requests, see what you think. Work on unsettings at the sprints. I'll probably be working on it at least a little, although I'll probably also work on Hendrix.

38:26

Speaker 8: Think of Django

38:27

Speaker 6: as a community and a movement, not just a product, and see if that starts to subtly change your view on the matter. And lobby your core team. Not that long ago, this was uh what felt to me an awfully controversial idea, but I'm now slowly hearing members of the core team, I think, tell me that they're coming around on it and starting to see that this is a viable way forward. I'm hope really hopeful that it is, and I do intend to keep working on it. Numbers the first and second phases of Django's puberty, as I call it, are some of the finest work by Russell Keith McGee and Andrew Godwin that I've seen in the open source movement

39:00

Speaker 1: I'm Justin Holmes. My email address is Justin at Justinholmes. com. My GitHub is github. com slash J Miles. This presentation was made with a little thing called Wheelman, which is a coffee script. library for creating presentation through uh persuasion through the scroll wheel feel free to check that out as well. Thank you very much.

39:22

Speaker 7: Alright, so Form Factor. Form Factor is a mobile data collection platform. What actually is that? It's a Django-powered platform for easy collection and submission of data from any device. So think of a field worker who needs to on their daily uh test they have to submit a form, they go from house to house. Uh form factor would allow them to do that from So how is it built? You build a form with Django, you define what your form is, and then Using Rust framework through a set of APIs, different forms are made available on

40:09

Speaker 7: any different platform and these are the keyword there being native forms. So simply define a form in on your server with Django and then expose that and you'll have using our iOS, Android, and with future JavaScript library you'll see a native form render. Um so what the challenge and why did we do this, why not just simply create in HTML f HTML form and then simply uh render it in a web view on iOS and Android. Um and the reason for that is we wanted that kind of native field. The real uh The real problem is these people need to uh

40:55

Speaker 7: submit data, right? So it needs to be as seamless as possible. It can't it can't be kind of a janky experience which we've uh we've experienced on uh HTML5 and using PhoneGap and and whatnot. So um these are all native forms and we didn't want to reinvent the wheel. We're constantly building forms and as you guys know building forms are hard. You have to have custom validation for each field and it takes a lot of time. So we wanted something out of the box that you could simply uh it just define Build it once on the back end and then all the different clients will sing will seamlessly update. This is a bit of uh modeling. So as we were building this we thought

41:40

Speaker 7: why um Why call it forms? Because we wanted forms to refer to some other piece of data we have. So we thought a bit more generic and why not call it entity. A form is just a is just a representation of that entity. So I'll get to a demo in the end, but an entity is something you actually define. It's a a thing, right? And then uh there's different entity attribute fields and these each are different uh types. So You know, your text field, select field related , and your actual data is an instance. So these instances um related to entities and then instance field with generic foreign key to a entity

42:26

Speaker 7: attribute. So that was kind of the form factor platform I just discussed, but a lot of the core of what we used to build it will be open sourcing and that will be Django app with that allows you to create new entities and then REST framework APIs allow you to create and edit these in instances of these entities And then iOS and Android code base that uh that are hooked up with these uh with the API so you can simply just plug it in and start using it. And then uh as you can see we have kind of a internal debate going on what we're going to use for the JavaScript framework. And

43:11

Speaker 7: What you guys can use to use a simple contact form, right? If you had uh an application, Android application, iOS application, you could simply just uh pull in our libraries and um pip install uh this form factor library and you're pretty much good to go. Um so it's really that simple. You could build your own form factor or whatever else you want. So quick demo of it. So this is me actually defining a entity right here. So Right, so let's give

43:56

Speaker 7: person a first name. And all your validation you can actually define right here. And the mobile app will dynamically pick that up. So, all right, now the entity ID is 27 So this is the uh simulator running. So there's our dynamic form that was just picked up. Simply submit it. And and no alert, but it's there.

44:42

Speaker 7: Um and you can change orders and it will pick it up in the app as well. That's great. Hate to cut you off, but we're we're over time. Um I need uh Jacob Birch. Uh

45:01

Speaker 6: so about a decade or ago in the Lawrence Journal World offices in Lawrence, Kansas, which is really, really close to the middle of the country, and really close. to the middle of nowhere. Simon Wilson, Adrian Holivardi, and Jacob Kaplan Moss started what became Django And actually exactly almost nine years ago, a little over nine years ago, Adrian actually made uh the work they had been doing for the journal world. open source, therefore sort of giving the birth to Django. To celebrate this event, we who are in Lawrence, Kansas, are going to be throwing a birthday party. Uh what that entails, we don't know, and that's up to you, everyone in the room, and everyone in social media

45:48

Speaker 6: who can sign up right now on DjangoBirthday. com Let us know if you're interested in visiting Lawrence, seeing a little bit of history, and having some fun. If we get a small amount of people, we'll have a little party. We've got a lot of people. We'll throw a bigger party and probably have some talks along with it So there's the information. You can talk to Frank, you can talk to me, you can talk to Jeff, you can talk to Flavio. All of us are here if you uh have any ideas for it, and we hope to see you in Lawrence in about a year. That's it

46:25

Speaker 7: So Tmux and T-Masil. Um Tmux is a this is just a tool talk. Um Tmux is a terminal multiplexer. How many people have used Tmux? Oh, well, I don't even need to be up here. Um well for the other half. Um if you have, for those who haven't, if you've used GNU screen, it's kind of similar to that. Um and then I'll explain Tmasil after I show Tmux. So wow, it's so small. Um so this is just my I term In Tmux, it just allows me to create windows and panes. You can't actually even see the PowerShell at the bottom, but you can have lots of different windows, and each window can have panes.

47:13

Speaker 7: So I can separate the screen into two panes, I can separate it into three panes, etc. And I can also do this on the I can use it like you use new screen to connect to servers and you your session um your your your SSH session on the server is is maintained. So for example, I can connect to this is actually one of our production servers. Um and as soon as I As soon as I SSH into it, um, you know, whatever server I had or whatever session I had running before is still there. So I can I can go and I can look at whatever I was looking at earlier and kind of, oh yeah, that's what I was working on, that's where the problem was, right.

47:59

Speaker 7: And I use I just use iterm. Tabs of as you can see at the top to kind of have I use that for separate sessions only, only when I'm connected to separate servers. And everything else can exist in Tmux. And then TMACL allows you to just kind of um save tmux configuration. So if I want to work on a particular project I just say TMACIL and say I want to work on my UREC project. Then I hit say TMASIL UREC and it just opens up my you know my four-pane tiled layout layout. And you can just specify commands that get run in each pane. So, you know, git status, git branches, so I can remember what branches there were. a listing of of the of the directory, et cetera. You can you can specify um

48:46

Speaker 7: whatever you want to run. And if I Let's see, since I can't oh so over on the left here is just what the what a TMSIL um configuration file looks like. It's just YAML. And so you The important part is really the so root is kind of the root directory, so it'll um change directory into that directory and and every pane. And then you just have a list of panes. Oh, you can give it a layout. So if you have a complicated layout, um for example, I bought a $300 um 4K TV that I've been using as a monitor and in It's really crappy in every way except for the fact that it has 4,000 pixels. Um and so on that one I have like this six or eight pane

49:32

Speaker 7: layout and you just kind of you can just put that in here and it it's beautiful But as you can see, it's just really simple. I just tell it what commands to run in each pane, and it just does it. Thank you.

49:49

Speaker 2: Alrighty, so um

49:51

Speaker 1: Carson Bates is a responsive image handler, and we built it at PBS Kits. Uh so around January we released a new version of our homepage. Pretty fancy. Lots of pictures and everything. And it's responsive, so that's good. When you change the viewport, it takes uh you know pixel density and everything into consideration and readjusts the layout. So if you're on a mobile phone, it would really suck if you had to load all the source images, because that'd be several megabytes. Um so when your site's responsive, you gotta take some of this other stuff into consideration. uh pretty overwhelming stuff. So you know in the beginning we were like, all right.

50:38

Speaker 1: We can use some of these solutions, but the first three at least, you still have to stamp out those different sizes yourself. The front-end piece of it, the JavaScript, uh, and in the f in the case of the picture element, the browser can take care of building the request so that it asks for the right size image. But you still have to make them. So my coworker Miguel is like, that's stupid. It is. So we built Bates, which is the back end in Django, that does all of that automatically. So the other two solutions are actually pretty good. Adaptive images, it's built in PHP and it uses a thin layer front end that sets cookie hints. Um so that the served image um will be the right size for whatever user agent detection um the cookie supplies.

51:26

Speaker 1: And uh resource it is also very good, I suppose. I never used it. I think you have to pay for it. It's a platform as a service. Um so here are some of the guiding philosophies for when we built our thing. We wanted to keep the markup as semantic as possible. And we want to not really break anything that you might have on your page, like you know, jQuery and other frameworks and whatnot. Um and we want it to be pretty agnostic about what you might want to use as a backend. So if you don't like Django, don't use it. Use your own thing. That's cool. Um just use the piece of JavaScript to make the request.

51:59

Speaker 8: Or vice versa, if you want to uh write your own front end to you know do your user agent detection and whatnot, build those rules yourself, use the Django backend as your image breaker, cool. Um there's the GitHub project. You can check it out. I will lunge and give you a chance to copy it down. Alright. So here's the Django backend, you just pip install baits. Um and here's the admin. There's a couple of things you need to configure. Like obviously you'd have to tell it where to fetch your images from. So there's the notion of origin. And then your image sizes.

52:38

Speaker 6: So that's just a width, a quality, so you can compress it if you want to. Um and there's a name, short name So on the front end, the way you would use this is you would include

52:49

Speaker 8: Carson. js and in your markup, your image tags, instead of saying your source attribute, you're gonna set something called data Carson Source. Um and that's gonna that URL is built from a combination of the namespace, which is the origin that you just set in the Django admin, and the other thing you just set, which is the sorry, the uh and the the other part of this is the image path. So in other words, it's And your origin is where you specify the domain. In this case for PBS kits, it's a kind of delivery network. But after that What's the path that comes after? That's this part. So the other attribute Carson size is the image profile that you define in the admin. So, you know, script tag, include Carson, and there's a method called init in the Carson object.

53:34

Speaker 8: Um right now it's in the PPS. kits. uh namespace but we're probably gonna factor that out. Um you just initi initialize it whenever you want to. In this case it's on document ready, but whatever. Uh other some other features that are in here. There are two built-in sizes, uh so that mezzanine is the um source size, and none is just I don't want to serve this image at all because I want to save bandwidth. Um rules.

54:00

Speaker 6: So sometimes it's useful to determine the size of your image in data carcin size dynamically at runtime. So you can do that with a callback. It's just a function that you can supply to

54:11

Speaker 8: you know do that maybe based on Some user

54:14

Speaker 6: agent detection or um like um uh capability detection. Event hooks are nice, so when images load you want to um control how it's presented to the user. So maybe it's kind of faded in and whatnot. Cars

54:27

Speaker 8: and done is the same thing but for all of the images. So maybe you want them all to fit in, uh fade in at the same time when they're downloading. The trade-off is now we've gotten rid of browser prefetch. That optimization in your browser where it inspects source uh attributes and gets the images beforehand. We don't have that anymore, unfortunately. Roadmap, better documentation. We're working on it. There's uh GitHub pages. Uh picture support.

54:51

Speaker 1: I'm out of time. Picture support and ad hoc sizes. Uh contact me for questions. Thank you.

55:03

Speaker 2: Okay, so um tomorrow we have the spints and people are generally advised. Not to run before they can walk, which is a shame when other people are sprinting and you want to sprint too. So don't worry about what people say, because like other people, you're probably full of good ideas and have solutions to problems and want to collaborate and contribute to what we're doing. And we want you to. But too often people lack technical expertise or they lack the collaboration expertise or they simply lack the confidence to become involved. And these are barriers preventing participation. People want to commit, but they're not sure how. So the workshop that I'll be running tomorrow

55:48

Speaker 8: On the first day of the sprints, it will run all day, we'll cover

55:54

Speaker 7: the four key tools that you need to get started and the process itself.

56:02

Speaker 8: And generally I find that even people who are doing this for the very first time and they don't even need to be Python users

56:10

Speaker 2: will normally leave the workshop at the end of the day with a commit in Django's code base. It might be a very small one, but it will be a c uh that 's something you've got a very good chance of something getting in. I'll be there to help you. The other sprinters will be there to help you. Our colleagues um

56:29

Speaker 7: in the Django project will also be online helping you. So if you've ever wondered what these sprints were about or you've wanted to Commit to Django or another Python project. Come along to this. We'll work at the speed of the slowest person, which is usually myself. It's not rocket science because rockets are useless in this kind of uh endeavor. And it'll be fun and you don't need to worry about um being the slowest person in the room or anything like that. Just come along and have fun. If it's too slow for you, then you can just ignore it and I won't be in the least bit offended if you drop in or in or out.

57:08

Speaker 2: And you might be able to help people who are going a little bit more slowly than you. So it's all online anyway. You don't have to come to the Sprints to do it. There's a tutorial there. There's a a repository for it that you can help improve. You can come and find me on IRC or in real life. Drop me a line. Don't be afraid to commit.

57:29

Speaker 8: Thank you. I still have I I've got I had two minutes left so I'm going to use the rest of my allocation. Now forgive me if you've seen forgive me if you've seen this before. I know some of you have but I had one original idea in July In May 2013, I'm still waiting for another one. I'm going to milk this one for all that it's worth because we have to work with what we can get. So, relationships. There are seven billion other people in this world.

58:03

Speaker 2: Are you sure that you have chosen the right one for yourself? Because right now there is probably someone out there with whom you would have been happier than the imperfect person you are having an imperfect relationship with right now What's worse is that your chances of discovering the perfect person are almost nil.

58:28

Speaker 7: Simple arithmetic means that almost any choice you'll make is the wrong one. And also that you will fail in any attempt to make a better choice. So stop worrying about making the right choice. Instead, commit to what you have already chosen and develop it into the best possible relationship.

58:56

Speaker 2: for you. And the same goes for your relationship with the software that you work with. Because there are seven billion web frameworks and languages and platforms to choose from. And any one that you choose will probably be imperfect and the wrong one. So stop trying to make the right choice and instead commit to the project you have already chosen and help turn it into the best possible one for you. Thank you.

59:42

Speaker 2: Yeah.

Questions this talk answers

How can a Django form builder support different field types and non-form content?

Widji uses a heterogeneous tree with a generic foreign key, so each field can have its own model and forms can contain images, videos, maps, or other content. Its compatibility system limits which types of nodes can be placed in particular relationships.

Discussed at 2:28

How can I let users build Django forms without programming?

Widji provides a drag-and-drop editor for a heterogeneous tree of field and content models. A form wrapper exposes the result through `getForm()`, which returns a normal Django form class that can be used by the application.

Discussed at 3:14

What is Toga, and how does it create native cross-platform Python GUIs?

Toga is a Python-native, cross-platform widget toolkit that uses each operating system's actual native APIs rather than themed imitations. It runs on OS X, Linux, and Windows, and the same Python source can also produce a native iOS application.

Discussed at 6:30

How can I encrypt Django model fields while still querying them?

The speaker's library integrates PostgreSQL's `pgcrypto` with Django so encrypted fields are transparently decrypted when accessed. Django custom lookups let you filter on encrypted values, with encryption, decryption, and ASCII-armoring handled in the database rather than by Python.

Discussed at 11:48

How do I implement hierarchical, multilingual facets with Haystack and Solr?

The approach stores hierarchy paths in a multi-valued Solr field, uses Solr's `tag` and `exclude` parameters to calculate facet counts correctly, and narrows results with the selected path. Language-specific suffixes distinguish Portuguese and English fields and facets.

Discussed at 15:22

What is a good Ansible project structure for Django deployments?

Keep environment inventories separate and organize the implementation into composable roles rather than one large Ansible file. Roles can cover Django, Python, uWSGI or Gunicorn, virtual environments, and the web server, with tasks, handlers, and templates inside each role.

Discussed at 20:38

What Ansible practices make Django deployments easier to maintain?

Use Ansible's purpose-built modules instead of shell commands, put environment differences in variables, use Jinja templates, and use gathered facts for server-dependent settings. Fabric can remain as a friendly wrapper around the larger Ansible commands.

Discussed at 21:21

What is Hendrix, and why use it to deploy a Django application?

Hendrix is a Django-plus-Twisted framework that replaces the usual WSGI server layer with Twisted. It simplifies some development and deployment tasks and brings Twisted features such as WebSockets, deferred work, and asynchronous I/O without requiring Celery for every background operation.

Discussed at 23:41

How can a Django application run slow work without making the user wait?

With Hendrix and Twisted, work such as sending email or calling a slow external API can be deferred to another thread and the response can return immediately. The speaker presents this as an alternative to using Celery for some asynchronous tasks.

Discussed at 26:46

How should Django and AngularJS be used together in a single-page application?

Keep them separate: Django REST Framework should provide the API, while AngularJS owns the single-page application's templates and assets. Avoid having Django generate Angular views or mixing the two systems, which the speaker calls “strangular JS.”

Discussed at 30:34

What tools and deployment approach work well for Django and AngularJS single-page apps?

Treat AngularJS as its own application and use Yeoman, Grunt, and related frontend tooling to scaffold, optimize, and deploy its static assets. Serve the built frontend and Django API on the same domain when possible, rather than trying to deploy the frontend through Django.

Discussed at 31:23

What problem is the UnSettings project trying to solve in Django?

UnSettings aims to replace Django's globally configured settings singleton with settings that can be injected into individual functions and objects. This would let reusable Python code, such as mail-sending code, work without requiring project-wide settings initialization.

Discussed at 34:16

How does UnSettings make Django APIs easier to use and test?

Its `uses_settings` decorator allows functions that currently depend on the settings singleton to receive those settings as keyword arguments instead. The proposed changes remove configuration-related import problems while preserving existing behavior for code that does not need injection.

Discussed at 36:35

What is Form Factor for Django?

Form Factor is a Django-powered mobile data-collection platform: forms are defined on the server and exposed through Django REST Framework APIs. Native clients for iOS and Android can then render and submit those forms across devices.

Discussed at 39:22

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos from DjangoCon US