Packaging Django Apps for Distribution on PyPI by Laura Hampton

Yeah, I think that's a good thing. Hello, my name is Laura and I am a Python developer. I live in New York City. I previously worked with Ernest and Dusted and a bunch of other awesome people on the Piprei project, and now I work at Datadog. So, as developers, we may need to write the same or similar code over and over. But we don't want to reinvent the wheel. If you have a single piece feature or piece of functionality on your Django project that you might want to use on another site, you might consider making it into a package. This talk is for people who want to make Django packages to share on PyPI or with their coworkers, or who are

curious about what goes into packaging a Django app. This talk will cover best practices for creating and distributing Django apps. However, there are a couple topics that I'm not going to have time to cover extensively. I'm not going to have an extensive discussion of security or licensing or how to choose a license. Or open -I won't really talk about open sourcing your package beyond making the code available to users. And I'm not really going to discuss open source community building and cultivation. So what is a package? We're familiar with the import statement, and that packages are importable, that you can use import syntax with them. Most packages give us access to classes and functions that are not defined in our current Python file.

So using them is like choosing chocolates from a chocolate box. Package is also an overloaded term in the Python world. So packages are something that contain multiple files and may contain things like compiled C extensions or data files or things like Django templates. Modules are a single file of Python code, and I'm not really going to talk that much about them. So all packages are modules, but not all modules are packages And a distribution, which we'll talk about eventually, is a package that has a version a version and is ready for publication. So there are some differences between a Django package like requests and a Django package. Django apps are mostly Python codes, so they can be packaged like

Requests or like a pa another type of utility package like that. But instead of using bits and pieces of the of the your Django package Django app You can install the whole app in your Django project. So it's sort of like adding a melted chunk of chocolate to a recipe when you're making chocolate cake or chocolate sauce. An app is a self-contained part of a Django project that you might take off your initial Django project and reuse. So it's something useful like user registration or a blog app or a contact form that will apply to a bunch of different Django projects. These apps can be installed but by pip installing them and then adding them to installed apps and setting. py and set the appropriate um URLs in the URL conference settings and you're off to the races.

So a simple Django project might contain a handful of apps like the Django Girls blog, the or the Pulse app from the Django documentation. Whereas a complex project might have tens or even hundreds of installed apps. That's a lot of chocolate. Let me turn this off. Okay. So, what makes a good app? Django apps tend to follow the Unix philosophy: do one thing and do it well. So it for instance we should think about them like ls, which lists files in a directory and gives information about them, like their permissions or their types. It doesn't search files for you and it doesn't edit them. And so we should be able to explain what your app does in one or two brief sentences. My app

signs up new users, my app displays blog posts, or my app collects polls, votes in a poll and displays them The installation of your app should be minimally invasive, so don't go go doing things like substituting SQL Alchemy for the Django ORM. Make sure Make sure all the files relevant to your app are in the app directory. And don't set things in stone for your users. Supply some basic templates or a default form, form class, but let the user substitute another if they wish. Don't make assumptions also about where the code will live. Don't force people to do strange things to their Python path in order to make their app work. Your app work. Also, it's a good idea to use URL namespaces because other installed apps may have identical URLs to the ones that you're using.

So when you're setting up your Django application as a package, you can either choose to use a source directory or not. The source directory sits one level down from the main folder of your app. And the source directory does a couple good things in terms of the functionality of your app. It forces you to pip install your app in order for your tests to work properly, which makes sure that pip install works and works the way you expect it. You can also use, you should also think about shipping a skeleton Django project or some scripts that emulate it so that your tests will run on your user's machine And please also do have tests because tests will lead to better contributions from you and from other people.

You should also think carefully about the name for your project. It should be unique and not one used already on PyPI because They are unique identifiers for your project and they're also used in the PyPI URL for your project. A valid name consists on PyPI, consists only of ASCII letters and numbers, period underscore and hyphen And start and end with a letter or number. And a package that works with Django should have Django in the name, partly so that it's not taking up module namespace that could be used for other projects. And also shows that it's Django specific. Also, don't name your app so it conflicts with an existing installed one of Django's existing installed apps, like auth admin or messages.

Also, don't name your project after an obscenity or an offensive word or dirty joke or something that might sound like it. Because people may use and discuss your app at work and it's also a good idea a good idea to be respectful of your users So you version numbering is also worth consideration. They're used to differentiate one release from all other releases. Version numbers must be unique and versions must be numbered so they consistently increase. And it's a good idea to follow PEP 440 closely. So PEP440 allows for semantic versioning like a. b. c, or it allows for year and month type versioning, as long as it conforms to the regex that's in PEP 440.

It's important also to have documentation for your package if you expect other people to use it. Sphinx is standard for building documentation for Python projects It's also a good idea to upload your project to read the docs and include docs in your package. If you do something with your package that's weird or awesome or cool or hacky, please document it. Document your dependencies. If you include any custom forms or templates, document those. And I want to remind you that doc strings are not documentation. Auto-generated documentation is not documentation, and code is not documentation. Documentation is at minimum full sentences in a human language telling your users the following.

How to install your package, what versions of Django and Python it works with. what the dependencies are, even if they're automatically installed, all of your app's public API, all of its models, views, and forms, what they're for, what they do. And what the user is expected to do with them, and also how to install the app in their project, and how the user should change their settings and their config items to make your package work. You should also include where to find source code and report bugs and suggest features. Your package also needs a README. And this can double as the long description for your package on PyPI. It's the first thing that potential users and contributors will see about your project, and it's not a replacement for full documentation.

But you should be able to provide enough information so your user can find out what your app does and if it'll work for them. Provide links to your full documentation in your README. And also provide information on dependencies and how to get the app working. Tell your users what level of support to expect and whether the project is actively maintained or updated or whether it's like a toy project that you've put online because It's there and you don't plan to support it. The options for formatting the README or long description formats on PyPI include GitHub flavored markdown, common mark, restructured text, and plain text. You should also choose a license, and this tool will help you choose one. It's useful if you plan to distribute your project so people outside your company can use it, and if you want the wider public to use it, it needs a license

So head over to choose a license and pick one. Now I'm going to talk about setup tools. It contains setup a function with a lot of arguments. It is the build script to make package in a consist uh your package in a consistent way that can then be installed on other people's computers. And it provides metadata about your package to PyPI and end users. And I'm only going to cover a certain a very small segment of SetupTools keyword arguments. It's important to work with your users' versions of dependencies. Avoid and avoid conflicting with a user's pro what a user might already have installed in his project. It's a good idea to aim to allow all versions of Django with upstream

support, and it's best to specify version ranges of other dependencies that your app will work with. It's also a good idea to work with your users' versions of Python and to write for the versions of Python that Django will support. And this will include Python 2. 7 through 2020. Or until 2020. Classifiers allow tagging your app to make it easy to find. And you should including versions of Python that your package will work with, which license your package uses, which OS it works on, and which versions of Django your app will work with. Setup Tools provides a command line tool, pythonsetup. py upload, but

based depending on your configuration on your machine, it may not use HTTPS. So it's a good idea to use twine instead, and I will speak about more about twine in a minute. You should also consider including non-code files like your license, your README, your documentation. In your package, and you can do this by passing include package data equals true, which will include the files listed in your manifest. in And this is a small manifest that can get quite long. And there's a tool for checking them called Check Manifest. Where'd you go? There you are. Okay, so Check Manifest. And it checks the Files entered in your manifest.

in against the files you've checked into Git, which makes sure that you haven't forgotten like pieces of your documentation or your license or your README. It's a good idea to choose to use setup tools instead of diskutils. Dist Utils was created in 1998 and is in the process of being phased out in favor of setup tools. Setup Tools is a drop-in replacement for Dist Utils and allows you to declare dependencies on other packages. Setup Tools has consistent behavior across Python versions and is more frequently updated than Dist Utils. I also want to speak briefly now about the difference between requirements. txt and setup. py or setup tools. Setup has an argument called install requires that specify other packages that your app depends on and will

automatically install them when the app is installed and when the package is installed. On another person's computer. So setup tools is for specifying dependencies and metadata for a package. Requirements. text is a list of packages in their versions that's been generated by Pipfreeze. And it's for replicating the packages that are installed in someone else's virtual environment. Requirements. txt is basically arguments that are passed to pip install when you want to use virtual env as a development environment. So now it's time to upload your app to the cheese shop. So there are two file formats to upload your package in, a wheel or built distribution or a source distribution.

Pip will preferentially install wheels, but upload both a wheel and a source distribution to PyP. And setup tools can create both a wheel and a source distribution in one shot. The wheel is a special package for working oops. The wheel is a compressed file format that has the code for your distribution and ends in WHL. Wheels just have to be moved to the right location in the file system to be installed. And they unzip themselves when they're placed in that location. Wheels have a faster installation of pure Python and instile installing compiled C extensions doesn't require require a compiler on the target compute on the target computer.

So the wheel comes from the old name for PyPI, which was the cheese shop, which was named for the Monty Python sketch in which a man attempts to buy cheese from a cheese shop that has no cheese. And now we have hundreds of thousands of packages in the in PyPI, so now it's called the warehouse and it is full of cheese. So why build source distributions if the wheel can do all this awesome stuff? The source or archive distribution contains tests, docs, and code. So your users can download all of that in one compressed tar file people or tar um folder. thing. People may want to download your code so they can look at it or modify it.

They may want the docs and the code together, or they may want to build different for different architectures such as ARM. So in the world of Python packaging, there's also something called an egg. It's a zip file with metadata, which is a built distribution with Python bytecode. They've been largely superseded by the wheel format. Don't make eggs. Don't upload eggs. Eggs don't unzip themselves when they're installed, so you need special file loaders to get access to things inside, like translated pages. Django no longer ships with these loaders. And eggs also can't declare dependencies on other packages, and finally, Pip does not support them. Why are they called eggs? Because Pythons lay eggs.

So finally we're ready to create the distribution. Before creating your package, make sure you have the latest versions of setup tool and wheel, and then you can create your source distribution and built distribution all in one shot. So there you go. And you'll see here that the build distribution is what ends in WHL and the source distribution is the um tar file. So now we're ready to talk about twine, which is a tool for uploading files for PyPI. It's named for tying up packages with twine to include them in the warehouse. And it's a good idea to use twine instead of setup. py upload because it uploads over HTTPS by default. Twine

only acts as an uploader. If you you have to build your source and wheel distribution first using setup. Twine can upload any packaging format. And it's a good idea to use Twine because setup. py upload also builds and uploads the project in a single step, so you can't test your built-in source distributions. first. So this is how you upload with Twine. Note that I have passed a site that is not the of Official PyPI endpoint. It is the PyPI sandbox. Uses the same code as pypi. org. And it's a good idea to upload to the sandbox if you want to make a toy project to see how this process works. Also to see make sure that your description and your files look okay and that your package is the way you want

want it, because you don't want to have to bump your version number if you have a formatting issue or a typo in your README. Finally, it's a good idea to think about the future of your of your Of your package. Don't upload your package and forget about it. Upload code to GitHub or another code hosting site so people can have a place to file issues and make pull requests. Tag your releases on your code hosting site so people can see the code where you made your releases. Put your docs and read the docs if you haven't already Be prepared to respond to bug reports and pull requests. And also have an email address, GPG keys if you roll that way, where people can report security issues without having to post them publicly on a code hosting site

And make sure that it is not your personal email address unless you want to get emails about your package in perpetuity. It's also a good idea to regularly run tests against it installed with any new versions. Of Python or Django and fix any issues and keep it up to date. And finally, I would like to thank James Bennett, Katie McLachlan, Russell Keith McGee. And Phil James and Nick James for their help reviewing my talk and for giving technical help with it. All of the errors in it are entirely my own. And thank you very much for coming.