Maps with Django with Paolo Melchiorre

Speaker 1: So hello everyone. It's a really a pleasure to be here with you and see all these people on the stage. And in this talk, we'll see how to build the web maps using Django. one of my past hike in the mountains where I took this this picture you can see here. But if you're asking yourself what type of maps we can build with Django, let's see an example. This map shows mountains, peaks all around the world. We all use maps like that every day , web or mobile app. And I built this map with Django.

Speaker 1: And in this talk, we'll be together a map similar to this one , but first I present a bit myself. So I'm Paolo Macchiore. This is the link of my blog. Someone already knows me here in the DjangoCon US. And in fact, Bartek took this picture of me in San Diego during DjangoCon US 2022. I like the Django and Python community and I try to be more involved in the last series and I don't know working in the global community. I tried also to help the local Italian and community and I founded a community in Pescara, my hometown. And I really like hiking in the mountains around Pescara in the Abruzzo

Speaker 1: region where I live. So I decided to build a map where I can have all the mountain peaks I've reached in bus hikings. But in this talk I want to share one particular hike I did some years ago. I took this picture from the starting point of this hike. It was very early in the morning. I was in the Italian Upennines in the center of Italy and the sun has not yet risen. We can see the lights of the cities along the coast, and in the distance there is the Adriatic Sea, but it's covered. With the dark. The making of this map, this talk will be a bit like this

Speaker 1: hike. We start from an easy stretch and then we go up in attitude. So let's start from the basics. A web map can have a lot of features, different features, can be dynamic, static, you can interact with the map or uh only view it the map can use raster vector tiles to represent the surface. Usually the Data is stored in a special database and a web map use JavaScript library to show dynamically your the mapping your web page The definition on Wikipedia say that map mapping is the process of using the maps delivered but a geographic information system.

Speaker 1: But implementing a GIS from scratch is beyond the scope of this talk. Instead, as a GIS, we are going to use Django. Our preferred web framework, and in particular the module, the Geo Django module. A warning: in the next slide, I show you some codes formatted to fit the slide space, sometimes in look. peculiar but you can copy and paste and run it and it worked perfectly. So we can start from the requirements of our project To create a Django project, we need a stable and supported Python version, a virtual environment where we can install the LED separation of Django.

Speaker 1: As usually to start creating the project, I switch to my project directory and I use the start project Django command and the basic files our project will be created for us. After entering the new MyMap directory, we create a marker sub and with the startup command and again all the file will be created for us. Now we have to activate our marker application inserting its name in the list of the installed apps in the setting file. We create a template file in a template directory and we add all

Speaker 1: only the usual border plates a title in the heads. but an empty body content that we'll fill in a bit. To show our template we create a new URL file in the marker apple. We add our path and a template view which use our template file. And as a last the last step is to include the URL file in the one of the project. So if we will start now the uh the project and visit the map part we'll see only a blank page. As for the starting of my high, there is nothing to see. So we can go up something more challenging now.

Speaker 1: I took this picture after working half an hour in the dark. You can see the starting sunrise in the distance, and we are about to start the high-altitude path here. As for our hike, we are starting to see something in our map as well. In fact, we hold an empty map using the Lifet library. Leaflet is one of the most used JavaScript libraries for web maps. It's free software and it's desktop and mobile friendly. It's very light and it has a very good online documentation. To add leaflet in our map, we need to link in the head of our template its JavaScript and CSS

Speaker 1: module. And in addition, here Using the Django static template tag, we are uh also link our custom CSS file In the body of the template file, we add a div tag with a specific ID, and below it we link our custom JavaScript file. And the CSFI we had um only the basic rules that are useful to show a full screen map Instead, in the JavaScript file, we add the code to show an empty leaflet map. Using the variable we initialize our OpenStreetMap

Speaker 1: layer. We connect the newly defined layer to our map PD. And in the last statement, uh we set a map view to show the whole world We can now run our Django project using the run server command. If you see logs like that in our term in your terminal without errors, the map is ready. Okay, we just created an empty map with Django. The result is pretty much what you see now. It's an empty map without marker showing the world world. So we start seeing something just like in my hike. This is a picture that shows across the road at the end of the first

Speaker 1: part of my hike, just before a very challenging appeal stretch. The sun risen and the landscape is visible around. So after I Having visualized our map, we'll now start with more elaborate part, writing more code to create our markers and show them in the map. To do that, it's time to get to know and activate GeoDjango, the Django JS module. Django had a geographic functionality a few years ago. with specific fields, database backend , special queries, and also the admin integration. But since then, many new useful features have been added over here until the latest

Speaker 1: version And to activate it, we need to install before some requirements. The first one is GDAL. uh which has a mandatory requirement for geo Django it is um os geo library for rendering and writing raster and vector geospatial data formats it has a varia um uh A lot of common line for geographic data processing. To install hits uh on a Debian-based system, you can use the AppT package manager or in Docker. If your Docker image is based on the Python book warm image, we can install a smaller package which contains only the library itself without all the commands that are useful in the development stage.

Speaker 1: We can now activate GeoDjango by adding the contrib GDS module to the installed apps in our project settings. But to use GeoDjango we need also a compatible special database backend. In this first stage of this talk, we'll use Specialite, the special extension for Sebolite. Segulite is the default database backend, it it provides uh vector database functionality. And Specialite is uh It has a very simple architecture, which is a complete database in an ordinary file, which is very convenient to experiment. As before, to install

Speaker 1: specialized package on a Debian-based system, we can use AppT Package Manager and in Docker , based on the Python bookworm image, we can install it. in this way. And after that, we add to specialized as a database backend in the default engine in our project settings. Now that we have set everything we can define our marker model to store a location and a name of the point we can want to add in the map. The two fields are both mandatory. The location is a point field that we can use in everything from Dio Django.

Speaker 1: And we use the name to represent our model. We use the Django Adim interface to insert new marker in the map We define a marker admin class by inheriting the GIS model admin class which use the OpenStreetMap layers in its widget. Before creating a marker instance in the admin, we have to generate a new database migration with the make migration command. And then we can apply the migration to our database, in this case for the fifth time, using the migrate command.

Speaker 1: So to actually access the admin interface after starting the project locally, we need to create a new user with the create super user command. And we can run again the Django project using RAN server and visit at the end our admin section in the browser. So this is the admin generated page for creating a new marker. We have a text field to enter the name of the point we want to reach and a map widget. In this case, we have to manually navigate the map to select a point in the space. This example I selected the point to the final destination of my hike

Speaker 1: But I added also other points in the database to see something on the map We can now show in our map all the markers I added through the admin interface. To do that I created a map view with subclass the Django generic list view We are going to use the existing map template, the marker model, and a custom context object name. We convert the markers retrieved by the list view to the GeoJSO format in the context data of our map using the serialized specific for GeoJSON.

Speaker 1: Now in the marker URL file we must assign the mappad to the newly created map view. we can add uh the the user json data in a secure way in our map map page using the json script template tag in this way the will the value of the markers in the context dictionary will be something more like this GeoJSON created automatically from all the markers but in in this case uh for space purpose i it contains only one marker but or with all the information

Speaker 1: the properties and the geometry information We need now to edit our JavaScript file and store the GeoJSON in a variable. Starting from this variable, we build a layer for our map and we extract also the name of the single marker and finally we add the layer in our map by setting the widget to contain all the markers as you can see from the last instruction. This way when we open the map we see all the points Okay, we can run again our project and see the marker in our map. Okay, in this map we see

Speaker 1: This mark map we see the few markers I created through the admin. They are inside the HTML code for the page. Sorry. But uh the rendering is still fluid and fast. You can also see the pop-up in the marker of the peak and of this ike. Don't know what is going on. Okay, but if we had more markers and our map loading will be much slower. And also Leaflet will be have a harder time to rendering it. In this case we will need a more complex solution.

Speaker 1: This picture show a beautiful landscape that came to my sight at the end of a very challenging climb and the ice peaks begin to be seen. But in front of us there is still uh challenging passages. before to reaching the highest peak. So we then continue implementing the final version of our map. This is the compatibility table of all the open source geographic backends created from the Django documentation. As you can see, PostJS is the only geographic backend that supports all the feature, all the GeoJango feature.

Speaker 1: So let's use PostgreS as a backend engine. PostJDS is a Postgres extension and it's the best database. The can for Djojango, it internally integrates all we need to work with maps To use PostGIS we need Postgres and so Psycho PG. But uh the requirements of our project is increasing, so I created now a requirements file with a list of packages In addition to Django and Psycho PG, we are going to use also Django Filter, Django Rest framework, and its geographic add-on. We install all the requirements uh

Speaker 1: from the file using the Python packager installer module And after that we can modify the project database settings, adding Post. js as a in the engine and all the connection parameters for our Postgres database. uh which you can have locally in Docker or remotely. We have to apply again all the migration to the newly created database uh because we already created the hit and and it contains nothing In this part of the map we want to create a restful geographic endpoint to serve our markers.

Speaker 1: We need to use Django Rest framework and its geographic head-on directly on our code, so we need to have them in the installed apps in the settings. First we create a serializer for our market class inheriting from Geo feature model serializer. have to define an optional list of fields to be shown as additional properties, the geographic field and the model itself Now we define a read-only model view set for the marker. We set the bounding box filter and To filter our markers and the type of filter. We are going to return all the marker instances in our database

Speaker 1: and the serializer class use the marker serializer that we already created. In the marker application, we define the URL of our new endpoint using the Django Rest framework default router to create a new API path. And finally we add uh the URL file to our project in the URL file of the general URL file. So move on on to update our JavaScript file as a last point. As we no longer have the data preloaded in the page. We no longer have a way to position the map. So

Speaker 1: to show all the markers, we try to locate the user through the browser that will ask the user to share their position and we'll use uh the shared position to center the map otherwise we'll center it in an arbitrary point with a low zoom level And we created also a layer group in the map that we'll use in a bit. We now load the marker fetching uh the GeoJSON representation from our endpoint. which will return only the markers to of the specified displayed area that we can provide using the bound box string

Speaker 1: To render the markers, we load them asynchronously. We clear all the markers in the layer group when we're moving the map. Then we extract the properties we want to show in the pop-ups, in this case the name. But we invoke this flow only when the user stops moving them up. And this way we retrieve a minimum number of markers, but only when we need to render them. So the response is light and fast. So let's run again our project one last time with the run server command and finally see our markers in the map. This is our final map. This is only a screenshot, but it represents what we

Speaker 1: see in this specific area. Tried to create an animated GIF with a low quality version, but I can assure that the loading is very fluid because the API calls occur only when the movement on the map uh stops. So the traffic data traffic is the essential and the rendering leaflet is very fluid Okay, our smart project as our map project the high has to reach the final destination, but unfortunately the view is in the photo is a bit covered by clouds. But I can add the new markers now in my project with these peaks I reached So I walked 25

Speaker 1: uh 27 kilometers and reached almost 2800 meters above the sea level. It was one of the longest hike I ever made and I hope to be able to do a new one soon so I can uh tell you about it. What's next? There are also many other features that we can add in a hypothetical map. For example, marker customization pop-ups, marker filtering based on relational data, the clustering of the points in front end on back end. the Jacob using for uh finding a point based on the address and so on. Before saying goodbye I want to share with you some tips based on my experience working with maps and Django

Speaker 1: and is read the documentation in DJ in Django because it's full of information. Read also information from PostGIS website because it contains a lot information to low-level queries and try also to read the source code from GitHub because there is something that you can learn only from the source code and last tip is search for question on the GIS stex change but try to solve the question by yourself you will learn more than reading the answer Last things you can also use this slide presentation to learn more about web maps because it's released with Creative

Speaker 1: Commons and you can find it using this QR code because it's the address on my website where I share in a few hours the the slides and also other information on the talk These also are my connection on social. You can reach me through that or you can also stop me here at the conference to ask other questions about maps or other things also hiking in the mountain. So thank you very thank you everyone.

Speaker 2: What would be the next peak if you're building out a real-world website? Is are there is there an obvious next level of difficulty or complexity when working with maps in Django? I know that's a broad question, but

Speaker 1: yeah, um I I wrote two articles about this this topic and I'm writing the the third one. Um I'm dealing with uh uh in in in yeah querying the open street map database remotely to um have more information based only from the points and also I'm trying to uh find a fluid way to extract the information the um from in the local information from our picture because now there is also metadata that contains all this information you can extract the latitude, longitude, the altitude also, and I want to try to

Speaker 1: upload an image and all this information extracted automatically from it. So this is the instead of navigating manually or writing manually the information. Think again.