Creating a Spotify-like personal streaming service - Emma Delescolle

Speaker 1: Hello and welcome to this uh three minute talk for Django. con Europe 2020. Today we will be talking about my experience trying to create a Spotify-like uh streaming service uh for personal use with Django EnView So first of all, who am I? My name is Emma. I am the co-founder of a small company in Belgium called Levit. And I'm the maintainer of the Aroschema Adapter as well as Ember Clive Fiddles and other libraries. So if you've known me and seen me at other conferences, you might be wondering, are you sure you want to talk about view?

Speaker 1: You usually talk about amber. But yeah. Uh I've been smitting talks uh about l about amber for a few years now and uh they always get rejected. So I thought, hey, let's try something else and let's try a few. But all that said, uh I will be at some time comparing View to Amber So first I had to get started and uh being a full stack developer I like to have my front end in the same repository as my back end So to do that, I wanted to have my frontend at the same level as any Django application that would be my project. And this is what we have on the screen here. To get to that point, all I had to do was to install a Vue

Speaker 1: CLI, which is the CLI tool for View. We will talk more about it later. And once that was done, I just had to type View Create Front, front being the name of my front-end application. And this is what I got. So on the front you see there are three directories. There's source, there's tests, and uh there is public. Public is for any uh resources like images and things like that. Um source is where the main uh code will be and test of course is for tests So the first issue uh I encountered was to uh try to have the two servers to talk nicely together because Django has its own development server, but Vue

Speaker 1: also has a development server with uh uh reload and other nice features. So to do that, you can see that you have a config file for view in which you can specify some information. That config file is used for view and it is later transformed to be used by Webpack as well. So the server is actually not directly from Vue. And you can see here that we are proxying every request to Django, every API request to Django, and also every WebSocket request to Django as well. So the other way around I'm used with uh Amber

Speaker 1: and the Ember Development Server whenever you make a modification it recompiles everything and writes files to disk. So The only thing that you have to do on the Django side is to create a template view that will just fetch the index. html that is created by Ember. This is not the case with Vue. Vue um creates all the files, all the temporary files in memory and serves those files directly from memory. This means that there is no way from ja for Django. to go fetch the file and display that as a template view. So an alternative to be um at ease in development is to uh just redirect every um have a casual

Speaker 1: URL in your Django URLs and redirect everything that is not known uh towards the view development server After that , I wanted some tools to be able to work with you and uh one of the major tools that uh I found was the browser extension. which is available for Firefox as well as Chrome. Um there is a similar tool for tool for Amber, Vact, Angular. So I knew something like that existed. You can go and get it from um the web store for Chrome or for Firefox and you can see here on the screen that uh it

Speaker 1: shows you every single component that is rendered in view. If you focus on a component in particular, you can have its properties, its data, everything like that. There are several tabs to look at what happens in different parts of you. The next thing that I found really, really useful was the cheat sheet. There is this PDF that's two pages long. And that has all the life cycles of all the components, how to pass properties, all the syntax you have to use and all that. That cheat sheet is available at the URL that you see on your screen. And it is honestly one of the most useful tools that you can have if you're either new to Vue

Speaker 1: or if you have not used Vue for a long time Or even if you use Vue along other front-end frameworks because it's easy to get confused between one and the other And finally, the next thing that is really useful is a CLI tool. So it's like manage. py for Django. You can use it to uh create new project to build your project to uh launch the development server you can add new tasks uh you can um also add some uh some blueprints to your project. So for example if you didn't select tests and you want to add tests you can add test on the fly to the project and things like that. So once again very, very useful.

Speaker 1: So before we go any further, let me ch give you a few basics of view for Someone who is coming from the Django world. So there are two ways to use Vue. You can either include the JavaScript of Vue in any regular Django server rendered view and it will load like any other uh any other JS, like Bootstrap for example. But in this case I wanted a single page app, so this means that my application, my front end application is gonna have different routes and for that I do have a file that's called uh rotor. js or router slash index. js And this is basically the same as the URL.

Speaker 1: py. It's going to uh match the URL and send it to the correct uh view uh component. Next we have app. view. App. view is very similar to your uh templates bash HTML. This is the shell of your application. This is where you're going to put your menus, your headers, your footers, everything that has to always be on the page. This is where it's gonna go. And finally you have components. Components, if you want to compare to something in Django, are like uh super uh inclusion tax. There are inclusion tags that are used everywhere. Every single page is usually a component, so it's uh

Speaker 1: top level. uh is going to be um included inside your app. pht ml So let's look a little bit deeper in those files. So as I mentioned, router. js or router slash index. js is the same thing. As far as JavaScript is uh concerned, it's a bit like if you have uh some module dot py uh or if you have some module some module and inside that directory you have uh dunder init dunder That's exactly the same technique. If you look at this file, you can see that the URLs are defined pretty similarly to Django. So there is a path.

Speaker 1: This uh to match the URL. There is a name which is optional, and uh there is uh a component which in Django would be uh associated to the view function. Uh that's going to be um actually uh rendering the content of your page. Then there is this app. view file. So, first of all, what is a. view file? That's something we don't know. It's a file extension that is specific to view Um and it's a bit messy in the way that it has HTML inside of a template tag, it's got JavaScript. inside of a script tag and it's got some CSS or SCSS

Speaker 1: inside of a style tag Um but we don't like that in in Django. In the Django world we we like to have things clean in different files. So one way to fix that is to keep using the the dot view application and have really really simple. View files that have three tags, one template that loads an HTML file one script tag that loads a JavaScript file and one uh style tag that loads either CSS or an SCSS file Here on the screen you can also see on the bottom left the HTML for this app. view And uh inside its

Speaker 1: HTML there are two important things. There is the first uh div that's the ID app. This is where view will know it has to render. It will replace the div with the ID. uh the ID app and inside of that you can see that you have um a router view. The router view is where uh The rest of your page that corresponds to the URL that you have been loading, the rest of your page is going to be rendered inside that tag. So those are the two most important tags in the View application. Um now there's a component. js. Um it comes with uh some

Speaker 1: some information so you have properties, properties are things that are passed to the component um usually from another component that a parent component that calls a child component and passes some uh variables. Uh once again same principle as uh templates uh an inclusion template type You have some data, the data is uh local to the component, and um you have some actions. You have some methods. Methods are being called from inside the component. They can modify the data of the component itself. And finally you have computed properties. Computed properties I used so that you don't have to do complex logic in code.

Speaker 1: So for example, here we have two computed properties that are going to be dependent on the uh z uh data and when the sp data uh changes those two computed properties will will also change And so if we look at the HTML for the template for that uh for for that component, uh you can see that uh we do use the data is pz. And we also use um some of the computed properties. So you can see here we are using columns. um disabled so this is an assignment to an HTML property and so

Speaker 1: uh this means that the disabled property is going to come from uh JavaScript We have an at click. At is for events. So when the button is clicked, it's going to be calling The method that is listed here, which is download, and then underneath we have some other assignments with the computed properties Uh so now that we have gone through the basics of Hue, uh we can look at something uh more interesting like How can we uh get data from Django to Hue? So in this case, in um single page application, there is no way to have uh Django

Speaker 1: pre-render the data somewhere in a JavaScript variable or something. So you have to call an API. And so as most uh Django projects, when you want to have an API, you are going to want to use Django Rest framework. So these are examples for a serializer in a view set for Django West framework. But since I mentioned I was the maintainer of the ARS Schema Adapter. I'm going to use uh the Reschema adapter and so here is an endpoint that does exactly the same as the two previous uh view set and serializer. plus a few extra features like searching and filtering. Um now on the view side

Speaker 1: um There are different ways to do things. The way I found what's the most uh familiar to me as a Django developer or an Amber developer was to use uh series of things. Uh first of all Axios. Axios is a library that does HTTP requests. It can be used with Vue, it can also be used with React, or it can even be used standalone. Axios just needs to be initialized. It takes a URL to uh start with and here we can see also the two last lines that are the configuration for the X uh SRF uh token and header

Speaker 1: Uh this is really useful if you're serving your front end from the same server as your backend and from the same domain. This means that you're going to be using uh Django's all of Django's security features. features including but not limited to uh CSURF tokens. So you have to uh specify those uh information in the connection and Axios is going to handle that uh completely transparently. Next to Axios, we are going to be using VuexORM. ViewX RM is an RM like Django's ORM uh the difference is that it's going to be uh storing some data inside uh your browser 's uh

Speaker 1: database and it's every time you add some data from the API or from somewhere else It's going to create basically a SQL database just in memory in your browser, which is really useful, for example, if you already have loaded a bunch of artists and you need uh to look up uh an artist in particular, you can just uh first check in memory if you have that artist before doing an API call to the back end You can do requests like you would do in SQL with sword statements and things like that. Um and uh here you can see how you um you register uh

Speaker 1: Vue XRM What you need to do is you need to have a model definition for all your models and then you simply register them to the database. The last thing we're going to be looking at is Vuex. Vuex is uh state management uh storage for uh View. It is used by Vuex RM since it's in the name. It's also the standards used with Vue. And here is the the configuration to to have it installed and as you see you you can register plugins and in this case the plugin that we want to register is the RM

Speaker 1: And finally, uh as far as the data exchange is co is uh concerned, we have the models This is the artist model and as you can see it is very similar to a Django model. So you what you do is that you list the properties, the fields of that model. Of course you don't have as many choices as you have in Django. You basically have uh string, number, date, uh Boolean, uh and undefined, which is the dot adder. But you do have interesting things like uh one too many relationships. So for example, if you look here you see that the artist has many songs Um and this is a relationship uh

Speaker 1: works to say moi as a Django relationship. Um or anything else, you can just access it with the dot notation. You can see at the beginning of the file that I'm inheriting from a base. js file. So this is a snippet from the base. js file. It's uh it's got some methods uh that are used everywhere like fetch all, fetch by ID, filter, things like that. Um Since this is something that you are going to be using with all your models, I like to have it in the base. js, but if you are using the Aeroschema adapter You can uh export all your uh endpoints, your back-end endpoints, directly to front-end

Speaker 1: models and it will write the best. js for you, and it will write uh your models also with every property and so on. So accessing the data in the front end, in the template As I mentioned, it works exactly like Django, you can use the dot notation. So for example, if you look into this one, we have a list And it's gonna loop over the artist. songs, so all the songs of the artist. And once we have the song, we can use uh the song dot um

Speaker 1: uh some dot file to get the the file for example and uh but you have to be careful In order to be able to use the dot notation, you have to uh warn the URM that you want to load the artist with the song. So if you look at the JS codes at the bottom of the screen, you will see that it is specifying that it loads with Song. It also has a word statement to filter the results. And if you don't use that whiz , it will not have access to the songs and you will run into trouble. Um this is a nice way to um avoid things like n plus one

Speaker 1: requests. Uh this is uh This can be seen either as a blessing or the or a curse, but it's how it works Next, uh if you want to write a single-page application nowadays, uh most likely you are going to be having uh some live data. So for example, uh in this application, which is a streaming service, you want to update the currently playing song, you want to update the list of coming up next songs and all that. So for that you need a web socket. For Django, on the Django site I picked uh Django channels and uh here's the the configuration for Django channels It's a very standard uh Django

Speaker 1: channels configuration. Uh note that uh we are using the authentication stack in here, which um is uh always a nice thing. And on the View side, there is a package that's called View Native Sockets, which is really handy. Um what you have to do is to declare the socket with some uh the URL is going to connect to uh some properties and then the socket is going to um be emitting some events and uh once an event is emitted it's going to uh call a transform um

Speaker 1: or a mutation. So uh here for example when the connection is open the mutation uh socket uh on open is called and it's going and you can perform things uh on open on message on things like that Mutations is a concept which is really important in view. It's about the same concept as is in React. This is how you change data. You cannot just simply take some data and change it in like in Django you have to go through mutation which is a concept that I'm not going to explain today because I don't have the time But if you are going to be using React or Vue, you really need to go in there and understand mutations.

Speaker 1: Next step is authentication. This is useful since I want to be able to create my playlist. I don't want to uh see my partner 's playlist. and things like that. So it's important that my application has some authentication. So authentication, there are basically two ways to do that for a front-end single page application. Either you can go with token, which is the preferred way to do things if your front-end application is served from another subdomain. then your main application. If you are using a CDN or something like that to serve your front end, you will want to use tokens. But since I this is a personal project And since I'm a full-side

Speaker 1: developer, I want I like to use um the session authentication because that way I can once again leverage all the security um tools that Django provides out of the box. So to have uh section authentication working uh on the Django side, you don't need much, you just need a login and a logout view That are going to be the logging view is going to take a username, a password, it's going to check it uh using the regular uh Django contrib. org tool and uh is going to return a user. Uh on the view side, uh a little bit more code is needed, but uh not that much.

Speaker 1: Uh you have We have here some actions that are defined in the store. So we have been seeing a lot of code that is in the store. So the store is what has to do with all the data. that view receives or view handles in one way or another. So here in the store we have uh two actions which are login and logout And upon a successful login we are calling immutation, which is uh old success, and that old success is going to um update the user that is stored in memory in um in view. So one uh once again this is a mutation this is really important to use mutations

Speaker 1: But that's that's the the gist of it. So uh login function, logout function, and then immutation to be able to update your user on the front end. And once your user is updated in the front end, you can uh use it uh in a lot of locations. One of the most uh useful locations is in the router, for example. So here you can see a route like uh the ones that we saw in the beginning, but this one has uh uh meta uh information. This is this which means require login. uh which is called require login. And so uh you can see at the bottom of the screen there is this code that is run before each uh transition

Speaker 1: from one uh route to to another route and it will check if that route needs authentication and if the route needs authentication will check if if the user is authenticated and if the user is not authenticated it it will redirect uh the user to the login page instead of showing them the page that they were that were they were asking to see So now we're going to be doing a demo. And um Here we have our application. It's got um a queue of uh things that are playing. Uh

Speaker 1: and this queue is uh always uh fed to a stream, so if I click play right now, uh I will be connecting to the stream Which is if if you recognize the song, it is the one that is said he's playing. So what Why did I want a stream? I wanted a stream so I can be listening to a playlist on my computer and then uh go to the living room and switch on my media player and uh retrieve the stream or where it was playing. Uh no, this is not this is of course not the only thing that uh

Speaker 1: this thing do. You can uh search for information. So I just searched for girl, I got an artist, I got several songs. With these songs I can just add them to the playlist. For example, if I go look at an artist, this one doesn't have any songs. But uh if I go look at Madonna, I see that I have two songs from Madonna. I also have her full biography, which uh comes from um Last FM Um since this is for personal use, something else I wanted uh was to be able to uh download things from um

Speaker 1: uh from YouTube so I can make a a search on YouTube and here is I I've got my uh YouTube search results. Once I have them I can download one of the songs This is going to take a minute or two and once the song is going to be downloaded , it's going to be available in my playlist and this is the demo effect this one didn't download let me try to download another one Maybe all my uh bandwidth is being taken by the stream uh from the from the conference. I'm not sure Let's see what happens here. Yes, the song has been downloaded and now I can just click play

Speaker 1: and here is my song play. So this is playing completely independently from the st uh from the from the stream. And if I want to go back to the stream, I can just uh reconnect to the stream and um and Get back to my uh regular stream. This um works completely independently and uh yeah here is the stream that's back up It is still playing the same song. And uh I guess that's about it for the demo.

Speaker 1: So now uh let's move to uh the conclusions. So uh the conclusion is that uh streaming is hard. Uh the major issue I got with this um uh with this application is that I thought I I thought I could just uh push every mp3 song to the stream and it would work uh correctly and uh it happens that it doesn't work correctly it just works correctly with one um player which is Cody because Cody does a resampling of the stream before playing it locally. Other stream players have uh difficulties with um um having the stream uh when changing from one mp tree to the next, changing bit

Speaker 1: rates or changing encoding or things like that. So the the stream finally is the is the worst part of it. Uh right now um it's still um It's still not working properly except on Cody, uh which is an issue. Uh but uh yeah. Uh unless I ask Django to do the same work as Cody and resample everything before uh streaming. Uh I don't see that there's a a good solution to that except re-encoding all my MP trees myself to make sure that they are of the same uh The same bitrate and everything. The next conclusion is that as you've seen, Vue

Speaker 1: is not that different from Django. If you are using the tools that I showed you, You can easily go from Django to View, yes, it's JavaScript, it's another language, but it really uses the same concepts as Django. And uh the another thing you have to be uh looking at, of course, is uh the data Don't forget that while we're using a VXRM you are going to be using local data and the local data is not always the same. as the data in the back end so you have to make sure to refresh that data often. And finally mutations. Mutation is a concept that

Speaker 1: is completely uh referring to Django developer, maybe you can try to think about it uh like um if you are changing information on a model in a view on Django uh it will not um it will not be in the database accessible to another view until you push the save you call the save method So that's a mutation. And I've run a little bit over time, so uh we can now go to uh the Q<unk>A And um that's it for the talk. Thank you for listening. Thank you to the organizers who did a really wonderful job and were really understanding of technical issues.

Speaker 1: Okay, so uh next one

Speaker 2: Uh yeah, uh I have some about the uh ORM. Uh like I never worked with that, so no I don't much idea how how this RM in the front end works. Uh but I suc suspect is it should be more like efficient or some way, uh uh instead of using like a store. I guess the view store is something similar with uh the store uh we use in React uh I mean it's like a huge uh data object with like all the the data you have uh in memory. So like do you have an any idea of like uh uh w a guideline to choose between like I I want

Speaker 2: us m this data of mine stored in this uh or M in the the front end or as uh uh like in the store objects uh Yeah.

Speaker 1: CRM D O RM is part of the store object. So if you're using DRM, you are going to be using the store. It's a double-edged sword. So uh if you start loading a bunch of data, if you load your entire back-end database in the front end Your browser is going to be slow, is going to be using a lot of memory, that's not going to be good. Although if you look at most Postgres databases, for example Uh a lot of them could uh actually be running in memory on most uh computers nowadays. Uh if you're using Chrome, anyway you're using uh

Speaker 1: four or eight gigs of RAM just just for Chrome. So it's it's it might not be that much of a problem because that store is going to be removed whenever you close the page. No, if you really uh don't want to uh use that all that memory, you can uh directly use Axios, which is the the part that does HTTP requests and you can just directly use Axios in your uh view component to load the data. not put it in the store and use it directly in your component. That way it's not going to be stored uh inside that store but this means that every time you will need that data you will have to do a back-end request which is uh going to uh

Speaker 1: be taking a lot more resources than uh just uh going back to the store if this is something that you already have loaded uh in a previous request

Speaker 2: Cool.

Speaker 1: Okay, so if that answers your question, uh I don't know who's next.

Speaker 3: Well I have a question.

Speaker 1: Yes.

Speaker 3: Yes, hello. I'm Jens from the Pacific Institute. We are also um uh going to implement um uh yeah. js uh together with Django um and but now we are going uh uh for GraphQL do you have any idea or what why did you choose Jungle Rest framework over GraphQL or do you do you have any tips on that or what should be different different

Speaker 1: I do not have some tips on GraphQL. My personal take on GraphQL is that I do not like it The reason I do not like it is that it's uh really easy for anybody to uh forge a request in GraphQL that will just uh completely kill your back-end server. Uh it is possible in GraphQL to ask from some data to to to force the backend to make some joints on the database. And you if if you craft uh if you look at the requests that go by in the browser and you craft uh something you can uh request the whole database at once with GraphQL

Speaker 1: This is there are mechanisms in GraphQL to uh prevent that from happening. Uh but uh in GraphQL if you're writing your API in GraphQL uh you do have to always have that information in mind, that security uh concerns you always have to have that in mind, uh which is something that you don't need to have in mind uh with uh Django Rest framework because you can the the user can only request whatever you say that they can request. So this is why I usually go for uh rest Django REST framework and not GraphQL. So I've not looked into uh the GraphQL options and I'm sure there are some perfectly nice libraries in View to use GraphQL, but I cannot help you with that

Speaker 3: Okay, thank you, but it's also good information. Well I I've also uh experience with Jungle REST framework. Okay, then we can think about it. Okay, thank you very much for this information.

Speaker 1: You're welcome. Um next person. Uh I see Adam is next on my screen.

Speaker 4: Yeah, I've I think uh everyone's questions look good.

Speaker 1: Okay Uh next on my screen is Zirina Verena, you are muted. Okay, so no question from the reina. Uh next is Telmo. Okay, no question from Thelma. Uh Volker?

Speaker 5: No, I didn't really have a question. I was interested actually in the thing that you were showing now, uh synchronization with the with the backend. But I think the yeah, the base model you're coming up now of uh answering that. So thank you

Speaker 1: Welcome.

Speaker 5: I'm sorry, uh this was uh for Volca. I I thought you said Wouter, which is my name.

Speaker 6: Oh don't worry.

Speaker 1: It's okay.

Speaker 6: Okay, may I? It's Folker. Um I was too slow to unmute my microphone. Um I have a question with regards to the VuexRM and the local forage plugin The local forage plugin allows to use the indexed db in the browser for offline storage. Do you have any experience with um storing offline data using this combination?

Speaker 1: Uh yes. Most of the time the data I want to store offline is the user information. So um as I said, when you log in you receive the info the the user information. So uh something I do is that I store the user information uh locally on the browser. So next time uh and offline so next time you use you you open that page uh I look into the information I say oh we already have a user so let me just try to uh Make a request to the backend and see if you're still authenticated. Uh since we're using session authentication. So it there is uh I have an endpoint that is uh called me

Speaker 1: And so what I do is that with this information I make an uh a call to the backend, uh to that endpoint, uh either I get a four or three response, which means that uh I'm not authenticated on the back end anymore. uh either I get a 200 response if I get a 200 response and the data uh from that response is different from the user that I have stored in the offline storage. I will log out the user And uh if the if the the data is the same I will uh keep the user on and so the that user uh can uh be uh Usually I've got a user model and uh I'm storing the the the user model from the Vue XRM uh

Speaker 1: data.

Speaker 6: Okay, that's quite nice. Um do you use um token authentication in uh any project that you do?

Speaker 1: I do use token authentication but not uh not in any view project uh currently.

Speaker 6: Oh sorry.

Speaker 1: No, it's it's okay. Uh I'm probably going to use one uh starting next week. Uh but But uh so far I've been using uh token authentication on other projects that were using Angular, React, and Ember, but I don't think I've ever done one with uh with you. It's pretty similar from what I've read. It's pretty similar to to any other uh front end authentication. Um uh framework so uh there are libraries uh for using odds for example uh or if you are using um the jungle rest framework uh Token, so the permanent token. Uh you can use that. I'm going to go back.

Speaker 1: Yes.

Speaker 6: Yes. I I do use token based authentication in a project.

Speaker 1: Yes.

Speaker 6: So if you are interested uh in an exchange um besides uh this uh web conference right now then uh you can c contact me on slack.

Speaker 1: Okay. Um

Speaker 6: because I'm using j Django Nox Because the base uh token authentication of Django Rest framework allows only for one token and that is not uh encrypted in the database. So we changed to Django REST mocks in this case because there can be multiple tokens and they are encrypted in the database.

Speaker 1: Okay, best thanks.

Speaker 6: Okay. Thank you very much.

Speaker 1: Thank you. And so