Reduce, Reuse, Recycle - Persisting WebSocket connections with SharedWorkers

Speaker 1: Yeah, so this is this is kind of new. This is the first time we've been on a stage with like proper spotlights. So I can't actually see anybody. I'm not decided yet if I like this or if this is a good thing or a bad thing. But yeah, thank you for my introduction. So um I'm Aaron Bassett. My talk is on reduced reuse recycle. It's actually going to be about WebSockets and how we can persist and reuse WebSockets using local shared workers. So what that's gonna do with Django is we're gonna be using uh Django channels um to actually create our WebSocket server as well. So My name again Aaron Bassett. You can find me in most places as Aaron Bassett. I am that imaginative. That's my Twitter and my GitHub and everything else. Travel pictures, how idiotic Brexit is, usual kind of stuff.

Speaker 1: I will be tweeting later on today though about all the code and things that I've written for this talk. So if you would like to have a deeper dive into the code itself or to give the demos a try yourself, you'll find that all on my my Twitter uh later on this afternoon. So I am a developer advocate. Uh for anybody who's never heard the term developer advocate before, it's basically a recovering software engineer who now spends a lot of time writing and sleeping on aeroplanes. I work for a company called Nexmo. Check out this Nazi branding. We have been a sponsor here of DjangoCon um for For three years now, I think, um, which is basically the length of time our developer relations team has has been in existence. We do APIs for telephone.

Speaker 1: telephone and SMS and all kind of stuff. I'm not gonna go into that now. Um if you want to talk to us about it, we have the booth there there. Come speak to us But DjangoCon is is one of our favorite uh conferences. It's it's personally one of my favorites as well. It's actually the conference where I give my first ever conference talk. Um was at DjangoCon Europe back in 2015 in Cardiff. Yeah, woo, four years. And at that talk, I was actually um weirdly talking about WebSockets. as well. It was another talk about uh doing effortless real-time apps in Django. Now at that time, um in order to do WebSockets in Django to do real-time apps, it was a little bit more complex than it is today. So Slightly. So yeah, so that was a good kind of 45-minute talk on basically how you could mangle together Django and uh Tornado and Redis to to allow you to have WebSocket connections.

Speaker 1: And my first conference talk, I was quite proud of it. I was quite looking forward to becoming one of the speakers who tours the world, you know, giving the same talk over and over again. Um and then after my talk, I went to uh the one of the bricks. And a gentleman, one of the core developers, sat down beside me and was um we had a very interesting conversation on what they felt the the direction for Django should be with um async. That person was Andrew. Hey Andrew. And basically two weeks to the day after my talk, this email aligned on the Dango developers mailing list. So um if it's too small for people to read, basically this is um Andrew announcing channels um and completely making my entire talk obsolete. So nice I know technology has a shelf life and things move on and like if you're watching a conference video or reading a blog post, you know, a couple of years later you expect things to be slightly different, but two weeks

Speaker 1: But yeah, I can't be better. As you saw from the the graph I had at the start there, um things were very, very complicated before channels came along. Um I did an entire 45 minute talk. Basically just on how to get WebSockets working in Django. I think in this talk, um setting up the our uh channel consumers is going to be maybe like four slides. So um yeah, as as much as as it pained me to have to to stop giving that talk after its first starting, um it's uh we've we really have moved a long way with channels and uh Andrew deserves a a lot of um kudos for that. So I keep talking about WebSockets and what are WebSockets. So if we go back to the the prehistoric days of browsers, you know, all the way back to 1997.

Speaker 1: Back then if you if you had a client come to you and wanted to have any kind of real time in their application, say their their kids are constantly on this new ALL Instant Messenger thing, you know, they want to add chat to their website as well. You basically only had one tool available to you, and that was iframes. You know, 1997 iframes were in just in Internet Explorer, and that was the first time you could actually update a partial part of the page. Before that point, if you wanted to update anything in a web browser, you had to issue a new full request uh browser refresh or full request to the server and do the whole request response cycle again. With the introduction of iframes, we could now start to update just partial parts of the page. Or if you used a technique called forever freams, then essentially you got your server to send um a header which says that the transfer encoding is chunked.

Speaker 1: So chunked means that the server doesn't actually know how big the file is going to be. So it asks the browser to essentially continue downloading the file until the server tells it to stop. So whenever you've got a forever frame open like that, then your server can start pushing out script tags to it. So essentially what we have here is the size of the chunk. And then a script tag with some JavaScript in it. So your server could continuously push out these script tags, and then this the JavaScript get executed within the iframe on the site. Um and until you basically sent a chunk of a zero size, then that frame would just keep trying to load for as long as it could. So that allowed you to essentially push out um things we rendered in the browser

Speaker 1: in a kind of real-time fashion. Now it only worked because JavaScript, the browser had rendered JavaScript as soon as it received it, it wouldn't wait for the entire page to download before it would execute the JavaScript. And this became the basis of a technique known as Comet. Now, Comet went through a couple of variations. You know, it started off we're using iframes. Flash made an appearance for a while as well. You could do comment with Flash. There was comment using ActiveX controls or Java applets. Eventually it moved on to JavaScript. Where we're using the XML HTTP request object or Ajax, which was the really cool thing like five years ago. So essentially in long polling, you're opening up a Ajax connection to the server, and it's just keeping that connection open for I think this one's like 60 seconds.

Speaker 1: Now, at any point during that 60 seconds, if the browser has some new information to provide to the client, then it sends that information through, the on -success handler fires, and we start a new polling back to the server again. Or if the server doesn't have any information within before the timeout, the timeout's reached, then we we fire our on-failure handler and we start a new poll again. So it's just we're constantly polling the server for new information. Now there's a few problems with this. The first is it's it's a hack. You know, this is not the way this technology was supposed to be used, it's not how it was envisioned it was going to be used. So it turned out it was pretty brittle. You know, and there'd be different ways in which different browsers dealt with um certain timeouts, etc. And yeah, it w it was something that you could never really depend upon.

Speaker 1: And that's why a lot of the different comet frameworks would have multiple different ways of doing this. You know, ones might fall back to iframes or what might have uh Active S controls that fall back to Flash and et cetera, et cetera, et cetera. You know, we had to have all these different kind of failovers and different methods, and each one had its own limitations, and you ended up like having different shims in from and it was just a mess. The um second problem with it is the servers are not really designed to hold open connections for this length of time. You know, um web servers at the time were essentially designed that whenever a request comes through, that it would get the answer to that request as quick as it could and send it as a response. So you could have a single process that you know could be dealing with multiple people because although the process the actual requests are queued, each request is finishing quickly enough that you can just go on straight to the next request.

Speaker 1: And I keep answering them that way Whenever we got to long polling, we're holding the connection open for you know multiple seconds, a minute, maybe more, then you can very quickly start to use up your server resources. And the third issue uh actually is file size. So whenever you're making a HTTP request, there's a lot of information that gets passed around in headers. So this is a HTTP request to the uh DjangoCon Europe website. It's a very typical cell request. It's actually smaller than a lot of them because I don't have any cookie information, I don't believe, in that one. So cookies can obviously take up a lot of information as well and they're passed back and forward in each request. You've got all the things there about which languages I'm willing to accept, what type of uh character encodings I can accept, et cetera, et cetera. So with the response as well, response is reasonably large.

Speaker 1: You know, it's going to be telling things like um when the file was last updated and et cetera, et cetera, et cetera. So with these kind of things then um it might only kind of add up to a couple of hundred uh bytes. But when you compare it to a WebSocket request. And a WebSocket response, there's obviously a huge difference in size. So that difference in size, I said probably only a couple hundred bytes, doesn't suddenly A huge amount in the grand scheme of things. And it's not, if you only have like maybe one or two connections, then that's fine. You know, but as the number of people trying to like access your Um website increases, then obviously you're you're gonna uh that couple hundred bytes or five, six hundred bytes is gonna start to build up. In fact, if you have a hundred thousand users polling once every second With a

Speaker 1: header size of I think like 760 bytes or so. That just in headers information, just information you don't actually require, is 660 megabytes per second. Now, if you compare that to 100,000 users over WebSockets, again, pushing data every second, that is 1. 5 megabytes per second. You know, so that's a saving of only for only a hundred thousand users, that's a saving of over like six hundred megs per second. You know, it it becomes substantial. Um and I'm I'm looking over here to my uh Australian friends as I'm saying most of this stuff because obviously if you're dealing with uh countries that have data caps Pushing out any additional information that you don't need to send is I see really detrimental to your users as well as your your own um kind of bandwidth usage. One of the other benefits of WebSockets is they are bi-directional. So in all the examples I had there of long polling, we could only receive information from the server.

Speaker 1: We couldn't actually send information back to the server without starting another request. You know, so if it in the middle of our long polling session, if we suddenly the user and there's some input or we have something else that we want to send to the server, we need to start a new request for it. With WebSockets, it's bi-directional, we can send the data back and forward all the time. Also, WebSockets, you don't need to just transfer text back and forward. They can take binary information as well. So in this example, this is a uh a recording of a uh workshop that I've given a couple of times, a Python workshop. And it's actually taking the audio off a telephone call and sending that over a web socket. And we take that audio data, we send it to IBM, they do some magic with it, and they send us back, send them send them analysis of that audio. So the things getting graphed here is

Speaker 1: whether the person is happy, sad, fearful, disgusted, or joyful And it's all getting done in real time. And that's all getting pushed over WebSockets. So the audio data goes through to IBM over WebSocket, and the data that we send to the browser is going over another WebSocket. So they can be pretty useful for some situations you may not. It's not just like all chat applications and stock tickers or sort of things you can do with it as well. Okay, before we get into the actual code, there's a a few terms that we need to know, make things a lot easier to understand. WebSockets are event driven, so all of these are like events that we need to handle. So these events can occur pretty much at any time. It's not like a request-response where you have a request and you have a response, you know this an event can be triggered, and we need to have a handler for that. So the first one is on open. You know, this is whenever we create our WebSocket

Speaker 1: connection. We have an onMessage. Now because WebSockets are bi-directional, this message could be could be triggered on the server or on the client, depending upon which one is sending information. We have on error, which as it sounds like is if there's any kind of error in the connection. And we have on close. Again, the on-close is probably only really ever going to get fired. normally by the on the server whenever the client um kills a connection. But it is possible for it to be on the the client as well if the server terminates it for whatever reason. Normally you don't need to worry about like actually triggering the on-close event yourself. If you close a tab or the browser window, whatever, it'll trigger it for you. We have a couple of methods. We have our send method. This is how we send information back and forward along that bi-directional web

Speaker 1: socket. And then we have our close, which I mentioned we don't really need to worry about too much normally whenever we're dealing with it in browser contexts. They'll send the close event for you. Okay, so some code. So this is the channels code that I kind of promised you'd look at. Don't worry, I'm gonna blow it out on the side, so hopefully it's a bit easier to see. So with channels are very much this are very similar to class-based uh views in Django. We have a routings file which points a URL at uh um one of our consumers. And our consumers then are set up as classes. Here you can see we have an async WebSocket consumer class. But instead of we're on our class based views, we would have a get

Speaker 1: method to handle a get request. Here we have a connect method to handle a on connection event. So in our connect method, all this uh consumer is going to do, all our WebSocket server is going to do, is count how many clients are connected to it. And return that information to all the clients that are connected. So when we see the demo, we can see every time I open a new browser context, it's going to increase the c the number of connections. So to do that, we just store the connections in Redis. So I have a connections value here that I'm incrementing. I'm also storing the time of the last connection. And then whenever there is a new connection, I'm going to tell all of the connected clients that there's a new connection. So we can see this updated in our browsers, etc. Our disconnect handler, pretty much the same idea.

Speaker 1: You know, except in this instance, instead of incrementing that value, we're going to decrement that value. And then we're going to push that out again to everything that's connected. So every time there's a new client connects, we increment the value. Every time we get a disconnect event. We decrease the value and we update everything that's connected so we can see that happening in real time. So that's the the channel side of code. As I said Last time I gave this talk about WebSockets, it took like forty-five minutes to get to that point. Um so yeah, thank you so much, Andrew. It made it an awful lot easier. Um but yeah, so let's look at some of the browser side. So in the browser um we have our chat our WebSocket connection. So here we're creating a connection to the the WebSocket

Speaker 1: server And then we have this on message event. So essentially this is what's triggered anytime there is new data being sent from the server to the client. And in this case, all that data is going to be is a The current connection count, how many clients are connected to our server, as well as uh when the last connection was. We also have a on-close, so if there is for whatever reason a an a uh connection is is killed that we didn't mean to, then we'll be notified of that as well. And this is what the demo kind of looks like. So every time I click this add button, what it's going to do is it's going to add a new iframe to the page So we can see there is each new iframe is added to the page, that's a new browser context. That creates a new connection to our WebSocket server. Okay, as I remove iframes from the page, again it's going to reduce the number of connections to that server.

Speaker 1: So we're back in this like too many cat situation here. You know, every time somebody opens a new browser tab or um we've got a fream on the page, whatever else, we're opening another connection to the WebSocket server. This can become problematic if you're using a um likes of pusher, which is a service for uh basically offering WebSockets as a service, um, because they charge per connection. So if you have a customer who's opened up 10 tabs as a browsing your site, you're paying 10 times as much for that customer as you could be. If you're doing something where you're transferring lots of data, like the demo I had of doing uh audio over WebSockets And you've got a customer who's got four or five uh tabs open, then you're pushing out four or five times the amount of data that you need to be. So, shared web workers to the rescue. So shared web

Speaker 1: worker is essentially well, it's a web worker that's shared. Um what's a web worker? So web worker is a piece of JavaScript that can run outside of the main browser context. So normally workers would be used if you have something that's very CPU intensive, that you can hand off the computation of it to essentially a background task, a different thread, without locking up your main browser context. So your site is still responsive. A shared web worker is the same idea, except it's shared between any pages that are on the same origin. So same port, same host, same protocol. And this is essentially our shared web worker. So here we have an on-connect event. This is not the on-connect event for our web sockets. This is the on-connect event for our web

Speaker 1: worker. This is the bit that tripped me up the most when trying to get my head around these concepts. Is they share so many terms that are the same. So here we're our client is not connecting to a WebSocket. We don't have a WebSocket connection yet. It's connecting to the web worker The shared web worker. Whenever it connects, we have this event that's passed through. It has a port on it, and that's the port that we can use then to communicate with the browser context, with the different pages, the different tabs, different frames. These ports are very much like the WebSocket connection itself in that they're bi-directional. So we can send information from our shared worker to our page and from our page back to our shared worker. So we're going to store a list of all of the workers that are sorry, all of the pages that are connected to our worker. Then we're going to check to see if we currently have a WebSocket connection.

Speaker 1: If we don't have a WebSocket connection, then we create it. And this is the WebSocket connection, then it will be persisted and it will be used by all of the browser contacts that connect to our shared worker. Then finally we have this uh little bit. So this isn't a bit confusing again. So I'm gonna break it down like line by line. So here we have our shared web socket on message. So this is anytime we receive a message on our web socket. Not our web worker, but the web socket. So anytime some new data essentially is getting passed from the server to our browser as a whole, so all of our contexts. So anything that's coming from the browser to the server is going to be this onMessage event. Then we have our connections. So this is the connections object that we were adding all of our browser context to.

Speaker 1: So we're going to loop over each of these connections. And these connections are the browser contexts. that are connected to our shared web worker. They're not WebSocket connections, they are the browser contents connected to our web worker. As we iterate over each of those, Then we're going to use this send message to send the data that we've just received over the WebSocket server to each of the browser contexts. So essentially we're using the shared web worker as essentially a proxy to all of our browser contacts. So our server has a single point of contact. Which will then disseminate all the information to all the browser contexts. We only have one WebSocket connection, multi- as many browser contexts as we wish. So this is what it kind of um looks like altogether

Speaker 1: then. And this is what we change on the brow on the browser side. So before Instead of having our web socket connection, we now have our web worker connection. So essentially it's just the location of the JavaScript file that we want to execute as a web worker. And again we have this on message. Now this is not as I said they use so much of the similar language, it can be so easy to get tripped up. These messages are not messages that are coming from the WebSocket connection. They are messages that are coming from the shared web worker. So whenever we iterated over all those web worker connections that we saw in the previous or a couple of slides ago, and we use that push message. This is the event that it's going to end up triggering. So these is this is running in each of our browser contexts. So each of those iframes as you've got connection to that shared web worker.

Speaker 1: That shared web worker is then opening up communication channel, bi-directional communication channel. And it can then push messages from the worker back into the browser context. So we have the server pushing messages to a worker over a WebSocket connection. And then we have the worker pushing messages to all of our browser contexts through its port. And what it looks like is this. So now we can see as I'm clicking the add button, we have a connection. But as I add more and more iframes, the number of connections doesn't increase. It stays the same So each of these iframes are using that same shared worker. One thing to note, however, you can't really see in the demo, is that shared worker will only exist as long as there's at least one browser context connected to it The second that you close that

Speaker 1: that last browser context, that worker's killed as well. So you can't start a worker doing something in the background, close your laptop. Go to close your browser, go do something else, come back in a couple irres and open up the browser and reconnect to that worker again. That worker is gone. It's been killed as soon as you close your browser. So as long as you have but as long as you have one at least one connection open to the worker, um then that will you can persist it and you can connect multiple browser contexts to it. It also only works within the same browser. So in this one on the top is Chrome, on the bottom is Firefox. And I have Firefox opened connected to the same WebSocket connection. And as you see, whenever I start adding them then, it then jumps to two connections. But as I add more in Chrome, it doesn't increase beyond two. So it has one connection per browser.

Speaker 1: But you can have as many connections within the same browser and I'll still use that same shared worker. They're not shared between browsers. Speaking of browsers, browser support is okay. No support in in IE or Edge, which is So surprising. Um I've not actually had an opportunity yet to check it in um the latest uh version of in Explorer, the one that's based in Chromium. Um I'm assuming it will work in that because it works for Chrome. Um but does not work in IE, works in Chrome, works in Firefox. It did work in Safari and then they removed it. Which is unfortunate because fine's been read from Safari, it's also not supported in iOS, and it could be pretty useful in iOS.

Speaker 1: I wouldn't hold my breath about it ever getting support again in Safari. This is one of the Safari developers whenever they were asked about whether or not it would be coming back into Safari. But you can use object detection on it. So as you saw, I didn't need to make any changes to my server side code at all. My channels code remained exactly the same. For my client side code, the code was more or less the same. I just abstracted part of it away into the worker, but the actual on-message events remained exactly the same. So having While also supporting older browsers is not an issue at all. It just means that any of your users that have

Speaker 1: a browser that supports it, they're obviously going to be using um it's going to be more responsive or sorry, uh they'll have less latency and it's going to be using less resources for you. So it's kind of a win-win for both. So that's pretty much everything I have on it. I'm gonna stick this slide up again so people can follow me and stuff. Um I wanted to leave a few minutes for questions. I'm also going to be on the the the booth for the rest of the day. So if you do have any uh want to talk about it in any detail or you want to ask me anything about channels that I'll just point at Andrew and go talk to him. But yeah, uh anybody got any questions now?

Speaker 1: Oh you get spotlight as well. This is awesome.

Speaker 2: Okay. I'm going to ask two questions actually. First. You said for each browser it's opening a new connection. What about incognito mode?

Speaker 1: Oh good question. Um I've not actually tried that to be honest. So I'm assuming because it's in it's another browser context that um Whenever you're if this is my assumption, so I would need to verify this. I would assume that it will treat um inconvenient mode the same as a separate browser. So the browser contexts within Enconito mode can probably share one web worker, but it won't be able to share it with tabs that are outside of Enconito mode would be my Kind of my gut reaction to this. I'll need to double check though.

Speaker 2: It's it's okay. I just I was curious. And my second question is you said like okay, you close our uh laptop and the connection got closed as well. And then we open it back, but the web page is there. Probably the I don't know the iframe, the chat screen is there. Okay Where do we go from there? Do we display remote or

Speaker 1: no it can just it can just start up again. What what I meant was if you have um Say you're running a long computational process in a web worker. If you close your tab, that will kill that process. So you couldn't like start it doing some major number crunching. And then like close out your tabs, go do something else, come back later, revisit the page, and expect it to connect back to that same worker Whenever you close that tab, that worker is closed as well. So um with the shared workers obviously they'll will remain open as long as there's at least one tab or browser context open for it. But as soon as you close it, it's going to kill the worker. Whenever you come back, obviously it can start a new worker, it can start a new WebSocket connection, that's all fine, not a problem there. But it it won't be the same worker that you left.

Speaker 1: So any additional uh data that you've stored in the worker, any kind of like context that you've made available, et cetera. That'll all be gone as well. So you'll need to have some other kind of storage or it'll be the browser storage, et cetera, that you can populate that from.

Speaker 2: Thank you.

Speaker 1: No problem.

Speaker 3: Um so some companies have the tendency to block long running HTTP requests and by extension then WebSockets since you can proxy everything over it. What do you think would be possible fallbacks in that regard if you have to assume that the WebSocket will get killed like after five seconds or something?

Speaker 1: So it'd be uh it would be um disappointing if they were killing WebSocket connections, because they're not a long-running uh HTTP request, essentially. They're a new type of connection. and they're running over their own port. Um it should be easy to identify that something is a WebSocket connection rather than a regular HTTP. No, no port. Sorry. Discard that but it's easy to to To detect when something is a WebSocket connection rather than a regular HTTP connection. So in that instance, it's really down to whoever is configuring the actual rules for the networks. You know, they they should be aware that this is not a runaway connection. Um you know, it's not something that is timing out or um That is not as that is behaving irregularly. This is how this is designed to behave. And I would say that's actually a configuration problem with them rather than something that we should be trying to work around with WebSockets

Speaker 4: Hello there, enjoyed your talk. Um I'm curious about how this changes uh when after we're using HTTP 1. 1 right now, but HTTP 2 is coming down the line at all and If it's a long answer, I can I'm happy, but like how does it change when all the browsers start using HTTP2 or something like that?

Speaker 1: Um I don't have the answer to that. I'm sorry. It's not something I've looked into. But yeah, I'd be happy to talk about it later on.

Speaker 4: Cool.

Speaker 5: Hello. Do you think it would be possible to have a JavaScript library that would seamlessly implement it in a web worker and if not available, like in Safari, it would fall back to the web socket?

Speaker 1: Potentially what I think probably would be easier to do is to switch out whether you're using a shared web worker or just a regular web worker. So most of the browsers have good support for web workers. So really it would just be changing what your your import was, essentially.

Speaker 6: Thank you very much, Erin. Can we get another big round of applause?