--- title: Let's Design A Simpler SocketIO date: 2021-10-16 description: >- SocketIO is packed with features. But do we really need all of them all the time? --- Listen, don't get me wrong. SocketIO is great. It provides tons of features, fantastic platform support, is widely deployed by a hugely diverse set of companies, and has been around long enough that it probably has most of the easily-encountered bugs ironed out. So why wouldn't you want to use it? Well, a couple of reasons occur to me. One, it's not exactly small. The unpacked library weighs in at just over 1MB, which isn't a lot if it's a core component of your application (e.g. if you're building a real time chat app) but is a bit much if you're only looking for a small subset of its features. Two, it's reasonably complex. Again, not insurmountably so, but complex enough that it probably isn't worth hacking together a basic SocketIO client in your REPL of choice if you just want to test something real quick. And on the server side, it's complex enough that you'll probably want to avoid rolling your own if possible. This becomes especially troublesome if you already have a working application and you just want to sprinkle in a little real-time interactivity, rather than building your whole application around that assumption. In my (admittedly limited) experience, the existing SocketIO integrations don't always play very nicely with the existing server frameworks, although if you stick to the most widely-used stuff you're probably fine. And honestly, it's just a lot of complexity to introduce if you just want a simple event stream. You could argue that you don't even need websockets for this - Server Sent Events are a thing, as are simple HTTP requests with a streaming response - but in this day and age, the set of solutions to the problem of "persistent communication between server and client" has pretty firmly coalesed around websockets. They're there, they're supported, there are lots of libraries - you might as well just go with the flow. ## Ok, so what are you saying? Basically, that we need something that's _like_ SocketIO but lighter-weight, and solves a more limited set of problems. Specifically, the problem I'm looking to solve is _event streams_ - given a web service and a client, how does the client detect that _things are happening_ in the background so that it can update itself accordingly? The use cases for this are pretty extensive. Most obviously, you can use it to implement a notifications system on pretty much any webapp that involves "users want to know when things happen," which is pretty broad. Maybe you're running an ecommerce shop and you want to notify your customers that the item they've had their eye on is back in stock. Or you've just opened up a new promotion and they should check it out. Maybe you're running a website that displays a stock ticker, and you need up-to-the-second data on stock prices. Or you've got a dashboard with some kind of real-time monitoring chart, whatever it's measuring, and you want to keep it up to date with a minimum of overhead. Pub/sub is a powerful concept, which is why people keep re-implementing it. Like I'm doing here. Get over it. ## But why can't I just use SocketIO for this, again? I mean, you _can._ But SocketIO does so much _more_ than simple pub/sub. Connection multiplexing, automatic reconnects, receipt acknowledgements, the list goes on. All of these features are great if, again, you are implementing an instant messenger. They even have a feature called "rooms," which mentions in its documentation that it "makes it easy to implement private messages," among other things, so it's pretty clear who their target is. And it's a great target! Lots of people need instant messaging. Every website in the world seems to pop up a bubble saying "Hi, I'm [friendly-sounding name]! Do you want to talk about giving us money?" 30 seconds after you hit their page. Everyone with a customer-service team has discovered or will soon discover that most issues are easier to resolve over text than over the phone, especially if your CS is outsourced to some foriegn country so your reps all have an accent. SocketIO exists for a reason, and it's a very good reason, and if that's what you need then go for it, knock yourself out. But if all you need is a simple event stream with pub/sub semantics, then keep reading, because that's what I want to talk about. ## Fine. Make your pitch, I'll listen. The protocol I'm imagining should solve three basic problems: * Authentication * Connection management (keepalive, automatic reconnects) * ...and the actual pub/sub itself, of course. Let's go through each of these in turn. ### Authentication The protocol purists might start to set up a bit of a racket here. Ignore those guys, they suck. Listen, every web-based protocol in the world should at least be _aware_ of the question of authentication. Maybe the awareness should stop at "that's being handled so I don't need to think about it," but at least that much is pretty necessary. I don't know exactly how much Web traffic is authenticated vs. unauthenticated (ask Cloudflare, they might) but according to some quick Googling an Akamai bigwig said in 2019 that 83% of the traffic they see is API traffic. I imagine that API traffic is overwhelmingly authenticated, and when you factor in the fact that a large part of the rest is social media, which is also going to be mostly-authenticated, I imagine you'll end up with somewhere between "a whole lot" and "the vast majority." So you need authentication, and websockets don't give it to you. Well, they leave it open, kinda - RFC 6455 says that the websocket opening handshake (which starts with an HTTP request) can include > Optionally, other header fields, such as those used to send cookies or request authentication to a server. But in practice, this still kinda sucks. It sucks because you can't have _one_ authentication method that's dead-simple and works for everybody. Either you're coming from the browser, in which case you're stuck with session cookies or URL params and that's it, or you're coming from some kind of scripting environment where you'd love to be able to just stick a bearer token in the `Authorization` header like everybody else does, but that's not how the browser does it so tough luck. The only solution that works easily with all clients is to put the auth in a URL param. So let's just do that. Unfortunately, that creates a new issue: we can't just use a plain bearer token any more, because now it's in the URL and URL's go all sorts of places - server logs, CDNs, browser address bars, etc. Probably the best thing to do here is to simply sign the URL [a la AWS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html#RESTAuthenticationQueryStringAuth). Fortunately, since we're only dealing with a very specific type of request, we don't need to bother with the authenticated headers business that AWS does. The browser has very limited capabilities when it comes to modifying the request, so we should probably stick to a signature that can be included directly in the URL as a couple of querystring params.