start writing kubernetes alternative post

src/routes/_posts/kubernetes-alternative-wishes.svx

@@ -30,12 +30,34 @@ So then why do you need an orchestration platform at all? Why not just use somet
 
 Okay, sure, maybe this is still a bit niche. But you know what? This is my blog, so I get to be unrealistic if I want to.
 
+## So what's different?
+
+Our hypothetical orchestrator system starts out in the same place as Kubernetes--you have a bunch of containerized applications that need to be run, and a pile of physical servers on which you'd like to run them. You want to be able to specify at a high level in what ways things should run, and how many of them, and so on. You don't want to worry about the fiddly details like deciding which container goes on which host, or manually moving all of `odin`'s containers to `thor` when the Roomba runs over `odin`'s power cable while you're on vacation on the other side of the country.
+
+So that much is the same. But we're going to do everything else differently.
+
+Where Kubernetes is intrusive, we want to be transparent. Where Kubernetes is flexible and pluggable, we will be opinionated. Where Kubernetes wants to proliferate statelessness and distributed-systems-ism, we will be perfectly content with stateful monotliths.<Sidenote>And smaller things, too. Microliths?</Sidenote> Where Kubernetes expects cattle, we will accept pets. And so on.
+
 ## The Goods
 
+### Docker-Image Based
+
+It's 2023 and the world has more or less decided on Docker<Sidenote>I know we're supposed to call them "OCI Images" now, but they'll always be Docker images to me. Docker started them, Docker popularized them, and then Docker died because it couldn't figure out how to monetize an infrastructure/tooling product. The least we can do is honor its memory by keeping the name alive.</Sidenote> images as the preferred format for packaging server applications. Are they efficient? Hell no. Are they annoying and fiddly, with plenty of [hidden footguns](https://danaepp.com/finding-api-secrets-in-hidden-layers-within-docker-containers)? You bet. But they _work_, and they've massively simplified the process of getting a server application up and running. As someone who has had to administer a Magento 2 installation, it's hard not to find that appealing.
+
+They're especially attractive to the self-hosting-ly inclined, because a well-maintained Docker image tends to keep _itself_ up to date with a bare minimum of automation. I know "automatic updates" are anathema to some, but remember, we're talking self-hosted stuff here--sure, the occasional upgrade may break your Gitea<Sidenote>Actually, probably not. I've been running Gitea for years now and never had a blip.</Sidenote> server, but I can almost guarantee that you'll spend less time fixing that than you would have manually applying every update to every app you ever wantedt to host, forever.
+
+So our hypothetical orchestrator is going to use Docker images. But there's a complication: It can't use Docker to run them, or even the lower-level components like `containerd` or `cri-o`, because it's going to be doing it all with...
+
 ### Firecracker
 
 You didn't write all these apps yourself, and you don't trust them any further than you can throw them. Containers are great and all, but you'd like a little more organization. Enter Firecracker. This does add some complexity where resource management is concerned, especially memory, since by default Firecracker wants you to allocate everything up front. But maybe that's ok, or maybe we can build in some [ballooning](https://github.com/firecracker-microvm/firecracker/blob/main/docs/ballooning.md) to keep things under control.
 
+Now, since we're running Docker images in Firecracker containers, we're going to need a method for converting Docker images _into_ Firecracker containers. Particularly we're going to need to convert a Docker image to a Firecracker rootfs, which is [definitely doable](https://fly.io/blog/docker-without-docker/) but not _completely_ trivial.
+
+### Networking
+
+Locked-down by default. You don't trust these apps, so they don't get access to the soft underbelly of your LAN. So it's principle-of-least-privilege all the way. Ideally it should be possible when specifying a new app that it gets network access to an existing app, rather than having to go back and modify the existing one.
+
 ### Storage
 
 Kubernetes tends to work best with stateless applications. It's not entirely devoid of [tools](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) for dealing with state, but state requires persistent storage and persistent storage is hard in clusters. I get the sense that for a long time you were almost completely on your own here, although recent options (Longhorn) are improving the situation.
@@ -45,3 +67,15 @@ Regardless, we're selfhosting here, which means virtually _everything_ has state
 * Asynchronous replication
 * Single-writer, multi-reader
 * Does this exist?
+
+### Configuration
+
+YAML, probably? It's fashionable to hate on YAML right now, but I've always found it rather pleasant.<Sidenote>Maybe people hate it because their primary experience of using it has been in Kubernetes manifests, which, fair enough.</Sidenote> JSON is out because no comments. TOML is out because nesting sucks. Weird niche supersets of JSON like HuJSON and JSON5 are out because they've been around long enough that if they were going to catch on, they would have by now. Docker Swarm config files<Sidenote>which are basically just Compose files with a few extra bits.</Sidenote> are my exemplar par excellence here. (comparison of Kubernetes and Swarm YAML?) (Of course they are, DX has always been Docker's Thing.)
+
+We are also _definitely_ going to eschew the Kubernetes model of exposing implementation details in the name of extensibility.<Sidenote>See: ReplicaSets, EndpointSlices. There's no reason for these to be first-class API resources like Deployments or Secrets, other than to enable extensibility. You never want users creating EndpointSlices manually, but you might (if you're Kubernetes) want to allow an "operator" service to fiddle with them, so you make them first-class resources because you have no concept of the distinction between external and internal APIs.</Sidenote>
+
+### Workload Grouping
+
+It's always struck me as odd that Kubernetes doesn't have a native concept for a heterogenous grouping of pods. Maybe it's because Kubernetes assumes it's being used to deploy mostly microservices, which are typically managed by independent teams--so workloads that are independent but in a provider/consumer relationship are being managed by different people, probably in different cluster namespaces anyway, so why bother trying to group them?
+
+Regardless, I think Nomad gets this exactly right with the job/group/task hierarchy. I'd like to just copy that wholesale, but with more network isolation.