From 8e58d6824a54967432977959223cfdab27f23c93 Mon Sep 17 00:00:00 2001 From: Joseph Montanaro Date: Sun, 7 Jul 2024 12:13:15 -0400 Subject: [PATCH] add ssh key format post --- src/lib/Post.svelte | 2 +- src/plugins/remark.js | 4 + src/routes/_posts/axes-of-fantasy.svx | 2 +- .../mixing-guis-and-clis-on-windows.svx | 2 +- src/routes/_posts/ssh-key-formats.svx | 85 +++++++++++++++++++ src/styles/main.scss | 7 ++ 6 files changed, 99 insertions(+), 3 deletions(-) create mode 100644 src/routes/_posts/ssh-key-formats.svx diff --git a/src/lib/Post.svelte b/src/lib/Post.svelte index 9998ee7..6f01f21 100644 --- a/src/lib/Post.svelte +++ b/src/lib/Post.svelte @@ -116,7 +116,7 @@
- {#if toc?.length !== 0} + {#if toc && toc.length !== 0} {/if}
diff --git a/src/plugins/remark.js b/src/plugins/remark.js index 1b119c2..8b35a97 100644 --- a/src/plugins/remark.js +++ b/src/plugins/remark.js @@ -6,6 +6,10 @@ import fs from 'node:fs'; // build table of contents and inject into frontmatter export function localRemark() { return (tree, vfile) => { + if (vfile.data.fm.toc === false) { + return; + } + let toc = []; let description = null; diff --git a/src/routes/_posts/axes-of-fantasy.svx b/src/routes/_posts/axes-of-fantasy.svx index f716440..39ce574 100644 --- a/src/routes/_posts/axes-of-fantasy.svx +++ b/src/routes/_posts/axes-of-fantasy.svx @@ -110,4 +110,4 @@ If anything, even rarer than the above. The only ones of which I am aware are th ### Middle Ages -The _vast_ majority of fantasy is set in something with approximately the aesthetics of Medieval Europe. Tolkien, obviously, was the trend-setter here,Although interestingly, Tolkien's work seems to clock in rather earlier than your Standard Formulaic Fantasy Setting 1A. Tolkien's world is closest to the _early_ middle ages (circa 1050 or so), from what I understand - e.g. his characters consistently use chain mail rather than plate armor; presumably plate armor doesn't exist in Middle Earth. The obvious reason for this, of course, is that this was the era Tolkien himself had studied most intensively - mostly its literature, from what I understand, but you can't become an expert in the literature of a period without developing at least _some_ sense of what life was like then. So naturally, this was what he drew on when crafting his fantasy settings. Later fantasy, on the other hand, seems to draw most heavily on the _late_ middle ages, circa 1300-1500, with plate armor, chivalry, the occasional joust, etc. but practically everyone +The _vast_ majority of fantasy is set in something with approximately the aesthetics of Medieval Europe. Tolkien, obviously, was the trend-setter here,Although interestingly, Tolkien's work seems to clock in rather earlier than your Standard Formulaic Fantasy Setting 1A. Tolkien's world is closest to the _early_ middle ages (circa 1050 or so), from what I understand - e.g. his characters consistently use chain mail rather than plate armor; presumably plate armor doesn't exist in Middle Earth. The obvious reason for this, of course, is that this was the era Tolkien himself had studied most intensively - mostly its literature, from what I understand, but you can't become an expert in the literature of a period without developing at least _some_ sense of what life was like then. So naturally, this was what he drew on when crafting his fantasy settings. Later fantasy, on the other hand, seems to draw most heavily on the _late_ middle ages, circa 1300-1500, with plate armor, chivalry, the occasional joust, etc. diff --git a/src/routes/_posts/mixing-guis-and-clis-on-windows.svx b/src/routes/_posts/mixing-guis-and-clis-on-windows.svx index 291be46..ee1a2a4 100644 --- a/src/routes/_posts/mixing-guis-and-clis-on-windows.svx +++ b/src/routes/_posts/mixing-guis-and-clis-on-windows.svx @@ -21,7 +21,7 @@ Alternatively, you can call `AttachConsole` with the PID of the parent process, Ok, so you do b) - flag your app as a CLI app, then call `FreeConsole` as soon as it launches to detach from the console that gets automatically assigned to it. Unfortunately this doesn't work either. When you launch a CLI app in a context that expects a GUI, such as the Start menu, it gets assigned a brand-new console window, again using whatever is the default terminal emulator. In my experience, it isn't consistently possible (from within the process at least) to call `FreeConsole` quickly enough to prevent this window from at least flashing briefly on the desktop. Livable? Sure, I guess, but it would be a sad world indeed if we never aimed higher than just _livable_. -Up until now, my solution has been to simply create two copies of my executable, one GUI and one CLI, put them in different directories, and add the directory of the CLI executable to my `PATH` so that it's the one that gets invoked when I run `mycommand` in a terminal. This works ok, despite being fairly inelegant, but just today I discovered a better way via [this rant](https://www.devever.net/~hl/win32con).With whose sentiments I must agree in every particular. Apparently you can specify the `CREATE_NO_WINDOW` flag when creating the process, which prevents it from creating a new window. Unfortunately, as that page notes, this requires you to control the invoation of the process, so the only way to make proper use of it is to create a "shim" executable that calls your main executable (which will be, in this case, the CLI-first version) with the `CREATE_NO_WINDOW` flag, for when you want to run in GUI mode. That post also points out that if you have a `.exe` file and a `.com` file alongside each other, Windows will prefer the `.com` file when the app is invoked via the CLI, so your shim can be `app.exe` and your main executable `app.com`. I haven't tried this yet myself, but it sounds like it would work, and it's a general enough solutionOne could even imagine a generalized "shim" executable which, when executed, simply looks at its current executable path, then searches in the same directory for another executable with the same name but the `.com` extension, and executes that with the `CREATE_NO_WINDOW` flag. that app frameworks such as [Tauri](https://tauri.app/)Building a Tauri app is how I encountered this problem in the first place, so I would be _quite_ happy if Tauri were to provide a might eventually handle it for you. +Up until now, my solution has been to simply create two copies of my executable, one GUI and one CLI, put them in different directories, and add the directory of the CLI executable to my `PATH` so that it's the one that gets invoked when I run `mycommand` in a terminal. This works ok, despite being fairly inelegant, but just today I discovered a better way via [this rant](https://www.devever.net/~hl/win32con).With whose sentiments I must agree in every particular. Apparently you can specify the `CREATE_NO_WINDOW` flag when creating the process, which prevents it from creating a new window. Unfortunately, as that page notes, this requires you to control the invocation of the process, so the only way to make proper use of it is to create a "shim" executable that calls your main executable (which will be, in this case, the CLI-first version) with the `CREATE_NO_WINDOW` flag, for when you want to run in GUI mode. That post also points out that if you have a `.exe` file and a `.com` file alongside each other, Windows will prefer the `.com` file when the app is invoked via the CLI, so your shim can be `app.exe` and your main executable `app.com`. I haven't tried this yet myself, but it sounds like it would work, and it's a general enough solutionOne could even imagine a generalized "shim" executable which, when executed, simply looks at its current executable path, then searches in the same directory for another executable with the same name but the `.com` extension, and executes that with the `CREATE_NO_WINDOW` flag. that app frameworks such as [Tauri](https://tauri.app/)Building a Tauri app is how I encountered this problem in the first place, so I would be _quite_ happy if Tauri were to provide a built-in solution. might eventually handle it for you. Another solution that was suggested to me recently (I think this is the more "old school" way of handling this problem) is to create a new virtual desktop, then ensure that the spurious console window gets created there so that it's out of sight. I haven't tried this myself, so I'm not familiar with the details, but my guess is that like the above it would require you to control the invocation of the process, so there isn't really any advantage over the other method, and it's still hacky as hell. diff --git a/src/routes/_posts/ssh-key-formats.svx b/src/routes/_posts/ssh-key-formats.svx new file mode 100644 index 0000000..6df8a97 --- /dev/null +++ b/src/routes/_posts/ssh-key-formats.svx @@ -0,0 +1,85 @@ +--- +title: 'Converting ssh keys from old formats' +date: 2024-07-06 +draft: true +--- + + + +Like a lot of people, my main experience with private keys has come from using them for SSH. I'm familiar with the theory, of course - I know generally what asymmetric encryption does,Although exactly _how_ it does so is still a complete mystery to me. I've looked up descriptions of RSA several times, and even tried to work my way through a toy example, but it's never helped. And I couldn't even _begin_ to explain elliptic curve cryptography beyond "black math magic". and I know that it means a compromised server can't reveal your private key, which is nice although if you only ever use a given private key to SSH into your server and the server is already compromised, is that really so helpful?Yes, yes, I know that it means you can use the same private key for _multiple_ things without having to worry, but in practice a lot of people seem to use separate private keys for separate things, and even though I'm not entirely sure why I feel uncomfortable doing otherwise. + +What I was less aware of, however, was the various ways in which private keys can be _stored_, which rather suddenly became a more-than-purely-academic concern to me this past week. I had an old private key lying around which had originally been generated by AWS, and used a rather old format,The oldest, I believe, that's in widespread use still. and I needed it to be comprehensible by newer software which loftily refused to have anything to do with such outdated ways of expressing itself.Who would write such obdurately high-handed software, you ask? Well, uh. Me, as it turns out. In my defense, though, I doubt it would have taken _less_ time to switch to a different SSH-key library than to figure out the particular magic incantation needed to get `ssh-keygen` to do it. No problem, thought I, I'll just use `ssh-keygen` to convert the old format to a newer format! Unfortunately this was frustratinglyAnd needlessly, it seems to me? difficult to figure out, so I'm writing it up here for posterity and so that I never have to look it up again.You know how it works. Once you've taken the time to really describe process in detail, you have it locked in and never have to refer back to your notes. + +## Preamble: Fantastic Formats and Where to Find Them + +If you're like me, you're probably aware that private keys are usually delivered as big blobs of Base64-encoded text prefaced by headers like `-----BEGIN OPENSSH PRIVATE KEY----`, and for some reason never use file extensions.Well, for the ones generated by `ssh-keygen`, at least. OpenSSL-generated ones often use `.key` or `.pem`, but those aren't typically used for SSH, so are less relevant here. There are three common formats you're likely to encounter in the wild: + +1. OpenSSH-formatted keys, which start with `BEGIN OPENSSH KEY`Plus the leading and trailing five dashes, but I'm tired of typing those out. and are the preferred way of formatting private keys for use with SSH, +2. PCKS#8-formatted keys, which start with `BEGIN PRIVATE KEY` or `BEGIN ENCRYPTED PRIVATE KEY`, and +3. PEM or PKCS#1 format, which starts with `BEGIN RSA KEY`. + +The oldest of these is PEM/PKCS#1 - the naming is a bit wishy-washy here. "PEM" when applied specifically to key files _for use with SSH_, i.e. the way that `ssh-keygen` uses it, seems to refer to this specific format. But "PEM" more generally is actually just a [generic container for binary data](https://en.wikipedia.org/w/index.php?title=Privacy-Enhanced_Mail&useskin=vector) that gets used a lot whenever it's helpful for binary data to be expressible as plaintext. In fact, _all_ of the private key formats I've ever seen used with OpenSSHPuTTY does its own thing, which is why it's a major pain in the neck to convert between them, especially from the OpenSSH side. Fortunately [this is much less of a problem than it used to be.](https://github.com/PowerShell/Win32-OpenSSH?tab=readme-ov-file) are some form of PEM file in the end. + +Whatever you call it, however, this format has the major limitation that it can only handle RSA keys, which is why it fell out of favor when elliptic-curve cryptography started becoming more popular. The successor seems to have been PKCS#8, which is pretty similar but can hold other types of private keys. I haven't researched this one in quite as much detail, but I'm guessing that it also is a little nicer in how it handles encrypting private keys, since when you encrypt a PKCS#1 key it gets a couple of extra headers at the top, like this: + +``` +-----BEGIN RSA PRIVATE KEY----- +Proc-Type: 4,ENCRYPTED +DEK-Info: AES-128-CBC,0B0A76ABB134DAFEB5C94C71760442EB + +tOSEoYVcYcVEXl6TfBRjFRSihE3660NGRu692gAOqdYayozIvU9xpfeVCSlYO... +``` + +whereas when you encrypt a PKCS#8 private key the header just changes from `BEGIN PRIVATE KEY` to `BEGIN ENCRYPTED PRIVATE KEY` before starting in on the Base64 bit. + +Both PKCS#1 and PKCS#8 use the same method for encoding the actual key data (which, when you get right down to it, is usually just a set of numbers with particular properties and relations between them): ASN.1, or Abstract Syntax Notation One. ASN.1 is... [complicated](https://en.wikipedia.org/wiki/ASN.1#Example). It seems very flexible, but it also seems like overkill unless you're trying to describe a very complex document, such as an X.509 certificate, or the legal code of the United States of America. But OpenSSH, it seems, longed for simpler days, when just reading a blasted private key didn't require pulling in a whole pile of parsing machinery, so it struck out on its own, and thereby was born the OpenSSH Private Key Format. This format does _not_ seem to be described in any RFCs, in fact the only detailed descriptions I can find are a [couple](https://coolaj86.com/articles/the-openssh-private-key-format/) of [blog posts](https://dnaeon.github.io/openssh-private-key-binary-format/) from people who figured it out for themselves from the OpenSSH source code.Presumably this format is much simpler to parse and allows OpenSSH to do away with all the cumbersome drugery of dealing with ASN.1... or would, except that OpenSSH is still perfectly happy to read PKCS#1 and #8 keys. Maybe the eventual plan is to stop supporting the older formats? It's been 10 years, according to [another random blog post](https://www.thedigitalcatonline.com/blog/2021/06/03/public-key-cryptography-openssh-private-keys/), but maybe give it another 10 and we'll see some change? I think this format can support every type of key that OpenSSH does, although I haven't personally confirmed that.This is coming across as sarcastic, but I actually don't blame OpenSSH for coming up with its own private key format. If I had to deal with parsing ASN.1 I'd probably be looking for ways out too. + +## The `ssh-keygen` Manpage is a Tapestry of Lies + +So, I thought, I can use `ssh-keygen` to convert between these various and sundry formats, right? It can do that, it _has_ to be able to do that, right? + +Well, yes. It _can_, but good luck figuring that out from the manpage. For starters, like many older CLI tools, `ssh-keygen` has an awful lot of flags and options, and doesn't clearly distinguish between which are _modifiers_ - "do the same thing, but differently" - and _modes of operation_ - "do a different thing entirely". The modern way to handle this distinction is with subcommands which take entirely different sets of arguments, but `ssh-keygen` dates back to a time before that was common. + +It also dates back to a time when manpages were the primary way of communicated detailed documentation for CLI tools,These days it seems more common to provide a reasonably-detailed `--help` output and then just link to web-based docs for more details. which you'd _think_ would make it possible to figure out how to convert from one private key format to another, but oh-ho-ho! Not so fast, my friend. Here, feast your eyes on this: + +``` +-i This option will read an unencrypted private (or public) key file in the format specified by the -m option and print an + OpenSSH compatible private (or public) key to stdout. This option allows importing keys from other software, including + several commercial SSH implementations. The default import format is “RFC4716”. +``` + +Sounds great, right? Import private keys from other formats, i.e. convert them to our format, right? But it's _lying_. The `-i` mode doesn't accept private keys _at all_, that I've been able to tell, whatever their format. Giving it one will first prompt you for the passphrase, if any (so it's lying about needing an unencrypted input, although that's not a big deal) and then tell you bluntly that the your file is not in a valid format. The specific error message varies slightly with the particular format - attempting to give it a PEM file (with the appropriate option) returns ` is not a recognized public key format`, PKCS8 gets `unrecognised raw private key format`, and speicfy OpenSSH format just says `parse key: invalid format`. So really, the only thing this mode is useful for is reading a _public_ key in some PEM-ish format, and spitting out the line that you can used in `authorized_keys` - the one that starts with `ssh-rsa`, `ssh-ed25519`, etc. + +## Enlightenment Ensues + +But wait! The `-i` option mentions that the formats accepted are specified by the `-m` option, so let's take a look there: + +``` +-m key_format + Specify a key format for key generation, the -i (import), -e (export) conversion options, and the -p change passphrase + operation. The latter may be used to convert between OpenSSH private key and PEM private key formats. The supported + key formats are: “RFC4716” (RFC 4716/SSH2 public or private key), “PKCS8” (PKCS8 public or private key) or “PEM” (PEM + public key). By default OpenSSH will write newly-generated private keys in its own format, but when converting public + keys for export the default format is “RFC4716”. Setting a format of “PEM” when generating or updating a supported pri‐ + vate key type will cause the key to be stored in the legacy PEM private key format. +``` + +Notice anything? I didn't, the first eleventy-seven times I read through this, because I was looking for a _list of known formats_, not a hint that you might be able to use _yet another option_ to do something _only marginally related to its core functionality_. So I missed that "The latter may be used to convert beteween OpenSSH private key and PEM private keyBy PEM here OpenSSH apparently means both PKCS#1 _and_ PKCS#8, since it works perfectly well for both. formats", and instead spent a while chasing my tail about RFC4716. This turns out to be not very helpful because it's _exclusively_ about a PEM-type encoding for _public_ keys, and doesn't mention private keys at all! OpenSSH seems to internally consider RFC4716 as the public-key counterpart to its home-grown private-key format, but this isn't explicitly laid out anywhere. Describing it as "RFC 4716/SSH2 public or private key" is confusing at best, because as we've established RFC 4716 doesn't mention private keys at all. "SSH2 private key" isn't obviously a thing: RFC 4716 public keys have the header `BEGIN SSH2 PUBLIC KEY`, but OpenSSH-formate private keys don't say anything about SSH2. The only way to figure out how it's being interpreted is to note that whenever `ssh-keygen` accepts the `-m` option _and_ happens to condescend to operate on private keys instead of public keys, giving it `-m RFC4716` produces and consumes private keys of the OpenSSH flavor. + +_Anyway_. The documentation is so obtuse here that I didn't even discover this from the manpage at all, in the end. I had to get it from some random Github gist that I unfortunately can't find any more, but was probably written by someone just as frustrated as I am over the ridiculousness of this whole process. The _only_ way to change the format of a private key is to tell `ssh-keygen` that you want to _change its passphrase_? It's [git checkout](https://stevelosh.com/blog/2013/04/git-koans/#s2-one-thing-well) all over again. + +At first I wondered whether maybe this is intentional, for security reasons, so that you don't accidentally remove the password from a private key while changing its format. But finally I don't think that makes sense, since if it's encrypted to start with then `ssh-keygen` is going to need its passphrase before it can make the conversion anyway, in which case there's no reason it can't just keep it hanging around and re-encrypt with the same passphrase after converting. + +Most probably this is just a case of a tool evolving organically over time rather than being intentionally designed from the top down, and sure, that's understandable. Nobody sets out to create a tool that lies to you on its manpageMaybe the `-i` option _did_ work with private keys at some point, although I have difficulty imagining why that functionality might have been removed. and re-purposes modes of operation for other, only marginally-related operations, it just happens gradually over time because updates and new features are made in isolation, without consideration of the whole. + +## Imagining a Brighter Tomorrow + +But it doesn't have to be this way! Nothing (that I can see) prevents the `-i` option from being updated to accept private keys as well as public keys: it's clearly perfectly capable of telling when a file _isn't_ a valid _public_ key in the specified format, so it it seems like it could just parse it as a private key instead, and keep going if successful. Or an entirely new option could be added for converting private keys. `-c` is already taken for changing comments, but there are a few letters remaining. I don't see a `-j` on the manpage, for instance, or an `-x`. + +I realize that an unforgiving reading of my travails in this endeavour might yield the conclusion that I'm an idiot with no reading comprehension, and that the manpage _clearly_ stated the solution to my problem _all along_, and if I had just RTFMNoob. then I could have avoided all this frustration, but that seems [a little unfair](https://xkcd.com/293/) to me.Besides, I'm annoyed, and it's more satisfying to blame others than admit any fault of my own. When you're writing the help message or manpage for your tool, you should _expect_ that people will be skimming it, looking to pick out the tidbits that are important to them right now, since for any tool of reasonable complexity 95% of the documentation is going to be irrelevant to any single user in any single situation. + +How could the manpage be improved? Well, for starters, it could _not lie_ about the fact that the `-i` option doesn't do anything with private keys at all. If it were _really_ trying to be helpful it could even throw in a hint that if you want to work with private keys, you're barking up the wrong tree. Maybe it could say "Note: This option only accepts public keys. For private key conversions see -p and -m." or something like that. The `-p` option should probably also mention somewhere in its description that it happens to also be the preferred method for converting formats, since right now it only talks about changing passphrases. + +Anyway, thanks for listening to my TED talk. At least now I'll never forget how to convert a private key again. diff --git a/src/styles/main.scss b/src/styles/main.scss index 40ce68c..dd6b3d9 100644 --- a/src/styles/main.scss +++ b/src/styles/main.scss @@ -21,6 +21,12 @@ --primary-color-faded: hsl(202deg 14% 36%); --accent-color: hsl(0deg, 92%, 29%); --accent-color-faded: hsl(0deg, 25%, 55%); + + @media(max-width: 640px) { + --content-line-height: 1.25; + --content-size: 1.15rem; + --content-size-sm: 0.9rem; + } } body { @@ -29,4 +35,5 @@ body { line-height: var(--content-line-height); letter-spacing: -0.005em; color: var(--content-color); + }