From 1385ad286fc219c7b61eba1e9d5079725c38a07e Mon Sep 17 00:00:00 2001 From: Sprite Date: Sat, 12 Sep 2026 17:32:17 +0000 Subject: [PATCH] docs: the server is on the sprite and internet-exposed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The long-standing open question "where will the server actually live" is answered for now: it stays on this sprite, bound to 0.0.0.0:8090 and published over HTTPS by the sprite proxy. Several docs asserted the opposite — HANDOFF said "127.0.0.1:8090, deliberately NOT internet-exposed (no --http-port, so the sprite proxy can't reach it)", which is flatly wrong today. The consequence is the part worth writing down: PocketBase's API rules are now the only thing between this library and the internet. There is no NAT, no VPN, no reverse proxy. So the anonymous-access curls stop being a formality, and they have to run against the PUBLIC hostname — localhost cannot tell you what the world can reach. Re-verified that way: books/shelves/bookcases LIST all 403, self-registration 403, health 200. One gap found while re-verifying, recorded but NOT fixed: users LIST answers 200 with an empty array instead of 403. Nothing is disclosed — two real accounts exist and the listRule filters both out — but it is the same wrong-signal quirk pb_hooks/main.pb.js exists to close, and that hook never listed the users collection. SPEC's offline-first rationale is amended rather than its rule: the reason is no longer residential NAT but a sprite that suspends when idle and wakes on request. The rule is unchanged and does not depend on which. server/deploy/ still documents systemd/Docker/Tailscale on home hardware; it now says up front that this is the intended end state, not what is running. Also corrected, since it was adjacent and plainly false: README still claimed the app had never run on a physical device. It has, since 2026-09-09. What is true is that no *automated* test runs on a device — there is no emulator on this box. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PPpdG8VnRfS3KkisR3HUAE --- README.md | 51 ++++++++++++++++++++++++------- docs/HANDOFF.md | 66 ++++++++++++++++++++++++++++++++++++----- docs/SPEC.md | 12 +++++--- server/README.md | 37 ++++++++++++++++++++--- server/deploy/README.md | 9 ++++++ 5 files changed, 149 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 5e61ba0..da48773 100644 --- a/README.md +++ b/README.md @@ -124,6 +124,34 @@ that keystore). ## Deploying the server +### Where this library's server actually runs today + +The live instance runs **on the development sprite** and is **reachable from +the open internet** over HTTPS, on a sprite-issued hostname. That is a +deliberate interim choice — it is what the phones are pointed at — and it has +one consequence worth stating plainly, because the rest of this README was +written under the opposite assumption: + +> **PocketBase's API rules are the only thing standing between this library +> and the internet.** There is no home NAT, no VPN, and no reverse proxy in +> front of it. The private-by-construction properties described above +> (`createRule = null` on `users`, every collection requiring a logged-in +> user, and the `pb_hooks` guard that turns an anonymous list into a 403 +> rather than an empty 200) are load-bearing rather than defence-in-depth. +> Verify them against the *public* hostname, not `127.0.0.1` — localhost +> cannot tell you what the world can reach. `server/README.md` has the curls. + +The sprite suspends when idle and wakes on an incoming request, so the first +call after a quiet period pays a cold start. The app is offline-first, so +this shows up as a slower sync rather than a failure. + +Moving to home hardware (a NUC, an old laptop, a Pi) behind Tailscale is +still the intended end state, and the deploy docs below describe it. Nothing +in the app has to change to make that move: the server URL is entered on the +setup screen, never compiled in. + +### Running it on your own hardware + See [`server/README.md`](server/README.md) for the schema and provisioning scripts, and [`server/deploy/`](server/deploy/) for running PocketBase long-term (systemd or Docker), reaching it from outside your home network @@ -153,17 +181,18 @@ meant to be, publicly distributed. Read this before assuming more polish than exists: -- **The app has never run on a physical device or emulator.** This - environment has no KVM, so there is no Android emulator available. - Everything here was verified via `./gradlew assembleDebug`, - `testDebugUnitTest` (JVM/Robolectric unit tests), and Paparazzi screenshot - rendering (`src/test/snapshots/images/`) — real logic paths (ISBN - checksums, metadata merging, sync conflict resolution, DAO queries) are - unit-tested, and every screen has been rendered to a static PNG in both - light and dark theme, but nothing has been tap-tested on an actual screen. - Camera/barcode scanning in particular has only been exercised through unit - tests of the pure logic (`IsbnBarcodeAnalyzer`/`ScanCodeFilter`), never a - live camera. +- **No automated testing runs on a device.** The app itself has been + installed and used on a real phone since 2026-09-09, and several rounds of + fixes came out of that; but this environment has no KVM, so there is no + emulator, and every *automated* check is a JVM one. Verification is + `./gradlew assembleDebug`, `testDebugUnitTest` (JVM/Robolectric unit + tests), and Paparazzi screenshot rendering + (`src/test/snapshots/images/`) — real logic paths (ISBN checksums, + metadata merging, sync conflict resolution, DAO queries) are unit-tested, + and every screen is rendered to a static PNG in both light and dark theme. + Camera/barcode scanning is covered only through unit tests of the pure + logic (`IsbnBarcodeAnalyzer`/`ScanCodeFilter`); the live camera path is + exercised by hand on a phone or not at all. - **Sync has been round-tripped against a real PocketBase exactly once** (a live-server test covering auth, push with client-generated ids, pull, last-write-wins, tombstones, and a byte-for-byte cover round-trip — see diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index c8ddc2f..3bf1518 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -64,11 +64,60 @@ change it, write a NEW file and use that for the next wave. - Android SDK: `~/toolchain/android-sdk` (platforms;android-37.0, build-tools;37.0.0, platform-tools) - **No KVM, no emulator.** Verify only via `./gradlew assembleDebug`, JVM unit tests, and Paparazzi PNG rendering. Never claim the app was "run". -- PocketBase v0.40.2 service, **127.0.0.1:8090, deliberately NOT internet-exposed** - (no `--http-port`, so the sprite proxy can't reach it). Restart: - `sprite-env services restart pocketbase`. Logs: `/.sprite/logs/services/pocketbase.log`. +- PocketBase v0.40.2 service, **bound to `0.0.0.0:8090` and INTERNET-EXPOSED** via the + sprite proxy at **https://bookshelf-dev-b2jqx.sprites.app** (the service is registered + with `--http-port 8090`, so the proxy routes to it). This is the live server the user's + phone talks to. Restart: `sprite-env services restart pocketbase`. + Logs: `/.sprite/logs/services/pocketbase.log`. See "Where the server lives" below. - Superuser creds: `server/.dev-credentials` (gitignored). +## Where the server lives — ANSWERED 2026-09-12: on this sprite, publicly +This was an open question through waves 1-7 and it now has an answer: **for now the +server stays here on the sprite and is reachable from the open internet.** The user's +phone talks to `https://bookshelf-dev-b2jqx.sprites.app`. Earlier notes in this file +and in `server/deploy/` said the opposite (localhost-only, unreachable from outside); +they were true when written and are now corrected in place. `server/deploy/` still +documents systemd/Docker/Tailscale for an eventual move to home hardware — that is a +future option, not what is running. + +Consequences that were not true when the app was designed: + +1. **PocketBase's own API rules are now the ONLY thing between this library and the + internet.** There is no NAT, no Tailscale, no reverse proxy in front of it. The + security curls in "Verification standard" below stopped being a formality the day + this changed — run them against the PUBLIC hostname, not `127.0.0.1`, because + localhost cannot tell you what the world can reach. +2. **The sprite auto-suspends when idle** and wakes on an incoming HTTP request + (that is what `--http-port` buys). So the server is still "often unreachable" in + the sense SPEC's offline-first rule cares about — just for a different reason than + residential NAT, and with a cold-start delay on the first request after a pause + rather than a hard failure. The size of that delay has **not** been measured; if a + sync or a first login ever looks pathologically slow, measure it before assuming a + bug in the app. +3. The URL is a sprite-scoped hostname. If this sprite is ever renamed or rebuilt the + URL changes, and every phone has to be re-pointed at it on the setup screen. That + is an argument for moving to Tailscale + real hardware eventually, not a reason to + hardcode anything. + +### Posture re-verified against the PUBLIC hostname, 2026-09-12 +| Check (anonymous, over the internet) | Result | +|---|---| +| `GET /api/collections/books/records` | **403** `{"message":"Authentication required."}` | +| `GET /api/collections/shelves/records` | **403** | +| `GET /api/collections/bookcases/records` | **403** | +| `POST /api/collections/users/records` (self-registration) | **403** | +| `GET /api/health` | 200 (intended — it is a health check and leaks nothing) | +| `GET /api/collections/users/records` | **200 `{"items":[]}`** — see below | + +**Known gap, not a leak:** `users` LIST answers **200 with an empty array** instead of +403. No data escapes — there are two real accounts in the DB and an anonymous caller +sees neither, because `listRule` filters the rows out — but the status code is wrong +and it is the exact PocketBase quirk `pb_hooks/main.pb.js` exists to paper over. That +hook lists only `bookcases`, `shelves`, `books`; `users` was never added to it. One +word of a fix, and worth doing now that the collection is world-reachable: some +clients read "200 with []" as an allowed request. **Not yet done — do not record it as +done until the curl above returns 403.** + ## STATE: what is DONE ### Wave 1A — server: COMPLETE and verified by the orchestrator (not just self-reported) `server/` contains `setup-schema.sh` (idempotent), `create-user.sh`, `pb_hooks/main.pb.js`, @@ -158,7 +207,8 @@ Workers self-report optimistically. Before accepting any wave: the user, not something to paper over — the user explicitly cares how this looks. ## Open questions for the user (not yet asked — deferred, not forgotten) -- Where the server will actually live (home box vs a sprite) — only affects the deploy README. +- ~~Where the server will actually live (home box vs a sprite)~~ — **ANSWERED 2026-09-12: + it stays on this sprite, internet-exposed. See "Where the server lives" above.** - Their two account emails, for `create-user.sh`. Not needed until the app can log in. ## HAZARD #5 — THE SPRITE AUTO-SUSPENDS; detached workers do NOT keep it awake @@ -283,8 +333,9 @@ library, scaffold — each light + dark). 3. **R8 is off.** Acceptable per F3's prompt, but the release APK is 41.8MB. Turning it on requires proving Room/Retrofit/kotlinx-serialization/ML Kit survive minification. -4. Still-open user questions, unchanged: where the server will actually live, and - the two account emails for `create-user.sh`. +4. Still-open user questions: the two account emails for `create-user.sh`. + (Where the server will live was ANSWERED on 2026-09-12 — it stays on this + sprite, internet-exposed. See "Where the server lives" near the top.) ## First on-device test — 2026-09-09 The user installed the signed APK on a real phone. It runs. This closes the @@ -372,7 +423,8 @@ that. **Still open, unchanged:** the free Google Books API key (keyless returns 429; worth doing on its own merits but no longer the leading theory), R8 still off so the release APK is 41.8MB and too large to send over the file channel (30MB cap), -where the server will live, and the two account emails for `create-user.sh`. +and the two account emails for `create-user.sh`. (Where the server will live +was ANSWERED on 2026-09-12: it stays on this sprite, internet-exposed.) ## Wave 5 — G-diagnostics: COMPLETE, verified by the orchestrator 2026-09-09 Commit `93f972b`. The app now distinguishes barcode-didn't-decode from diff --git a/docs/SPEC.md b/docs/SPEC.md index dc0e7ec..443830a 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -4,10 +4,14 @@ Two-person shared home library. Android app + self-hosted PocketBase. ALL workers must follow this exactly. Do not invent alternative names. ## Non-negotiables -- Offline-first. Home server is often unreachable (residential NAT). Every read - comes from Room. Every write lands in Room first, syncs later. No screen may - block on network. -- Private. No public registration. Auth required for all data access. +- Offline-first. The server is often slow or unreachable. Every read comes from + Room. Every write lands in Room first, syncs later. No screen may block on + network. (The original reason was residential NAT; the server currently runs + on a sprite that suspends when idle and wakes on request, so the same rule + holds for a different reason. The requirement does not depend on which.) +- Private. No public registration. Auth required for all data access. As of + 2026-09-12 the server is INTERNET-EXPOSED, so these rules are the only thing + protecting the library — not defence-in-depth behind a home NAT. - Server URL is NOT hardcoded; user enters it on first run. ## Repo layout diff --git a/server/README.md b/server/README.md index 543642c..a5c3b02 100644 --- a/server/README.md +++ b/server/README.md @@ -66,16 +66,45 @@ no recompilation, no framework — so it ships and deploys exactly like ## Verifying a deployment +**Run these against the address the outside world uses, not `127.0.0.1`.** +A localhost curl cannot tell you what is reachable from the internet, and +the live instance of this server (see below) *is* on the internet, with +these rules as its only protection. + ```sh -# Anonymous requests must be rejected (4xx) for all three collections: -curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8090/api/collections/books/records -# -> 403 +BASE=https://your-server.example # the PUBLIC base URL, not localhost + +# Anonymous requests must be rejected for all three collections: +for c in books shelves bookcases; do + curl -s -o /dev/null -w "$c %{http_code}\n" "$BASE/api/collections/$c/records" +done +# -> 403, 403, 403 # Self-registration must be rejected: -curl -s -X POST http://127.0.0.1:8090/api/collections/users/records \ +curl -s -X POST "$BASE/api/collections/users/records" \ -H 'Content-Type: application/json' -d '{"email":"x@x.com","password":"password123","passwordConfirm":"password123"}' # -> 403 "Only superusers can perform this action." ``` +A `200` with a non-empty `items` array from any of those is a breach, and a +`200` with an empty one means the `pb_hooks` guard is not loaded — see the +quirk described above. + +> **Known gap:** `GET /api/collections/users/records` currently answers +> **200 with an empty array** rather than 403. Nothing is disclosed — the +> `listRule` filters every row out, so an anonymous caller sees no accounts, +> no emails, no ids — but it is the same wrong-signal quirk as above, and +> `pb_hooks/main.pb.js` does not yet cover the `users` collection. Adding it +> to that hook's collection list closes it. + +## Where the live server runs + +The instance these phones talk to runs **on the development sprite**, bound +to `0.0.0.0:8090` and published over HTTPS on a sprite-issued hostname, so +it is **internet-exposed**. That is an interim arrangement; `deploy/` covers +the intended move to home hardware behind Tailscale. The app never hardcodes +a server URL — it is entered on the first-run setup screen — so that move +needs no code change, only re-pointing each phone. + See `deploy/README.md` for running the server long-term, reaching it from outside your home network, and backups. diff --git a/server/deploy/README.md b/server/deploy/README.md index 0870904..f655bb3 100644 --- a/server/deploy/README.md +++ b/server/deploy/README.md @@ -1,5 +1,14 @@ # Deploying Bookshelf's PocketBase server +> **This is not what is running today.** The live server currently runs on the +> development sprite, bound to `0.0.0.0:8090` and published over HTTPS on a +> sprite-issued hostname — **internet-exposed**, with PocketBase's API rules as +> its only protection (see `../README.md` § "Where the live server runs"). This +> document describes the *intended* end state: your own hardware at home, not +> directly exposed. Where it says PocketBase is bound to localhost and +> unreachable from outside, that is a statement about the deployment described +> here, not about the instance the phones are currently pointed at. + This assumes a spare always-on machine at home (a mini PC, NUC, Raspberry Pi, or an old laptop) running Linux. Pick **one** of the two run methods below — systemd or Docker — not both.