bookshelf/docs/HANDOFF.md

# Bookshelf — session handoff
Written 2026-09-06 by the Opus orchestrator, after a sprite restart forced a fresh session.

## FIRST COMMAND OF A NEW SESSION — is a wave already finished?
Workers run fully detached (hazard #5) and OUTLIVE the orchestrator. A wave can finish
while no orchestrator exists. Nothing will "notify" a session that was not running, so
completion is recorded ON DISK. Run this before anything else:

    cd ~/bookshelf && cat logs/WAVE*-DONE 2>/dev/null; \
      pgrep -fc 'run-task\.sh' ; tail -2 logs/*.state

- `logs/WAVE<N>-DONE` exists  -> that wave's workers have STOPPED. Read it, then
  independently verify (assembleDebug + testDebugUnitTest + `git status --porcelain`)
  before accepting anything. Workers self-report optimistically; two of three waves so
  far over-claimed.
- no sentinel + `pgrep` count > 0 -> still running; arm a Monitor and wait.
- no sentinel + count 0 -> workers were KILLED. Check `logs/<name>.json`: 0 bytes means
  killed, not failed (hazard #3). Relaunch with `setsid` per hazard #5; run-task.sh will
  RESUME the existing session id rather than restart, so context/quota is preserved.

The sentinel is written by `tasks/wave-sentinel.sh`, itself launched detached:
    setsid nohup ./tasks/wave-sentinel.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 &
An in-process Monitor is only a convenience for a LIVE orchestrator; it dies with the
process and caps at 1h. Never rely on it as the record that a wave completed.

## Read these first, in order
1. `docs/SPEC.md` — the authoritative product/technical contract. Unchanged and still correct.
   Every worker prompt must point at it. Do not restate it; do not let it drift.
2. This file — operational state, what's done, what bit us, what's next.

## Operating model (the user explicitly asked for this — keep it)
The user is on the **$20/mo Pro plan** and wants Opus used sparingly.
- **Opus = orchestrator only.** Write specs, launch workers, verify results, decide.
  Do NOT write app code yourself. Do NOT read large files into Opus context.
- **Sonnet = all implementation**, via `claude -p` (NOT the Agent tool — the user asked
  for `claude -p` specifically, and it keeps worker output out of the orchestrator's context).
- Read worker output via `logs/<name>.summary` / `jq -r '.result'`, never by cat-ing source.
- Quota status: user reported **62% of the 5h window consumed** at ~19:45 on 09-05.
  Worker A alone cost **$2.40 / 85 turns**. Budget accordingly; prefer resuming a
  session over restarting one.

### How to launch a worker (BOTH steps — the guard is not optional)
```
cd ~/bookshelf
setsid nohup ./tasks/run-task.sh <NAME> ./tasks/<NAME>.txt </dev/null >/dev/null 2>&1 &
setsid nohup ./tasks/wave-guard.sh WAVE<N>-DONE <NAME> [<NAME>...] </dev/null >/dev/null 2>&1 &
```
Without the guard the sprite auto-suspends as soon as the user's console goes idle and
the whole wave is lost (hazard #5). The guard holds a `/v1/tasks` lease, renews it every
15 min, writes `logs/WAVE<N>-DONE` at the end, and releases the lease so the box can
sleep. Check it with `sprite-env curl /v1/tasks` and `cat logs/wave-guard.log`.
`tasks/run-task.sh` is quota-aware: on a usage-limit error it sleeps `POLL` (600s) and
resumes the SAME session rather than restarting, up to `MAX_WALL` (24h), and does not
count quota waits against its 3-strike hard-failure budget. It writes:
`logs/<name>.json` (final result), `.err`, `.sid` (session id), `.state` (progress), `.summary`.

**HAZARD — do not repeat:** never edit `run-task.sh` while workers are running. Bash reads
scripts by byte offset; swapping the file mid-run makes live workers resume inside unrelated
code and can spawn duplicate `claude` processes that burn quota on finished work. If you must
change it, write a NEW file and use that for the next wave.

## Environment
- JDK 21: `~/toolchain/jdk21` (system java is 25 — too new for AGP, do not use it)
- Android SDK: `~/toolchain/android-sdk` (platforms;android-37.0, build-tools;37.0.0, platform-tools)
- **No KVM, no emulator.** Verify only via `./gradlew assembleDebug`, JVM unit tests, and
  Paparazzi PNG rendering. Never claim the app was "run".
- PocketBase v0.40.2 service, **127.0.0.1:8090, deliberately NOT internet-exposed**
  (no `--http-port`, so the sprite proxy can't reach it). Restart:
  `sprite-env services restart pocketbase`. Logs: `/.sprite/logs/services/pocketbase.log`.
- Superuser creds: `server/.dev-credentials` (gitignored).

## STATE: what is DONE
### Wave 1A — server: COMPLETE and verified by the orchestrator (not just self-reported)
`server/` contains `setup-schema.sh` (idempotent), `create-user.sh`, `pb_hooks/main.pb.js`,
`pb_migrations/`, `deploy/` (systemd unit, Dockerfile, compose, backup.sh, README covering
Tailscale vs port-forward+Caddy), `README.md`, `.gitignore`.

Independently re-verified on 09-06 after fixing the service:
| Check | Result |
|---|---|
| anonymous LIST books/shelves/bookcases | **403 / 403 / 403** |
| anonymous self-registration | **403** |
| `/api/health` | 200 |

**`pb_hooks/main.pb.js` is INTENTIONAL, not scope drift.** PocketBase's `listRule` is a row
filter, so anonymous LIST would otherwise return `200 []` instead of an error. The hook forces
403. Keep it; it is why the table above passes. It is auto-loaded by the stock binary.

### Wave 1B — Android scaffold + design system: COMPLETE, verified by the orchestrator
The pre-restart worker had gotten much further than the last handoff recorded. On 09-06 the
orchestrator found everything on disk (theme, 7 shared components, MainActivity,
BookshelfApplication, Paparazzi test, all 8 Literata TTFs) and only THREE compile errors,
all the same class of trivial import bug — fixed directly by the orchestrator rather than
spending a worker session on two-line edits:
- `import androidx.compose.foundation.layout.weight` (x2: BookshelfScaffold.kt, the Paparazzi
  test) — that resolves to the *internal* `RowColumnParentData.weight`. `weight` is a
  ColumnScope/RowScope member; it needs NO import. Delete the line.
- SyncStatusBar.kt was missing `import androidx.compose.runtime.getValue`, so `val x by
  transition.animateFloat(...)` had no delegate.

| Check | Result |
|---|---|
| `./gradlew assembleDebug` | **exit 0** — app-debug.apk, 47MB |
| `./gradlew testDebugUnitTest` | **exit 0** |
| `./gradlew recordPaparazziDebug` | **exit 0** — 10 PNGs, light+dark |

Snapshots: `app/app/src/test/snapshots/images/`. The orchestrator eyeballed scaffold-light:
warm paper ground, Literata serif title, thin gold hairline rule. Matches the design language.

### Repo is now a git repo
`git init` + baseline commit `8bcd9f7` at the 1B-green point. This is deliberate: it lets the
orchestrator verify a wave with `git diff --stat` / `git log` instead of reading source files
into Opus context, and gives a rollback that isn't a whole-sprite checkpoint restore.
Root `.gitignore` covers build outputs, `server/pb_data`, `.dev-credentials`, worker logs.

## STATE: what is IN FLIGHT
### Wave 2 — C (data layer) + D (metadata/scanning): LAUNCHED 09-06 ~01:59Z, running in parallel
Prompts: `tasks/C-data.txt`, `tasks/D-metadata.txt`. Sessions:
C-data=c2b92ca5-55d7-49b2-8a89-dc36e3ba4c9f, D-metadata=9a2c0de8-e475-4a43-9302-bc66889ee2bd

Two coordination devices were put in place before launch; keep them for wave 3:
1. **`tasks/gw` — a `flock`-serialized gradle wrapper.** Both workers share ONE Gradle project
   dir; concurrent `./gradlew` runs clobber each other's outputs. Both prompts forbid
   `./gradlew` and require `tasks/gw`. Reuse this for every future parallel wave.
2. **Disjoint file ownership, stated as a hard boundary in each prompt.** C owns data.local,
   data.remote, data.repo, data.prefs, AppContainer, BookshelfApplication. D owns data.metadata
   and ui.scan plumbing. NEITHER may touch `app/build.gradle.kts` or `libs.versions.toml` —
   the orchestrator confirmed every wave-2 dependency is ALREADY declared and wired, so there
   is no legitimate reason for a worker to edit a build file. D must not wire MetadataRepository
   into AppContainer (C owns it); D reports the one-line snippet instead, to be applied later.

## STATE: what is NOT done
### Waves 3-4 — not started. Prompts not yet written.
- **Wave 3 (after C+D land and are verified):** E — the six screens (setup, library, detail, scan, locations, settings).
- **Wave 4:** F — Paparazzi screenshots for the user to judge the look, release keystore +
  signed APK, top-level README, end-to-end sync test against the live PocketBase.

## Gotchas already paid for — do not rediscover
1. **Migration filename ↔ `_migrations` desync.** Worker A renamed `1788636563_created_books.js`
   to `...564...` to fix an alphabetical-replay ordering bug (`books` sorted before `shelves`,
   breaking the relation). Correct for fresh instances, but the dev DB still had the old name
   recorded applied, so PocketBase tried to re-create `books` and crash-looped 9 times.
   Fixed via `UPDATE _migrations SET file=...`. **If you ever rename a migration, update that
   table too.** DB backup: scratchpad `data.db.bak`.
2. Rule semantics: in PocketBase `""` means PUBLIC, `null` means superuser-only. Confusing these
   is exactly how the library would end up world-readable.
3. `claude -p --output-format json` writes its log only at exit; a 0-byte `.json` means the
   worker is still running or was killed, not that it failed.
4. System JDK is 25 and will break AGP. Workers must export `JAVA_HOME=~/toolchain/jdk21`
   (run-task.sh already does).

## Verification standard (hold workers to this)
Workers self-report optimistically. Before accepting any wave:
- Re-run the security curls above yourself. The user's stated requirement is that this not be
  "accessible to everyone in the world"; that check is non-negotiable and cheap.
- Require `assembleDebug` + `test` exit 0, and confirm artifacts exist on disk.
- Treat "I couldn't get Paparazzi working so I skipped screenshots" as a finding to report to
  the user, not something to paper over — the user explicitly cares how this looks.

## Open questions for the user (not yet asked — deferred, not forgotten)
- Where the server will actually live (home box vs a sprite) — only affects the deploy README.
- Their two account emails, for `create-user.sh`. Not needed until the app can log in.

## HAZARD #5 — THE SPRITE AUTO-SUSPENDS; detached workers do NOT keep it awake
This is the real cause of the wave-3 loss on 09-06, and an earlier note in this file
blamed the wrong thing (it claimed `nohup` process-group semantics). Correct diagnosis,
credit to the user: `/.sprite/llm.txt` says "When idle, sprites pause automatically" and
"Services and sessions keep sprites alive." A detached background process is on NEITHER
list. Wave 3 was launched at 03:25 with `setsid nohup`, the user's console session went
away, the sprite went COLD, and all process state was lost. Evidence: at 10:07 `uptime`
read "up 2 min" (boot 10:05:40) while the workers' session files stopped at 03:25 — a
machine that stopped and rebooted, not a signalled process.

**FIX — hold a sprite task lease for the duration of the wave.**
Undocumented in /.sprite/docs but live on the API socket:
    POST   /v1/tasks   {"name":"<lease>","expire":"3600s"}   -> holds the sprite HOT
    GET    /v1/tasks                                          -> list active leases
    DELETE /v1/tasks/<lease>                                  -> release
  - max expire is 3600s (a 2h request is rejected: "exceeds maximum 3600 seconds")
  - re-POSTing a live name returns 409, so RENEWAL = DELETE then POST
  - the lease lives server-side, so it keeps the box up independently of any process;
    a renewal loop running on the sprite therefore sustains itself

`tasks/wave-guard.sh` does all of this: acquires the lease, renews every 15 min while
workers run, then writes `logs/WAVE<N>-DONE` and RELEASES the lease so the sprite can
suspend instead of idling hot on the user's dime. Launch it detached alongside a wave:
    setsid nohup ./tasks/wave-guard.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 &
Still launch workers with `setsid` (needed so they survive the orchestrator exiting),
but understand that alone it does NOT survive a suspend. The lease is what does.

## HAZARD #6 — `pgrep -f` / `pkill -f` match YOUR OWN shell
Bitten three times in one session, once fatally: `pkill -f wave-sentinel.sh` killed the
orchestrator's own shell (exit 144) because the bash -c command line contained that
literal string. Same bug made `pgrep -f "claude -p"` report a phantom running worker.
Use the bracket trick (`ps aux | grep "[c]laude -p"`) or match on argv shape
(`ps -eo pid,args | grep -E "tasks/(wave-guard|run-task)" | grep -v grep`).

## HAZARD #7 — never `git add -A` while workers are running
The orchestrator committed twice (10:09, 10:22) while E1/E2 were actively writing
files. `git add -A` swept half-finished wave-3 SOURCE into commits whose messages said
"orchestration tooling". That silently defeats the whole reason this repo exists as git:
per-wave `git diff --stat` verification. It also produced a fake-clean `git status`,
which briefly looked like the workers had produced nothing at all.
**Rules:**
- While a wave is in flight, commit with an EXPLICIT pathspec only, e.g.
  `git add docs/HANDOFF.md tasks/ && git commit ...` — never `-A`, never `.`.
- Do the wave's own `git add -A` commit only AFTER the sentinel exists and the build
  and tests have been independently verified.
- If it happens anyway: `git reset --soft <last-good>`, `git reset`, then re-commit in
  honest slices. Safe here — the repo has no remote and checkpoints exist. Done once
  already (commits d947aa7/058e864 were rebuilt into dbb0726 + a022a1b).

## Wave 3 — COMPLETE, verified by the orchestrator on 09-06
E1 (nav/setup/locations/settings) and E2 (library/detail/scan) both reported SUCCESS;
E2 took one quota wait and run-task.sh resumed it correctly.
| Check | Result |
|---|---|
| `assembleDebug` | **exit 0** |
| `testDebugUnitTest` | **exit 0 — 91 tests, 0 failures, 0 errors** (was 68) |
| boundary check | clean: neither touched build files or the other's packages |
Commit `a022a1b` (32 files, +3081).

**Known gaps carried into wave 4 — do not lose these:**
1. **Cover pipeline has NEVER run against a real PocketBase.** Download-on-create and
   multipart upload-on-sync were only exercised against fakes (worker C's own report).
   This is the single most likely place a real bug is hiding. Wave 4 must do a live
   round-trip against 127.0.0.1:8090.
2. Settings shows the PocketBase user id, not the email — `AuthRepository`/`SettingsStore`
   never persist the login email (E1's report). Cosmetic, needs a data-layer change.
3. No Room foreign keys between books/shelves/bookcases (deliberate, worker C).
4. No emulator on this box: nothing has ever been *run*, only compiled and unit-tested.
   Paparazzi PNGs are the only evidence of how any of it actually looks.

## Wave 4 — COMPLETE, verified by the orchestrator on 2026-09-08
This is the last planned wave. All of SPEC's build/verify gates now pass.

| Check | Result |
|---|---|
| `./tasks/gw assembleDebug` | **exit 0** |
| `./tasks/gw testDebugUnitTest` | **exit 0 — 102 tests, 1 skipped, 0 failures, 0 errors** |
| `./tasks/gw assembleRelease` | **exit 0** |
| signed APK | `app/app/build/outputs/apk/release/app-release.apk`, 41,777,344 bytes |
| `apksigner verify --print-certs` | V2 signer `CN=Bookshelf, O=Montanaro` — the real release key, not the debug cert |
| secrets | `app/release-keystore.jks` + `app/keystore.properties` gitignored and NOT committed — re-verified against the staged file list before committing |

The skipped test is `LiveSyncTest` — opt-in, it needs the live PocketBase. It
PASSED in wave 4's first half; the cover round-trip gap from wave 3 is closed.

Commits `5455df2` (app work) + `0f47ee9` (orchestration). Pushed to `origin/main`
(`ssh://git@git.jfmonty2.com:2022/jfmonty2/bookshelf.git`) — the repo now has a
remote, so pushing after each commit is the norm.

### How wave 4 actually ended — F3 never reported
F3-release wrote all four deliverables between 20:34 and 20:46 on 09-08 and then
could not finish. Two separate mechanisms:
- `claude -p` terminates background tasks after 600s (`Background tasks still
  running after 600s; terminating. Set CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 to
  wait indefinitely.`). F3 had launched `assembleRelease` in the background with a
  Monitor and ended its turn saying it would report when done — the exact failure
  its own prompt forbade.
- It then hit the 5h quota and went into a 600s wait loop, so the service
  supervisor would have kept re-running verification on finished work indefinitely.

The orchestrator ran the verification itself, wrote `logs/WAVE4-DONE` by hand
(noting it was NOT written by `service-worker.sh`), and stopped the service.
**If you launch another worker, set `CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0` in
`run-task.sh`, or tell the worker to run builds in the FOREGROUND.** A worker that
backgrounds a 10-minute Gradle build cannot ever report on it.

### The one thing wave 4 owed and did not deliver
F3 was asked to say candidly what the rendered screens get WRONG against SPEC's
design language, now that they can finally be seen. It never reported. **Fourteen
new PNGs are on disk, unreviewed by anyone.** The user explicitly cares how this
looks, so this is the top open item — not a build problem, a design-review one:
`app/app/src/test/snapshots/images/` (setup, detail, scan ×2, locations, settings,
library, scaffold — each light + dark).

## STATE: what is NOT done (as of 2026-09-08)
1. **Nobody has looked at the screenshots.** See above. Highest-value next step.
2. **The app has still never run on a device or emulator** (no KVM on this box).
   Everything is compile + unit-test + Paparazzi evidence only. Installing the
   signed APK on a real phone is the only way past this, and it needs the user.
3. **R8 is off.** Acceptable per F3's prompt, but the release APK is 41.8MB.
   Turning it on requires proving Room/Retrofit/kotlinx-serialization/ML Kit
   survive minification.
4. Still-open user questions, unchanged: where the server will actually live, and
   the two account emails for `create-user.sh`.