bookshelf/docs/HANDOFF.md

Bookshelf — session handoff

Written 2026-09-06 by the Opus orchestrator, after a sprite restart forced a fresh session.

FIRST COMMAND OF A NEW SESSION — is a wave already finished?

Workers run fully detached (hazard #5) and OUTLIVE the orchestrator. A wave can finish while no orchestrator exists. Nothing will "notify" a session that was not running, so completion is recorded ON DISK. Run this before anything else:

cd ~/bookshelf && cat logs/WAVE*-DONE 2>/dev/null; \
  pgrep -fc 'run-task\.sh' ; tail -2 logs/*.state

• logs/WAVE<N>-DONE exists -> that wave's workers have STOPPED. Read it, then independently verify (assembleDebug + testDebugUnitTest + git status --porcelain) before accepting anything. Workers self-report optimistically; two of three waves so far over-claimed.
• no sentinel + pgrep count > 0 -> still running; arm a Monitor and wait.
• no sentinel + count 0 -> workers were KILLED. Check logs/<name>.json: 0 bytes means killed, not failed (hazard #3). Relaunch with setsid per hazard #5; run-task.sh will RESUME the existing session id rather than restart, so context/quota is preserved.

The sentinel is written by tasks/wave-sentinel.sh, itself launched detached: setsid nohup ./tasks/wave-sentinel.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 & An in-process Monitor is only a convenience for a LIVE orchestrator; it dies with the process and caps at 1h. Never rely on it as the record that a wave completed.

Read these first, in order

1. docs/SPEC.md — the authoritative product/technical contract. Unchanged and still correct. Every worker prompt must point at it. Do not restate it; do not let it drift.
2. This file — operational state, what's done, what bit us, what's next.

Operating model (the user explicitly asked for this — keep it)

The user is on the $20/mo Pro plan and wants Opus used sparingly.

• Opus = orchestrator only. Write specs, launch workers, verify results, decide. Do NOT write app code yourself. Do NOT read large files into Opus context.
• Sonnet = all implementation, via claude -p (NOT the Agent tool — the user asked for claude -p specifically, and it keeps worker output out of the orchestrator's context).
• Read worker output via logs/<name>.summary / jq -r '.result', never by cat-ing source.
• Quota status: user reported 62% of the 5h window consumed at ~19:45 on 09-05. Worker A alone cost $2.40 / 85 turns. Budget accordingly; prefer resuming a session over restarting one.

How to launch a worker (BOTH steps — the guard is not optional)

cd ~/bookshelf
setsid nohup ./tasks/run-task.sh <NAME> ./tasks/<NAME>.txt </dev/null >/dev/null 2>&1 &
setsid nohup ./tasks/wave-guard.sh WAVE<N>-DONE <NAME> [<NAME>...] </dev/null >/dev/null 2>&1 &

Without the guard the sprite auto-suspends as soon as the user's console goes idle and the whole wave is lost (hazard #5). The guard holds a /v1/tasks lease, renews it every 15 min, writes logs/WAVE<N>-DONE at the end, and releases the lease so the box can sleep. Check it with sprite-env curl /v1/tasks and cat logs/wave-guard.log. tasks/run-task.sh is quota-aware: on a usage-limit error it sleeps POLL (600s) and resumes the SAME session rather than restarting, up to MAX_WALL (24h), and does not count quota waits against its 3-strike hard-failure budget. It writes: logs/<name>.json (final result), .err, .sid (session id), .state (progress), .summary.

HAZARD — do not repeat: never edit run-task.sh while workers are running. Bash reads scripts by byte offset; swapping the file mid-run makes live workers resume inside unrelated code and can spawn duplicate claude processes that burn quota on finished work. If you must change it, write a NEW file and use that for the next wave.

Environment

• JDK 21: ~/toolchain/jdk21 (system java is 25 — too new for AGP, do not use it)
• Android SDK: ~/toolchain/android-sdk (platforms;android-37.0, build-tools;37.0.0, platform-tools)
• No KVM, no emulator. Verify only via ./gradlew assembleDebug, JVM unit tests, and Paparazzi PNG rendering. Never claim the app was "run".
• PocketBase v0.40.2 service, 127.0.0.1:8090, deliberately NOT internet-exposed (no --http-port, so the sprite proxy can't reach it). Restart: sprite-env services restart pocketbase. Logs: /.sprite/logs/services/pocketbase.log.
• Superuser creds: server/.dev-credentials (gitignored).

STATE: what is DONE

Wave 1A — server: COMPLETE and verified by the orchestrator (not just self-reported)

server/ contains setup-schema.sh (idempotent), create-user.sh, pb_hooks/main.pb.js, pb_migrations/, deploy/ (systemd unit, Dockerfile, compose, backup.sh, README covering Tailscale vs port-forward+Caddy), README.md, .gitignore.

Independently re-verified on 09-06 after fixing the service:

Check	Result
anonymous LIST books/shelves/bookcases	403 / 403 / 403
anonymous self-registration	403
/api/health	200

pb_hooks/main.pb.js is INTENTIONAL, not scope drift. PocketBase's listRule is a row filter, so anonymous LIST would otherwise return 200 [] instead of an error. The hook forces 403. Keep it; it is why the table above passes. It is auto-loaded by the stock binary.

Wave 1B — Android scaffold + design system: COMPLETE, verified by the orchestrator

The pre-restart worker had gotten much further than the last handoff recorded. On 09-06 the orchestrator found everything on disk (theme, 7 shared components, MainActivity, BookshelfApplication, Paparazzi test, all 8 Literata TTFs) and only THREE compile errors, all the same class of trivial import bug — fixed directly by the orchestrator rather than spending a worker session on two-line edits:

• import androidx.compose.foundation.layout.weight (x2: BookshelfScaffold.kt, the Paparazzi test) — that resolves to the internal RowColumnParentData.weight. weight is a ColumnScope/RowScope member; it needs NO import. Delete the line.
• SyncStatusBar.kt was missing import androidx.compose.runtime.getValue, so val x by transition.animateFloat(...) had no delegate.

Check	Result
./gradlew assembleDebug	exit 0 — app-debug.apk, 47MB
./gradlew testDebugUnitTest	exit 0
./gradlew recordPaparazziDebug	exit 0 — 10 PNGs, light+dark

Snapshots: app/app/src/test/snapshots/images/. The orchestrator eyeballed scaffold-light: warm paper ground, Literata serif title, thin gold hairline rule. Matches the design language.

Repo is now a git repo

git init + baseline commit 8bcd9f7 at the 1B-green point. This is deliberate: it lets the orchestrator verify a wave with git diff --stat / git log instead of reading source files into Opus context, and gives a rollback that isn't a whole-sprite checkpoint restore. Root .gitignore covers build outputs, server/pb_data, .dev-credentials, worker logs.

STATE: what is IN FLIGHT

Wave 2 — C (data layer) + D (metadata/scanning): LAUNCHED 09-06 ~01:59Z, running in parallel

Prompts: tasks/C-data.txt, tasks/D-metadata.txt. Sessions: C-data=c2b92ca5-55d7-49b2-8a89-dc36e3ba4c9f, D-metadata=9a2c0de8-e475-4a43-9302-bc66889ee2bd

Two coordination devices were put in place before launch; keep them for wave 3:

1. tasks/gw — a flock-serialized gradle wrapper. Both workers share ONE Gradle project dir; concurrent ./gradlew runs clobber each other's outputs. Both prompts forbid ./gradlew and require tasks/gw. Reuse this for every future parallel wave.
2. Disjoint file ownership, stated as a hard boundary in each prompt. C owns data.local, data.remote, data.repo, data.prefs, AppContainer, BookshelfApplication. D owns data.metadata and ui.scan plumbing. NEITHER may touch app/build.gradle.kts or libs.versions.toml — the orchestrator confirmed every wave-2 dependency is ALREADY declared and wired, so there is no legitimate reason for a worker to edit a build file. D must not wire MetadataRepository into AppContainer (C owns it); D reports the one-line snippet instead, to be applied later.

STATE: what is NOT done

Waves 3-4 — not started. Prompts not yet written.

• Wave 3 (after C+D land and are verified): E — the six screens (setup, library, detail, scan, locations, settings).
• Wave 4: F — Paparazzi screenshots for the user to judge the look, release keystore + signed APK, top-level README, end-to-end sync test against the live PocketBase.

Gotchas already paid for — do not rediscover

1. Migration filename ↔ _migrations desync. Worker A renamed 1788636563_created_books.js to ...564... to fix an alphabetical-replay ordering bug (books sorted before shelves, breaking the relation). Correct for fresh instances, but the dev DB still had the old name recorded applied, so PocketBase tried to re-create books and crash-looped 9 times. Fixed via UPDATE _migrations SET file=.... If you ever rename a migration, update that table too. DB backup: scratchpad data.db.bak.
2. Rule semantics: in PocketBase "" means PUBLIC, null means superuser-only. Confusing these is exactly how the library would end up world-readable.
3. claude -p --output-format json writes its log only at exit; a 0-byte .json means the worker is still running or was killed, not that it failed.
4. System JDK is 25 and will break AGP. Workers must export JAVA_HOME=~/toolchain/jdk21 (run-task.sh already does).

Verification standard (hold workers to this)

Workers self-report optimistically. Before accepting any wave:

• Re-run the security curls above yourself. The user's stated requirement is that this not be "accessible to everyone in the world"; that check is non-negotiable and cheap.
• Require assembleDebug + test exit 0, and confirm artifacts exist on disk.
• Treat "I couldn't get Paparazzi working so I skipped screenshots" as a finding to report to the user, not something to paper over — the user explicitly cares how this looks.

Open questions for the user (not yet asked — deferred, not forgotten)

• Where the server will actually live (home box vs a sprite) — only affects the deploy README.
• Their two account emails, for create-user.sh. Not needed until the app can log in.

HAZARD #5 — THE SPRITE AUTO-SUSPENDS; detached workers do NOT keep it awake

This is the real cause of the wave-3 loss on 09-06, and an earlier note in this file blamed the wrong thing (it claimed nohup process-group semantics). Correct diagnosis, credit to the user: /.sprite/llm.txt says "When idle, sprites pause automatically" and "Services and sessions keep sprites alive." A detached background process is on NEITHER list. Wave 3 was launched at 03:25 with setsid nohup, the user's console session went away, the sprite went COLD, and all process state was lost. Evidence: at 10:07 uptime read "up 2 min" (boot 10:05:40) while the workers' session files stopped at 03:25 — a machine that stopped and rebooted, not a signalled process.

FIX — hold a sprite task lease for the duration of the wave. Undocumented in /.sprite/docs but live on the API socket: POST /v1/tasks {"name":"","expire":"3600s"} -> holds the sprite HOT GET /v1/tasks -> list active leases DELETE /v1/tasks/ -> release

• max expire is 3600s (a 2h request is rejected: "exceeds maximum 3600 seconds")
• re-POSTing a live name returns 409, so RENEWAL = DELETE then POST
• the lease lives server-side, so it keeps the box up independently of any process; a renewal loop running on the sprite therefore sustains itself

tasks/wave-guard.sh does all of this: acquires the lease, renews every 15 min while workers run, then writes logs/WAVE<N>-DONE and RELEASES the lease so the sprite can suspend instead of idling hot on the user's dime. Launch it detached alongside a wave: setsid nohup ./tasks/wave-guard.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 & Still launch workers with setsid (needed so they survive the orchestrator exiting), but understand that alone it does NOT survive a suspend. The lease is what does.

HAZARD #6 — pgrep -f / pkill -f match YOUR OWN shell

Bitten three times in one session, once fatally: pkill -f wave-sentinel.sh killed the orchestrator's own shell (exit 144) because the bash -c command line contained that literal string. Same bug made pgrep -f "claude -p" report a phantom running worker. Use the bracket trick (ps aux | grep "[c]laude -p") or match on argv shape (ps -eo pid,args | grep -E "tasks/(wave-guard|run-task)" | grep -v grep).

HAZARD #7 — never git add -A while workers are running

The orchestrator committed twice (10:09, 10:22) while E1/E2 were actively writing files. git add -A swept half-finished wave-3 SOURCE into commits whose messages said "orchestration tooling". That silently defeats the whole reason this repo exists as git: per-wave git diff --stat verification. It also produced a fake-clean git status, which briefly looked like the workers had produced nothing at all. Rules:

• While a wave is in flight, commit with an EXPLICIT pathspec only, e.g. git add docs/HANDOFF.md tasks/ && git commit ... — never -A, never ..
• Do the wave's own git add -A commit only AFTER the sentinel exists and the build and tests have been independently verified.
• If it happens anyway: git reset --soft <last-good>, git reset, then re-commit in honest slices. Safe here — the repo has no remote and checkpoints exist. Done once already (commits d947aa7/058e864 were rebuilt into dbb0726 + a022a1b).

Wave 3 — COMPLETE, verified by the orchestrator on 09-06

E1 (nav/setup/locations/settings) and E2 (library/detail/scan) both reported SUCCESS; E2 took one quota wait and run-task.sh resumed it correctly.

Check	Result
assembleDebug	exit 0
testDebugUnitTest	exit 0 — 91 tests, 0 failures, 0 errors (was 68)
boundary check	clean: neither touched build files or the other's packages
Commit a022a1b (32 files, +3081).

Known gaps carried into wave 4 — do not lose these:

1. Cover pipeline has NEVER run against a real PocketBase. Download-on-create and multipart upload-on-sync were only exercised against fakes (worker C's own report). This is the single most likely place a real bug is hiding. Wave 4 must do a live round-trip against 127.0.0.1:8090.
2. Settings shows the PocketBase user id, not the email — AuthRepository/SettingsStore never persist the login email (E1's report). Cosmetic, needs a data-layer change.
3. No Room foreign keys between books/shelves/bookcases (deliberate, worker C).
4. No emulator on this box: nothing has ever been run, only compiled and unit-tested. Paparazzi PNGs are the only evidence of how any of it actually looks.

Wave 4 — COMPLETE, verified by the orchestrator on 2026-09-08

This is the last planned wave. All of SPEC's build/verify gates now pass.

Check	Result
./tasks/gw assembleDebug	exit 0
./tasks/gw testDebugUnitTest	exit 0 — 102 tests, 1 skipped, 0 failures, 0 errors
./tasks/gw assembleRelease	exit 0
signed APK	app/app/build/outputs/apk/release/app-release.apk, 41,777,344 bytes
apksigner verify --print-certs	V2 signer CN=Bookshelf, O=Montanaro — the real release key, not the debug cert
secrets	app/release-keystore.jks + app/keystore.properties gitignored and NOT committed — re-verified against the staged file list before committing

The skipped test is LiveSyncTest — opt-in, it needs the live PocketBase. It PASSED in wave 4's first half; the cover round-trip gap from wave 3 is closed.

Commits 5455df2 (app work) + 0f47ee9 (orchestration). Pushed to origin/main (ssh://git@git.jfmonty2.com:2022/jfmonty2/bookshelf.git) — the repo now has a remote, so pushing after each commit is the norm.

How wave 4 actually ended — F3 never reported

F3-release wrote all four deliverables between 20:34 and 20:46 on 09-08 and then could not finish. Two separate mechanisms:

• claude -p terminates background tasks after 600s (Background tasks still running after 600s; terminating. Set CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 to wait indefinitely.). F3 had launched assembleRelease in the background with a Monitor and ended its turn saying it would report when done — the exact failure its own prompt forbade.
• It then hit the 5h quota and went into a 600s wait loop, so the service supervisor would have kept re-running verification on finished work indefinitely.

The orchestrator ran the verification itself, wrote logs/WAVE4-DONE by hand (noting it was NOT written by service-worker.sh), and stopped the service. If you launch another worker, set CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 in run-task.sh, or tell the worker to run builds in the FOREGROUND. A worker that backgrounds a 10-minute Gradle build cannot ever report on it.

The one thing wave 4 owed and did not deliver

F3 was asked to say candidly what the rendered screens get WRONG against SPEC's design language, now that they can finally be seen. It never reported. Fourteen new PNGs are on disk, unreviewed by anyone. The user explicitly cares how this looks, so this is the top open item — not a build problem, a design-review one: app/app/src/test/snapshots/images/ (setup, detail, scan ×2, locations, settings, library, scaffold — each light + dark).

STATE: what is NOT done (as of 2026-09-08)

1. Nobody has looked at the screenshots. See above. Highest-value next step.
2. The app has still never run on a device or emulator (no KVM on this box). Everything is compile + unit-test + Paparazzi evidence only. Installing the signed APK on a real phone is the only way past this, and it needs the user.
3. R8 is off. Acceptable per F3's prompt, but the release APK is 41.8MB. Turning it on requires proving Room/Retrofit/kotlinx-serialization/ML Kit survive minification.
4. Still-open user questions, unchanged: where the server will actually live, and the two account emails for create-user.sh.

First on-device test — 2026-09-09

The user installed the signed APK on a real phone. It runs. This closes the "never been run" gap that waves 1-4 all carried. Six issues came back; five were fixed directly by the orchestrator in commit 356f639 (they were small, and spinning up Sonnet workers for two-line Compose edits costs more than it saves).

The one worth remembering — BookCover branched on painter.state, but in coil3 that is a StateFlow<State>, not a State. Every is AsyncImagePainter.State.X arm was therefore always false, and because a when used as a statement needs no else, it compiled clean and drew NOTHING — no cover, no placeholder, no error icon. Kotlin emitted "Check for instance is always 'false'" as a warning on four consecutive lines and the build stayed green. Grep the build log for always 'false' before accepting a wave; that warning class is a silent-dead-code detector and this build had it for months.

Two more cover defects sat behind it, both verified against the live service:

• covers.openlibrary.org/b/isbn/{isbn}-L.jpg answers 200 with a 43-byte 1x1 transparent GIF for an edition with no art. Any image loader calls that a successful load. Only ?default=false turns a miss into a 404.
• OL's DTO synthesized that URL unconditionally, so MetadataMerger's fill-blanks rule could never reach Google Books' thumbnail. The SPEC'd cover fallback was dead code. Cover URLs now come from OL's own cover object.

Also fixed: sync bar clipped by rounded display corners (now owns its navigation-bar inset, wider horizontal padding, moved into Scaffold's bottomBar slot); setup screen's Password field hidden behind the IME (safeDrawingPadding outside verticalScroll, plus Next/Next/Done IME actions and windowSoftInputMode=adjustResize); library card titles reflowed (20sp leading, author gets its own 4dp gap); scan sheet now names the ISBN and says "Searching…" instead of showing a bare spinner.

Check	Result
./tasks/gw assembleDebug	exit 0
./tasks/gw testDebugUnitTest	exit 0 — 106 tests, 1 skipped, 0 failures
./tasks/gw verifyPaparazziDebug	exit 0 against re-recorded snapshots
./tasks/gw assembleRelease	exit 0 — 41,777,376 bytes
apksigner verify	V2 signer CN=Bookshelf, O=Montanaro — real release key

Open, not started: metadata coverage

The user reported 1 of 3 scans resolving, and asked for research, not a change. Findings are in docs/METADATA-SOURCES.md. Headline: Open Library answered 88% of a 60-ISBN sample, keyless Google Books returned 429 on 60 of 60 requests, and both clients collapse every non-200 into null — so a rate-limited lookup reaches the user as "No match found." Recommended order is a free Google Books API key, then distinguishing "couldn't ask" from "not found", then retry/backoff, before adding any new source. Awaiting the user's decision; do not implement unasked.

Wave 5 — G-diagnostics: IN FLIGHT, launched 2026-09-09 10:42Z

Prompt: tasks/G-diagnostics.txt. Session id in logs/G-diagnostics.sid. Lease bookshelf-wave held and VERIFIED by tasks/wave-guard.sh, sentinel logs/WAVE5-DONE. Launched from a console the user was about to disconnect, so the lease is the only thing keeping the sprite hot — check it first if anything looks stalled: sprite-env curl /v1/tasks and tail logs/wave-guard.log.

Scope: make the app distinguish three outcomes it currently conflates — barcode-didn't-decode (silent today), lookup-request-failed (reported as "No match found"), and genuinely-not-found. SPEC's "Book metadata lookup" and "Barcode scanning" sections were rewritten to state the three-way contract (Found / NotFound / Unavailable) BEFORE launch, so the worker implements a spec rather than inventing one. This is the only wave-5 task; there is no parallel worker, so the disjoint-ownership device isn't needed, but the prompt still forbids build files and every package outside data.metadata + ui.scan.

Why this and not more sources: the two books that failed on the phone (9781883937386, 9781883937676) are BOTH fully present in Open Library, with cover art, and the app's own parser handles their real responses — there are regression fixtures and a test proving it. The coverage hypothesis is dead. See docs/METADATA-SOURCES.md § "What actually failed". Do not let a future worker "fix" this by bolting on a third data source.

Verify before accepting (workers self-report optimistically; two of five waves over-claimed): cd ~/bookshelf && ./tasks/gw assembleDebug && ./tasks/gw testDebugUnitTest
&& ./tasks/gw recordPaparazziDebug && git status --porcelain 107 tests pass today; the count must go UP and nothing may regress. Also grep the build log for always 'false' — that warning class silently blanked every book cover in this app for months and is now an explicit item in the worker's prompt. Then eyeball the two new Paparazzi PNGs (LookupFailed sheet, rejected-barcode overlay) — the user cares how this looks and no worker has ever been trusted on that.

Still open, unchanged: the free Google Books API key (keyless returns 429; worth doing on its own merits but no longer the leading theory), R8 still off so the release APK is 41.8MB and too large to send over the file channel (30MB cap), where the server will live, and the two account emails for create-user.sh.

Wave 5 — G-diagnostics: COMPLETE, verified by the orchestrator 2026-09-09

Commit 93f972b. The app now distinguishes barcode-didn't-decode from lookup-request-failed from genuinely-not-found; see the commit message and docs/METADATA-SOURCES.md.

Check	Result
assembleDebug	exit 0
testDebugUnitTest	exit 0 — 138 tests, 1 skipped, 0 failures (was 107)
verifyPaparazziDebug	exit 0
assembleRelease	exit 0 — 41,793,760 bytes, V2 signer CN=Bookshelf
boundary check	clean — no build files, no forbidden packages
grep "always 'false'"	0 hits on touched files

The worker was honest this time: everything it claimed checked out. Cost $0.28, 6 turns, one quota wait that run-task.sh resumed correctly.

The orchestrator added one thing the worker's brief didn't cover: the manual-ISBN dialog silently discarded an unparseable entry — the same silent failure this wave existed to eliminate, sitting just outside the prompt's scope. It now marks the field in error and disables "Look up" until the checksum passes. Lesson for future prompts: scope a wave by failure class, not by file list, or the instances of the class that live outside the listed files survive.

HAZARD #8 — the wave-guard can die without writing its sentinel

logs/WAVE5-DONE was written BY HAND. The guard renewed at 11:13, the worker succeeded at 11:21, and the guard neither wrote the sentinel nor logged its "guard exiting" trap line — it was killed outright. The lease expired on its own an hour later.

This breaks the first-command heuristic at the top of this file. "no sentinel

• pgrep count 0 -> workers were KILLED" was WRONG here: the worker had finished successfully. Use these instead, in this order:
  1. ls -l logs/<name>.json — 0 bytes means killed; non-zero means it finished.
  2. tail logs/<name>.state — says SUCCESS / GIVING UP / WALL CLOCK explicitly.
  3. git status --porcelain — is there actually work in the tree? The sentinel is a convenience, not the record of truth. logs/<name>.state is.