233 lines
15 KiB
Markdown
233 lines
15 KiB
Markdown
# Bookshelf — session handoff
|
|
Written 2026-09-06 by the Opus orchestrator, after a sprite restart forced a fresh session.
|
|
|
|
## FIRST COMMAND OF A NEW SESSION — is a wave already finished?
|
|
Workers run fully detached (hazard #5) and OUTLIVE the orchestrator. A wave can finish
|
|
while no orchestrator exists. Nothing will "notify" a session that was not running, so
|
|
completion is recorded ON DISK. Run this before anything else:
|
|
|
|
cd ~/bookshelf && cat logs/WAVE*-DONE 2>/dev/null; \
|
|
pgrep -fc 'run-task\.sh' ; tail -2 logs/*.state
|
|
|
|
- `logs/WAVE<N>-DONE` exists -> that wave's workers have STOPPED. Read it, then
|
|
independently verify (assembleDebug + testDebugUnitTest + `git status --porcelain`)
|
|
before accepting anything. Workers self-report optimistically; two of three waves so
|
|
far over-claimed.
|
|
- no sentinel + `pgrep` count > 0 -> still running; arm a Monitor and wait.
|
|
- no sentinel + count 0 -> workers were KILLED. Check `logs/<name>.json`: 0 bytes means
|
|
killed, not failed (hazard #3). Relaunch with `setsid` per hazard #5; run-task.sh will
|
|
RESUME the existing session id rather than restart, so context/quota is preserved.
|
|
|
|
The sentinel is written by `tasks/wave-sentinel.sh`, itself launched detached:
|
|
setsid nohup ./tasks/wave-sentinel.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 &
|
|
An in-process Monitor is only a convenience for a LIVE orchestrator; it dies with the
|
|
process and caps at 1h. Never rely on it as the record that a wave completed.
|
|
|
|
## Read these first, in order
|
|
1. `docs/SPEC.md` — the authoritative product/technical contract. Unchanged and still correct.
|
|
Every worker prompt must point at it. Do not restate it; do not let it drift.
|
|
2. This file — operational state, what's done, what bit us, what's next.
|
|
|
|
## Operating model (the user explicitly asked for this — keep it)
|
|
The user is on the **$20/mo Pro plan** and wants Opus used sparingly.
|
|
- **Opus = orchestrator only.** Write specs, launch workers, verify results, decide.
|
|
Do NOT write app code yourself. Do NOT read large files into Opus context.
|
|
- **Sonnet = all implementation**, via `claude -p` (NOT the Agent tool — the user asked
|
|
for `claude -p` specifically, and it keeps worker output out of the orchestrator's context).
|
|
- Read worker output via `logs/<name>.summary` / `jq -r '.result'`, never by cat-ing source.
|
|
- Quota status: user reported **62% of the 5h window consumed** at ~19:45 on 09-05.
|
|
Worker A alone cost **$2.40 / 85 turns**. Budget accordingly; prefer resuming a
|
|
session over restarting one.
|
|
|
|
### How to launch a worker (BOTH steps — the guard is not optional)
|
|
```
|
|
cd ~/bookshelf
|
|
setsid nohup ./tasks/run-task.sh <NAME> ./tasks/<NAME>.txt </dev/null >/dev/null 2>&1 &
|
|
setsid nohup ./tasks/wave-guard.sh WAVE<N>-DONE <NAME> [<NAME>...] </dev/null >/dev/null 2>&1 &
|
|
```
|
|
Without the guard the sprite auto-suspends as soon as the user's console goes idle and
|
|
the whole wave is lost (hazard #5). The guard holds a `/v1/tasks` lease, renews it every
|
|
15 min, writes `logs/WAVE<N>-DONE` at the end, and releases the lease so the box can
|
|
sleep. Check it with `sprite-env curl /v1/tasks` and `cat logs/wave-guard.log`.
|
|
`tasks/run-task.sh` is quota-aware: on a usage-limit error it sleeps `POLL` (600s) and
|
|
resumes the SAME session rather than restarting, up to `MAX_WALL` (24h), and does not
|
|
count quota waits against its 3-strike hard-failure budget. It writes:
|
|
`logs/<name>.json` (final result), `.err`, `.sid` (session id), `.state` (progress), `.summary`.
|
|
|
|
**HAZARD — do not repeat:** never edit `run-task.sh` while workers are running. Bash reads
|
|
scripts by byte offset; swapping the file mid-run makes live workers resume inside unrelated
|
|
code and can spawn duplicate `claude` processes that burn quota on finished work. If you must
|
|
change it, write a NEW file and use that for the next wave.
|
|
|
|
## Environment
|
|
- JDK 21: `~/toolchain/jdk21` (system java is 25 — too new for AGP, do not use it)
|
|
- Android SDK: `~/toolchain/android-sdk` (platforms;android-37.0, build-tools;37.0.0, platform-tools)
|
|
- **No KVM, no emulator.** Verify only via `./gradlew assembleDebug`, JVM unit tests, and
|
|
Paparazzi PNG rendering. Never claim the app was "run".
|
|
- PocketBase v0.40.2 service, **127.0.0.1:8090, deliberately NOT internet-exposed**
|
|
(no `--http-port`, so the sprite proxy can't reach it). Restart:
|
|
`sprite-env services restart pocketbase`. Logs: `/.sprite/logs/services/pocketbase.log`.
|
|
- Superuser creds: `server/.dev-credentials` (gitignored).
|
|
|
|
## STATE: what is DONE
|
|
### Wave 1A — server: COMPLETE and verified by the orchestrator (not just self-reported)
|
|
`server/` contains `setup-schema.sh` (idempotent), `create-user.sh`, `pb_hooks/main.pb.js`,
|
|
`pb_migrations/`, `deploy/` (systemd unit, Dockerfile, compose, backup.sh, README covering
|
|
Tailscale vs port-forward+Caddy), `README.md`, `.gitignore`.
|
|
|
|
Independently re-verified on 09-06 after fixing the service:
|
|
| Check | Result |
|
|
|---|---|
|
|
| anonymous LIST books/shelves/bookcases | **403 / 403 / 403** |
|
|
| anonymous self-registration | **403** |
|
|
| `/api/health` | 200 |
|
|
|
|
**`pb_hooks/main.pb.js` is INTENTIONAL, not scope drift.** PocketBase's `listRule` is a row
|
|
filter, so anonymous LIST would otherwise return `200 []` instead of an error. The hook forces
|
|
403. Keep it; it is why the table above passes. It is auto-loaded by the stock binary.
|
|
|
|
### Wave 1B — Android scaffold + design system: COMPLETE, verified by the orchestrator
|
|
The pre-restart worker had gotten much further than the last handoff recorded. On 09-06 the
|
|
orchestrator found everything on disk (theme, 7 shared components, MainActivity,
|
|
BookshelfApplication, Paparazzi test, all 8 Literata TTFs) and only THREE compile errors,
|
|
all the same class of trivial import bug — fixed directly by the orchestrator rather than
|
|
spending a worker session on two-line edits:
|
|
- `import androidx.compose.foundation.layout.weight` (x2: BookshelfScaffold.kt, the Paparazzi
|
|
test) — that resolves to the *internal* `RowColumnParentData.weight`. `weight` is a
|
|
ColumnScope/RowScope member; it needs NO import. Delete the line.
|
|
- SyncStatusBar.kt was missing `import androidx.compose.runtime.getValue`, so `val x by
|
|
transition.animateFloat(...)` had no delegate.
|
|
|
|
| Check | Result |
|
|
|---|---|
|
|
| `./gradlew assembleDebug` | **exit 0** — app-debug.apk, 47MB |
|
|
| `./gradlew testDebugUnitTest` | **exit 0** |
|
|
| `./gradlew recordPaparazziDebug` | **exit 0** — 10 PNGs, light+dark |
|
|
|
|
Snapshots: `app/app/src/test/snapshots/images/`. The orchestrator eyeballed scaffold-light:
|
|
warm paper ground, Literata serif title, thin gold hairline rule. Matches the design language.
|
|
|
|
### Repo is now a git repo
|
|
`git init` + baseline commit `8bcd9f7` at the 1B-green point. This is deliberate: it lets the
|
|
orchestrator verify a wave with `git diff --stat` / `git log` instead of reading source files
|
|
into Opus context, and gives a rollback that isn't a whole-sprite checkpoint restore.
|
|
Root `.gitignore` covers build outputs, `server/pb_data`, `.dev-credentials`, worker logs.
|
|
|
|
## STATE: what is IN FLIGHT
|
|
### Wave 2 — C (data layer) + D (metadata/scanning): LAUNCHED 09-06 ~01:59Z, running in parallel
|
|
Prompts: `tasks/C-data.txt`, `tasks/D-metadata.txt`. Sessions:
|
|
C-data=c2b92ca5-55d7-49b2-8a89-dc36e3ba4c9f, D-metadata=9a2c0de8-e475-4a43-9302-bc66889ee2bd
|
|
|
|
Two coordination devices were put in place before launch; keep them for wave 3:
|
|
1. **`tasks/gw` — a `flock`-serialized gradle wrapper.** Both workers share ONE Gradle project
|
|
dir; concurrent `./gradlew` runs clobber each other's outputs. Both prompts forbid
|
|
`./gradlew` and require `tasks/gw`. Reuse this for every future parallel wave.
|
|
2. **Disjoint file ownership, stated as a hard boundary in each prompt.** C owns data.local,
|
|
data.remote, data.repo, data.prefs, AppContainer, BookshelfApplication. D owns data.metadata
|
|
and ui.scan plumbing. NEITHER may touch `app/build.gradle.kts` or `libs.versions.toml` —
|
|
the orchestrator confirmed every wave-2 dependency is ALREADY declared and wired, so there
|
|
is no legitimate reason for a worker to edit a build file. D must not wire MetadataRepository
|
|
into AppContainer (C owns it); D reports the one-line snippet instead, to be applied later.
|
|
|
|
## STATE: what is NOT done
|
|
### Waves 3-4 — not started. Prompts not yet written.
|
|
- **Wave 3 (after C+D land and are verified):** E — the six screens (setup, library, detail, scan, locations, settings).
|
|
- **Wave 4:** F — Paparazzi screenshots for the user to judge the look, release keystore +
|
|
signed APK, top-level README, end-to-end sync test against the live PocketBase.
|
|
|
|
## Gotchas already paid for — do not rediscover
|
|
1. **Migration filename ↔ `_migrations` desync.** Worker A renamed `1788636563_created_books.js`
|
|
to `...564...` to fix an alphabetical-replay ordering bug (`books` sorted before `shelves`,
|
|
breaking the relation). Correct for fresh instances, but the dev DB still had the old name
|
|
recorded applied, so PocketBase tried to re-create `books` and crash-looped 9 times.
|
|
Fixed via `UPDATE _migrations SET file=...`. **If you ever rename a migration, update that
|
|
table too.** DB backup: scratchpad `data.db.bak`.
|
|
2. Rule semantics: in PocketBase `""` means PUBLIC, `null` means superuser-only. Confusing these
|
|
is exactly how the library would end up world-readable.
|
|
3. `claude -p --output-format json` writes its log only at exit; a 0-byte `.json` means the
|
|
worker is still running or was killed, not that it failed.
|
|
4. System JDK is 25 and will break AGP. Workers must export `JAVA_HOME=~/toolchain/jdk21`
|
|
(run-task.sh already does).
|
|
|
|
## Verification standard (hold workers to this)
|
|
Workers self-report optimistically. Before accepting any wave:
|
|
- Re-run the security curls above yourself. The user's stated requirement is that this not be
|
|
"accessible to everyone in the world"; that check is non-negotiable and cheap.
|
|
- Require `assembleDebug` + `test` exit 0, and confirm artifacts exist on disk.
|
|
- Treat "I couldn't get Paparazzi working so I skipped screenshots" as a finding to report to
|
|
the user, not something to paper over — the user explicitly cares how this looks.
|
|
|
|
## Open questions for the user (not yet asked — deferred, not forgotten)
|
|
- Where the server will actually live (home box vs a sprite) — only affects the deploy README.
|
|
- Their two account emails, for `create-user.sh`. Not needed until the app can log in.
|
|
|
|
## HAZARD #5 — THE SPRITE AUTO-SUSPENDS; detached workers do NOT keep it awake
|
|
This is the real cause of the wave-3 loss on 09-06, and an earlier note in this file
|
|
blamed the wrong thing (it claimed `nohup` process-group semantics). Correct diagnosis,
|
|
credit to the user: `/.sprite/llm.txt` says "When idle, sprites pause automatically" and
|
|
"Services and sessions keep sprites alive." A detached background process is on NEITHER
|
|
list. Wave 3 was launched at 03:25 with `setsid nohup`, the user's console session went
|
|
away, the sprite went COLD, and all process state was lost. Evidence: at 10:07 `uptime`
|
|
read "up 2 min" (boot 10:05:40) while the workers' session files stopped at 03:25 — a
|
|
machine that stopped and rebooted, not a signalled process.
|
|
|
|
**FIX — hold a sprite task lease for the duration of the wave.**
|
|
Undocumented in /.sprite/docs but live on the API socket:
|
|
POST /v1/tasks {"name":"<lease>","expire":"3600s"} -> holds the sprite HOT
|
|
GET /v1/tasks -> list active leases
|
|
DELETE /v1/tasks/<lease> -> release
|
|
- max expire is 3600s (a 2h request is rejected: "exceeds maximum 3600 seconds")
|
|
- re-POSTing a live name returns 409, so RENEWAL = DELETE then POST
|
|
- the lease lives server-side, so it keeps the box up independently of any process;
|
|
a renewal loop running on the sprite therefore sustains itself
|
|
|
|
`tasks/wave-guard.sh` does all of this: acquires the lease, renews every 15 min while
|
|
workers run, then writes `logs/WAVE<N>-DONE` and RELEASES the lease so the sprite can
|
|
suspend instead of idling hot on the user's dime. Launch it detached alongside a wave:
|
|
setsid nohup ./tasks/wave-guard.sh WAVE3-DONE E1-shell E2-books </dev/null >/dev/null 2>&1 &
|
|
Still launch workers with `setsid` (needed so they survive the orchestrator exiting),
|
|
but understand that alone it does NOT survive a suspend. The lease is what does.
|
|
|
|
## HAZARD #6 — `pgrep -f` / `pkill -f` match YOUR OWN shell
|
|
Bitten three times in one session, once fatally: `pkill -f wave-sentinel.sh` killed the
|
|
orchestrator's own shell (exit 144) because the bash -c command line contained that
|
|
literal string. Same bug made `pgrep -f "claude -p"` report a phantom running worker.
|
|
Use the bracket trick (`ps aux | grep "[c]laude -p"`) or match on argv shape
|
|
(`ps -eo pid,args | grep -E "tasks/(wave-guard|run-task)" | grep -v grep`).
|
|
|
|
## HAZARD #7 — never `git add -A` while workers are running
|
|
The orchestrator committed twice (10:09, 10:22) while E1/E2 were actively writing
|
|
files. `git add -A` swept half-finished wave-3 SOURCE into commits whose messages said
|
|
"orchestration tooling". That silently defeats the whole reason this repo exists as git:
|
|
per-wave `git diff --stat` verification. It also produced a fake-clean `git status`,
|
|
which briefly looked like the workers had produced nothing at all.
|
|
**Rules:**
|
|
- While a wave is in flight, commit with an EXPLICIT pathspec only, e.g.
|
|
`git add docs/HANDOFF.md tasks/ && git commit ...` — never `-A`, never `.`.
|
|
- Do the wave's own `git add -A` commit only AFTER the sentinel exists and the build
|
|
and tests have been independently verified.
|
|
- If it happens anyway: `git reset --soft <last-good>`, `git reset`, then re-commit in
|
|
honest slices. Safe here — the repo has no remote and checkpoints exist. Done once
|
|
already (commits d947aa7/058e864 were rebuilt into dbb0726 + a022a1b).
|
|
|
|
## Wave 3 — COMPLETE, verified by the orchestrator on 09-06
|
|
E1 (nav/setup/locations/settings) and E2 (library/detail/scan) both reported SUCCESS;
|
|
E2 took one quota wait and run-task.sh resumed it correctly.
|
|
| Check | Result |
|
|
|---|---|
|
|
| `assembleDebug` | **exit 0** |
|
|
| `testDebugUnitTest` | **exit 0 — 91 tests, 0 failures, 0 errors** (was 68) |
|
|
| boundary check | clean: neither touched build files or the other's packages |
|
|
Commit `a022a1b` (32 files, +3081).
|
|
|
|
**Known gaps carried into wave 4 — do not lose these:**
|
|
1. **Cover pipeline has NEVER run against a real PocketBase.** Download-on-create and
|
|
multipart upload-on-sync were only exercised against fakes (worker C's own report).
|
|
This is the single most likely place a real bug is hiding. Wave 4 must do a live
|
|
round-trip against 127.0.0.1:8090.
|
|
2. Settings shows the PocketBase user id, not the email — `AuthRepository`/`SettingsStore`
|
|
never persist the login email (E1's report). Cosmetic, needs a data-layer change.
|
|
3. No Room foreign keys between books/shelves/bookcases (deliberate, worker C).
|
|
4. No emulator on this box: nothing has ever been *run*, only compiled and unit-tested.
|
|
Paparazzi PNGs are the only evidence of how any of it actually looks.
|