Spriteandclaude
ab83287b5d
Two user-reported shelf-testing symptoms, both a third party answering
misleadingly and the app believing it.
1. Open Library's /api/books?bibkeys=... now 404s for EVERY ISBN, including
books OL demonstrably still holds, with OL's own x-ol-stats header on the
response while /isbn/, /search.json, /api/volumes/brief and covers.* all
serve normally. OL's docs call it the "Legacy Books API" that "may be phased
out" and it is gone from their API index, so this reads as a retirement
rather than an outage. The app had been effectively single-sourced on Google
Books since it broke: every failure the user saw was a book GB lacks.
Lookup now uses /isbn/{isbn}.json — current, non-legacy, edition-level, and
the only option of the three that carries a description. /api/volumes/brief
is a near drop-in for the old response shape and was rejected precisely
because it is also legacy.
Its costs, all handled: authors are references, so AuthorNameCache resolves
and caches them for the process lifetime (books by one author get scanned in
runs off one shelf); an edition may carry NO authors, in which case they live
on the work — 9780898707168 on the user's own shelf is exactly this, so
without the work fallback the move would have silently dropped its author;
author/work requests are best-effort and can only degrade a record, never
turn Found into Unavailable.
404 on this endpoint is authoritative NotFound. The legacy endpoint reported
a miss as 200 with an empty object, which is why every non-2xx there was a
failure. Every other non-2xx still is.
2. Google Books answers zoom=2 with a grey "image not available" PNG at HTTP
200 — not a 404 — for any volume it holds no full preview of. Coil loads it
as a success, so BookCover's placeholder never fires and the cover pipeline
uploads Google's placeholder to PocketBase as the book's cover. Measured over
18 real volumes: 11 placeholders at zoom=2, 0 at zoom=1&w=400. zoom=0/3/6 are
placeholders too. normalizeCoverUrl now pins zoom=1, adds w=400 and strips
edge=curl.
SPEC.md's "Book metadata lookup" is rewritten with both rules and the evidence
for them — it was the source of the zoom=2 instruction, and would otherwise be
the reason someone restores it.
Also fixed, because it blocked verification: LibraryViewModelTest never cleared
the view models it built, and LibraryViewModel's eleven WhileSubscribed(5_000)
flows kept running five seconds into later tests, racing resetMain(). It now
cancels each viewModelScope in tearDown.
NOT fixed, reported instead: AddBookViewModel.performSave's in-flight guard is a
check-then-act and two coroutines can both pass it. Unrelated to this change
(that VM has no metadata dependency) and out of scope. See HAZARD #13.
Verified: assembleDebug exit 0; testDebugUnitTest --rerun-tasks 378 tests,
2 skipped, 0 failures (was 355); verifyPaparazziDebug exit 0, no pixels moved;
0 "always 'false'" warnings; no build files touched. LIVE_METADATA=1 live test
ran (not skipped): 9/9 Found with cover art, with the GB key absent, so Open
Library alone answered through the new endpoint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
202 lines
12 KiB
Markdown
202 lines
12 KiB
Markdown
# Bookshelf — authoritative spec
|
|
|
|
Two-person shared home library. Android app + self-hosted PocketBase.
|
|
ALL workers must follow this exactly. Do not invent alternative names.
|
|
|
|
## Non-negotiables
|
|
- Offline-first. The server is often slow or unreachable. Every read comes from
|
|
Room. Every write lands in Room first, syncs later. No screen may block on
|
|
network. (The original reason was residential NAT; the server currently runs
|
|
on a sprite that suspends when idle and wakes on request, so the same rule
|
|
holds for a different reason. The requirement does not depend on which.)
|
|
- Private. No public registration. Auth required for all data access. As of
|
|
2026-09-12 the server is INTERNET-EXPOSED, so these rules are the only thing
|
|
protecting the library — not defence-in-depth behind a home NAT.
|
|
- Server URL is NOT hardcoded; user enters it on first run.
|
|
|
|
## Repo layout
|
|
~/bookshelf/
|
|
server/ PocketBase binary(gitignored), pb_migrations/, setup-schema.sh, deploy/
|
|
app/ Android Gradle project
|
|
docs/ this spec
|
|
|
|
## Android
|
|
- applicationId/namespace: org.modg.bookshelf
|
|
- minSdk 26, compileSdk 37, targetSdk 37, JDK 21, Kotlin, Jetpack Compose, Material 3
|
|
- SDK at ~/toolchain/android-sdk ; JDK at ~/toolchain/jdk21
|
|
- DI: manual `AppContainer` held by Application. NO Hilt/kapt. Room uses KSP.
|
|
- Libs: Compose BOM, room(+ksp), retrofit2 + kotlinx-serialization converter,
|
|
okhttp logging, coil3 compose, camerax(core/camera2/lifecycle/view),
|
|
com.google.mlkit:barcode-scanning, androidx.work runtime-ktx, datastore-preferences,
|
|
navigation-compose, lifecycle-viewmodel-compose, accompanist-permissions (or manual)
|
|
|
|
## Package structure (org.modg.bookshelf.*)
|
|
data.local Room: entities, daos, BookshelfDatabase, Converters
|
|
data.remote PocketBaseApi (retrofit), dtos, PbAuthInterceptor
|
|
data.metadata OpenLibrary + GoogleBooks lookup
|
|
data.repo BookRepository, LocationRepository, SyncEngine, AuthRepository
|
|
data.prefs SettingsStore (DataStore)
|
|
ui.theme Color/Type/Theme
|
|
ui.library, ui.detail, ui.scan, ui.locations, ui.settings, ui.setup
|
|
ui.nav BookshelfNavHost
|
|
|
|
## Data model — Room mirrors PocketBase 1:1
|
|
IDs: 15-char lowercase alnum, GENERATED CLIENT-SIDE for new records
|
|
(PocketBase accepts client-supplied ids on create). Never remap ids after push.
|
|
|
|
BookEntity(id PK, title, subtitle, authorsJson, isbn13, isbn10, publisher,
|
|
publishedDate, pageCount:Int?, description, coverUrl, coverSourceUrl,
|
|
shelfId:String?, notes, addedBy, deleted:Boolean, createdAt:Long, updatedAt:Long,
|
|
syncState:SyncState, localCoverPath:String?)
|
|
BookcaseEntity(id PK, name, note, position:Int, deleted, createdAt, updatedAt, syncState)
|
|
ShelfEntity(id PK, bookcaseId, label, position:Int, deleted, createdAt, updatedAt, syncState)
|
|
|
|
enum SyncState { SYNCED, PENDING_CREATE, PENDING_UPDATE, PENDING_DELETE }
|
|
|
|
All queries filter `deleted = 0`. Deletion is ALWAYS soft (tombstone) so sync can
|
|
propagate it and nothing is silently lost from a shared library.
|
|
|
|
## PocketBase schema (collections)
|
|
bookcases: name(text,req), note(text), position(number), deleted(bool)
|
|
shelves: bookcase(relation->bookcases,req,maxSelect 1), label(text,req),
|
|
position(number), deleted(bool)
|
|
books: title(text,req), subtitle(text), authors(json), isbn13(text), isbn10(text),
|
|
publisher(text), published_date(text), page_count(number), description(text),
|
|
cover(file,maxSelect 1,image mimes,thumbs 100x150+300x450),
|
|
cover_source_url(text), shelf(relation->shelves,maxSelect 1),
|
|
notes(text), added_by(relation->users,maxSelect 1), deleted(bool)
|
|
All three get autodate created/updated.
|
|
Indexes: books(isbn13), books(updated), shelves(updated), bookcases(updated).
|
|
|
|
API rules — all of list/view/create/update/delete on the three collections:
|
|
"@request.auth.id != \"\""
|
|
users collection: createRule = null (SUPERUSER ONLY — this is what keeps the
|
|
world out), listRule/viewRule = "@request.auth.id != \"\"",
|
|
updateRule = "id = @request.auth.id", deleteRule = null.
|
|
NOTE: rule "" means PUBLIC in PocketBase; null means superuser-only. Do not confuse.
|
|
|
|
## Sync design (SyncEngine)
|
|
Pull: GET /api/collections/{c}/records?filter=(updated>'{cursor}')&sort=updated
|
|
&perPage=200&page=N — paginate to exhaustion. Cursor per collection in
|
|
DataStore, stored as PB UTC string. Include tombstones.
|
|
Push: records where syncState != SYNCED. PENDING_CREATE -> POST (with our id),
|
|
PENDING_UPDATE -> PATCH, PENDING_DELETE -> PATCH {deleted:true}.
|
|
On 404 for update/delete: drop local record. On 409/duplicate id: switch to PATCH.
|
|
Auth: EVERY sync starts with POST /api/collections/users/auth-refresh and stores the
|
|
new token, BEFORE any push or pull. 401/403 (from refresh or any later call)
|
|
-> clear the token only (keep URL + email), stop, report AuthExpired: the UI
|
|
says "sign in again" and routes to setup, pre-filled. Offline -> ordinary
|
|
failure, token kept. WHY: PocketBase does not reject an expired token, it
|
|
treats the request as anonymous, and the rules then 404 on records that
|
|
exist — so the 404 rule above is only safe on a freshly-proven token
|
|
(2026-09-17: five books hard-deleted locally this way). User tokens last
|
|
180 days server-side (migration 1789608448).
|
|
Sign-in clears all pull cursors, so every fresh session reconciles fully.
|
|
Order: push THEN pull (so our writes come back canonical).
|
|
Conflict: last-write-wins on `updated`. Document this in README; do not build
|
|
anything cleverer.
|
|
Covers: on save, app downloads cover from metadata source and multipart-uploads it
|
|
to the book's `cover` file field, so covers survive upstream URL rot. If offline,
|
|
store localCoverPath and upload on next sync.
|
|
Trigger: app start, manual pull-to-refresh, WorkManager periodic (~6h, network-constrained).
|
|
Never let sync failure surface as a crash or a blocking dialog — a quiet status line only.
|
|
|
|
## Book metadata lookup
|
|
Primary Open Library EDITIONS API: https://openlibrary.org/isbn/{isbn}.json (302 -> the edition record)
|
|
Fallback Google Books: https://www.googleapis.com/books/v1/volumes?q=isbn:{isbn} (keyed)
|
|
Merge: prefer whichever has a title; fill blanks from the other.
|
|
|
|
DO NOT go back to `/api/books?bibkeys=...&jscmd=data`. On 2026-09-20 it was found
|
|
answering 404 to EVERY bibkey form for EVERY ISBN — including books Open Library
|
|
demonstrably still holds — while /isbn/, /search.json, /api/volumes/brief and
|
|
covers.openlibrary.org all served normally. OL's own docs call it the "Legacy Books
|
|
API" and say it "may be phased out in the future", and it is absent from their API
|
|
index, so treat it as retired. /api/volumes/brief (the "Legacy Partner API") is a
|
|
near drop-in for the old response shape and is ALSO legacy — that is why it was not
|
|
chosen. See docs/METADATA-SOURCES.md.
|
|
|
|
The Editions API costs extra requests, by design:
|
|
- `authors` are REFERENCES (`{"key": "/authors/OL26320A"}`), not names; each needs
|
|
a GET of /authors/{id}.json. `AuthorNameCache` holds resolved names for the
|
|
process lifetime — books by one author are scanned in runs off one shelf.
|
|
- an edition record may carry NO authors at all (real: 9780898707168), in which
|
|
case they are on the work; fall back to /works/{id}.json. Losing this fallback
|
|
silently drops the author for such books.
|
|
- author/work requests are BEST-EFFORT: the edition request already succeeded, so
|
|
a failure there degrades the record, it must never turn Found into Unavailable.
|
|
|
|
Cover: OL `covers: [id]` -> https://covers.openlibrary.org/b/id/{id}-L.jpg
|
|
else GB imageLinks (force https, **zoom=1**, add `w=400`, strip `edge=curl`)
|
|
Cover URL comes from a source that REPORTS one. Never synthesize the by-ISBN cover
|
|
URL as if it were evidence: for an edition with no art that endpoint returns 200 +
|
|
a 43-byte 1x1 transparent GIF, which loads "successfully" and paints nothing. As a
|
|
last resort it may be used only with `?default=false`, which makes a miss a 404.
|
|
OL's `covers` array uses -1 as a "no cover here" sentinel; take the first POSITIVE id.
|
|
|
|
DO NOT force `zoom=2` on a Google Books cover (this spec said to, and was wrong).
|
|
For a volume Google holds no full preview of, zoom=2 is not a valid rendition and
|
|
Google answers 200 with a grey "image not available" PNG rather than a 404 — which
|
|
loads successfully, so BookCover's placeholder never fires and the cover pipeline
|
|
uploads Google's placeholder to PocketBase as the book's cover. Measured 2026-09-20:
|
|
11 of 18 real volumes did this at zoom=2; 0 of 18 at zoom=1 with a width. zoom=0/3/6
|
|
are placeholders too, so there is no larger-zoom escape.
|
|
|
|
Lookup outcome is THREE-WAY, never a bare null. A source that could not be reached
|
|
must never be reported to the user as a book that does not exist:
|
|
Found(metadata) - at least one source returned a record
|
|
NotFound - EVERY source answered authoritatively and none had it
|
|
Unavailable - no source could be reached (non-2xx, timeout, transport error)
|
|
and none of the reachable ones had it
|
|
On the Editions API a 404 IS authoritative NotFound — unlike the legacy endpoint,
|
|
which reported a miss as 200 with an empty object and so made every non-2xx a
|
|
failure. Every OTHER non-2xx remains Unavailable.
|
|
UI: Found -> the save sheet. NotFound -> manual entry pre-filled with the scanned
|
|
ISBN. Unavailable -> a retry affordance, with manual entry as the escape hatch;
|
|
it must NOT claim the book is unknown.
|
|
|
|
## Barcode scanning
|
|
CameraX Preview + ImageAnalysis -> ML Kit BarcodeScanning (EAN_13, EAN_8, UPC_A).
|
|
Validate ISBN-13 checksum before lookup; ignore non-book barcodes. Debounce repeats.
|
|
A rejected barcode is NOT silent: the camera screen must say a code was read and
|
|
was not a book ISBN, or the user cannot tell a non-book barcode from a dead camera.
|
|
Throttle that message — a non-book barcode sits in frame emitting continuously.
|
|
Continuous mode: after a save, stay on camera for the next book (shelving a box of
|
|
books is the real use case). Show a running "added this session" count.
|
|
Handle: camera permission denial, torch toggle, and a manual-ISBN-entry escape hatch.
|
|
|
|
## Design language — "feels like books"
|
|
Warm paper, dark mahogany, gold + silver metallics. Restrained, not skeuomorphic.
|
|
Light: paper #F5EDE0, paperAlt #EDE3D2, ink #2B211A, inkSoft #5A4A3D,
|
|
mahogany #5C2E23, mahoganyDeep #3E1E17, gold #C0932F, goldSoft #D9B45B,
|
|
silver #9CA3AF, silverSoft #C7CCD1
|
|
Dark: ground #1C1411, surface #241A15, paperText #E8DCC8, mahogany #7A3E2F,
|
|
gold #D9B45B, silver #C7CCD1
|
|
Type: serif display (Literata, OFL, bundle the TTF) for titles/headers;
|
|
system sans for body/UI. Generous line-height.
|
|
Motifs: subtle spine/edge treatments, thin gold hairline rules, gentle paper-grain
|
|
on large surfaces. Covers are the hero — let them carry the color.
|
|
Both light and dark themes required. Dynamic color OFF (it would fight the palette).
|
|
|
|
## Screens
|
|
setup First run: server URL (+ https scheme validation, trailing-slash strip,
|
|
reachability probe), email, password. Clear errors for wrong URL vs bad creds.
|
|
library Cover grid (2-3 col adaptive). Search title/author/ISBN. Filter by
|
|
bookcase/shelf. Sort title/author/added. Empty state invites first scan.
|
|
FAB -> scan. Sync status line.
|
|
detail Big cover, title/subtitle/authors/publisher/year/pages/ISBN, description
|
|
(collapsible), notes (editable), location picker, edit, soft-delete w/ undo.
|
|
scan Camera + reticle; on hit -> bottom sheet w/ fetched book + shelf picker +
|
|
Save / Skip. Duplicate-ISBN warning if already owned.
|
|
locations Bookcases -> shelves tree. CRUD + reorder. Book counts per shelf.
|
|
Tap a shelf -> library filtered to it. "Move books" bulk action.
|
|
settings Server, account, sign out, manual sync + last-sync time, book/cover counts.
|
|
Session expired -> "sign in again" + Sign in button instead of Sync now.
|
|
|
|
## Quality bar
|
|
- No emulator on this box (no KVM). Verify via: `./gradlew assembleDebug`,
|
|
JVM unit tests, and Paparazzi screenshot rendering.
|
|
- Unit-test the real logic: ISBN checksum, metadata merge, sync conflict resolution,
|
|
DAO queries (Robolectric). Do not write assertion-free tests.
|
|
- App must compile and run with NO server configured (setup screen) and must not
|
|
crash when the server is unreachable.
|