docs: first on-device test results and metadata-source research

Records the coil3 StateFlow trap (a Kotlin warning, not an error, that
silently blanked every book cover) so a future session greps for it, and
the two Open Library cover defects behind it.

METADATA-SOURCES.md answers the user's research question about
alternative lookup sources: measurements over a 60-ISBN sample drawn
from a third-party catalogue, the options with costs, and a
recommendation to fix diagnosis before buying coverage. Nothing there is
implemented — the user asked to be consulted first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDcPottghJXEvfYKqFM7zf
This commit is contained in:
Sprite
2026-09-09 10:13:36 +00:00
co-authored by claude
parent 356f639cdd
commit bd24ecbafd
2 changed files with 215 additions and 0 deletions
+166
View File
@@ -0,0 +1,166 @@
# Book metadata lookup — why scans miss, and what else we could ask
Research note, 2026-09-09. Written in response to "out of the 3 barcodes I've
scanned, only 1 has been discovered properly."
**Nothing in here has been implemented.** SPEC's two-source design (Open Library
primary, Google Books fallback) is unchanged. This is the evidence for deciding
whether to change it.
---
## Short version
Three separate defects were making lookups *look* far worse than the underlying
data actually is, and all three are now fixed (commit `356f639`). They are not
the same problem as "this book isn't in the database":
1. A cover that loaded fine still rendered as nothing, so a **successful** lookup
looked like a failed one. That alone could account for the book you did find
appearing broken.
2. Open Library's cover URL was synthesized for every book whether or not art
existed, and a missing cover comes back as a **200 with a 43-byte 1×1
transparent GIF** — a "successful" load that paints nothing.
3. Because that synthesized URL was never blank, the merge rule could never fall
through to Google Books' thumbnail. The documented fallback was dead code for
covers.
What is left is a real coverage question, and there the measurements point at one
thing above all others: **the Google Books fallback is probably not answering at
all.** Every keyless request from this machine returned HTTP 429, and the app
turns any non-200 into `null`, which the UI presents as "No match found — enter
the details by hand." A rate-limited lookup and a book that genuinely exists
nowhere are, right now, indistinguishable to both you and me.
My recommendation is to fix the diagnosis before buying more data. Details in
[Recommendation](#recommendation).
---
## What I measured
**Sample.** 60 ISBNs drawn from the Harvard Library catalog — deliberately a
third party, so the sample doesn't presuppose the answer by coming from one of
the two sources under test. Ten publishers, weighted toward the small Catholic
and homeschool presses that a MODG family's shelf actually carries (Ignatius,
TAN, Sophia Institute, Bethlehem Books, Baronius) alongside mainstream trade
(Penguin, Random House, Scholastic, Crossway, Loyola).
**Method.** Direct HTTP against each API, one ISBN at a time, 1.2 s apart. A
source "hits" only if it returns a usable title.
### Results
| Source | Hit | Miss | Error | Hit rate |
|---|---|---|---|---|
| Open Library Books API (what the app calls today) | 53 | 4 | 3 | **88%** |
| Google Books, keyless (the app's fallback) | 0 | 0 | **60 × HTTP 429** | **0%** |
| Harvard LibraryCloud | 56 | 4 | 0 | 93% * |
| Open Library cover art exists for the ISBN | 47 | 13 | 0 | 78% |
\* Harvard is where the sample came from, so its number is inflated by
construction. It's here to show the API works and answers by ISBN-13, not as a
fair comparison.
Two further observations from the same runs:
- **Concurrency is punished.** The same 60 ISBNs run six-at-a-time dropped Open
Library from 88% to 70%, entirely through transport errors. The app makes one
request per scan, so this doesn't bite in normal use — but it does mean
"Open Library missed" in a log is not proof the book is absent. By the end of
this research my own IP was refused outright for a while.
- **17% of successful Open Library lookups have no cover art at all** (9 of 53).
Even with everything working, roughly one book in six will legitimately show
the placeholder. That is a data fact, not a bug, and it's worth knowing before
you read a placeholder as a failure.
### What this does not tell us
Worth saying plainly, because it bounds how much weight the numbers carry:
- n = 60, and the sample comes from a research library. It under-represents
recent mass-market paperbacks, reprints and print-on-demand editions — which
is exactly where Open Library is thinnest. Real shelf coverage is probably
*below* 88%.
- Every request came from a datacenter IP. The Google Books 429 may partly be
this host sharing a quota pool with other tenants; **your phone, on a
residential or mobile IP, may well get answers.** That's precisely why the
app needs to be able to tell us which it got.
- I don't know which three ISBNs you scanned. If you still have the books to
hand, those three numbers are worth more than another 60 sampled ones.
---
## The options
### A. Give Google Books an API key
Free, 1,000 requests/day, no billing account required. Turns the fallback from
"silently 429" into a working source. Roughly a dozen lines: a key in
`local.properties``BuildConfig``&key=` on the query.
The key ships inside the APK and can be extracted, so restrict it to the Books
API in the Google Cloud console. At our volume, someone stealing it costs us
nothing but the quota.
**Effort: hours. Cost: free. Likely the single biggest win.**
### B. Tell the difference between "not found" and "couldn't ask"
Both clients collapse every non-200, timeout and parse failure into `null`, and
`MetadataRepository` collapses that into "no match", and the UI writes "No match
found." A book that's offline, rate-limited, or hit a 500 is reported to you as
a book that does not exist.
Distinguishing these gets you a retry button instead of a manual-entry form, and
gets me a real answer next time you say "it missed."
**Effort: half a day. Cost: free. Do this regardless of what else we choose.**
### C. Retry with backoff
One retry on 429/5xx, a couple of seconds apart. Standing at a bookshelf, a
two-second retry is invisible; a manual-entry form is not.
**Effort: an hour. Cost: free.**
### D. Add a third free source
Only worth doing after AC, when we can see what's actually still missing.
Ranked by what I'd try first:
| Source | Key? | Notes |
|---|---|---|
| **Open Library `search.json`** | No | Searches the whole OL index rather than the edition table the Books API reads. Cheapest possible fallback — same service, one more request, no new failure modes. |
| **Harvard LibraryCloud** | No | Free, no registration, answers by ISBN-13, verified working. Strong on older, scholarly and small-press books — the shape of gap we'd expect. Returns MODS; no cover art, and no ISBN-13 for pre-EAN books unless we convert. |
| **Library of Congress** | No | The SRU endpoint (port 210) is blocked from here; the `loc.gov` JSON API responds. Excellent for US imprints. Needs more probing before I'd commit. |
| **K10plus SRU** | No | Free German-led union catalogue, large and international. Cataloguing conventions differ enough that merging would need care. |
Dead ends, so nobody re-investigates them: **OCLC Classify** (retired 2021),
**Goodreads API** (retired 2020), **Amazon Product Advertising API** (requires an
affiliate account with qualifying sales), **WorldCat Search** (requires OCLC
membership — institutional pricing).
### E. Pay for ISBNdb
~$1550/month depending on tier. Genuinely better coverage than anything free,
including cover art, and a single clean API. It is also a subscription for a
two-person home library, and I'd want proof that AD leave a real gap before
recommending it.
**Effort: hours. Cost: $180600/year.**
---
## Recommendation
Do **A + B + C** together — they're cheap, they're independent of any decision
about new sources, and between them they cover the most likely explanation for
1-in-3. Then rescan the same books and let the app tell us what it actually got.
If a real gap survives that, add **Open Library `search.json`** (D) as an
in-family fallback before reaching for a new organisation's API, and Harvard
after that.
I'd hold off on **E** entirely until we have numbers from your own shelf. Paying
for coverage we might already have would be the wrong order.
One thing worth deciding separately: 17% of books legitimately have no cover art
anywhere. The placeholder now looks deliberate rather than broken, but if you
want covers on everything, that's a different feature — photograph the book,
store it as the cover — and not a metadata-source problem at all.