jfmonty2/bookshelf
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

The two books that failed are in Open Library after all

Browse Source 
The user supplied the failing ISBNs: 9781883937386 (The Hittite Warrior)
and 9781883937676 (Shadow Hawk), both Bethlehem Books. Both resolve
against the source the app already queries — title, author, publisher,
page count, cover art — and both real responses parse correctly through
the app's own OpenLibraryClient. Captured as fixtures with a regression
test, because these are the specific books that motivated the research.

That kills the coverage hypothesis for these two and demotes Harvard,
which holds neither: its 93% in the sample table is inflated by
construction (the sample was drawn from Harvard) and misleading in
exactly the direction that matters — it is a research library and does
not carry small-press children's historical fiction.

The failure is upstream of the metadata sources. Documented the three
candidates; the leading one is that the barcode never decoded into a
valid ISBN-13, which ScanCodeFilter drops silently with no UI feedback
whatsoever. Revised the recommendation accordingly: make the app say
what happened before adding any source.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDcPottghJXEvfYKqFM7zf
This commit is contained in:
Sprite
2026-09-09 10:36:33 +00:00
co-authored by claude
parent bd24ecbafd
commit 1ba22a9f36
4 changed files with 94 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/METADATA-SOURCES.md
+60 -9
View File
@@ -7,6 +7,17 @@ scanned, only 1 has been discovered properly."
primary, Google Books fallback) is unchanged. This is the evidence for deciding
whether to change it.
> **Update, same day — the coverage hypothesis is dead.** The user supplied the
> two ISBNs that failed: 9781883937386 (*The Hittite Warrior*) and 9781883937676
> (*Shadow Hawk*), both Bethlehem Books. **Both are fully present in Open
> Library** — title, author, publisher, page count and cover art — and the app's
> own parser handles both real responses correctly (regression test:
> `OpenLibraryClientTest.parses the real responses for the two books the app
> failed to identify`). Whatever went wrong on the phone was upstream of the
> metadata sources. Everything below still holds as background, but do not act
> on "add a third source" until we know why a book Open Library *has* did not
> reach the lookup. See [What actually failed](#what-actually-failed).
---
## Short version
@@ -128,7 +139,7 @@ Ranked by what I'd try first:
| Source | Key? | Notes |
|---|---|---|
| **Open Library `search.json`** | No | Searches the whole OL index rather than the edition table the Books API reads. Cheapest possible fallback — same service, one more request, no new failure modes. |
| **Harvard LibraryCloud** | No | Free, no registration, answers by ISBN-13, verified working. Strong on older, scholarly and small-press books — the shape of gap we'd expect. Returns MODS; no cover art, and no ISBN-13 for pre-EAN books unless we convert. |
| ~~**Harvard LibraryCloud**~~ | No | **Demoted.** Free, no registration, answers by ISBN-13 (`?identifier=<isbn>`), verified working — but it returned `numFound: 0` for *both* of the user's real failing books. It is a research library: strong on scholarly and older material, and it simply does not hold small-press children's historical fiction. The 93% in the table above is inflated by construction (the sample came from Harvard) and is misleading in exactly the direction that matters. Not a fit for this shelf. |
| **Library of Congress** | No | The SRU endpoint (port 210) is blocked from here; the `loc.gov` JSON API responds. Excellent for US imprints. Needs more probing before I'd commit. |
| **K10plus SRU** | No | Free German-led union catalogue, large and international. Cataloguing conventions differ enough that merging would need care. |
@@ -147,18 +158,58 @@ recommending it.
---
## What actually failed
Both failing ISBNs resolve cleanly against the source the app already uses:
| Check | 9781883937386 | 9781883937676 |
|---|---|---|
| Open Library Books API | *Hittite warrior*, Joanne S. Williamson | *Shadow hawk*, Andre Norton |
| Cover art | yes | yes |
| App's own parser (unit test) | parses | parses |
| ISBN-13 checksum | valid | valid |
| Harvard | not held | not held |
So the request either never went out, or went out and failed in a way the app
reported as "not found". Candidates, in the order I'd chase them:
1. **The barcode never decoded into a valid ISBN-13.** `ScanCodeFilter` drops
anything that fails the checksum, silently and with no UI feedback at all —
no sheet, no message, nothing. A book whose barcode carries a price add-on,
or is worn, or is a UPC-A rather than a Bookland EAN, looks to the user
exactly like a camera that isn't working. This is my leading theory, and it
fits "wasn't able to scan" better than "wasn't found".
2. **The HTTP request failed.** Both clients turn every non-200, timeout and
parse error into `null`, which reaches the user as "No match found". The
metadata `OkHttpClient` is constructed with no call timeout, so a stalled
connection hangs on default socket timeouts.
3. **The lookup ran and the sheet was dismissed before it landed.** While
Loading, the sheet passes an empty `onDismissRequest`, so it can't be
swiped away — but `onScanned` early-returns whenever a sheet is already
showing, so a stuck sheet blocks every subsequent scan.
Nothing here is a data-source problem. Note the sting in (1) and (2): both
failure modes are invisible or actively misleading, which is why three scans
produced no usable diagnosis.
## Recommendation
Do **A + B + C** together — they're cheap, they're independent of any decision
about new sources, and between them they cover the most likely explanation for
1-in-3. Then rescan the same books and let the app tell us what it actually got.
**Revised after the two real ISBNs came in.** Adding sources is now the *wrong*
next move: the books that failed are already in the source we query.
If a real gap survives that, add **Open Library `search.json`** (D) as an
in-family fallback before reaching for a new organisation's API, and Harvard
after that.
Do **B** first and on its own — make the app say what happened. A scan that
decodes nothing should say so on the camera screen; a lookup that fails should
offer retry, not a manual-entry form captioned "No match found". Add a call
timeout while in there.
I'd hold off on **E** entirely until we have numbers from your own shelf. Paying
for coverage we might already have would be the wrong order.
Then rescan those two books. The app will tell us which of the three candidates
above it is, and that determines everything after it. **A** (the free Google
Books key) and **C** (retry/backoff) are still worth doing — cheap, and the 429
result is real — but they are no longer the leading theory.
**D** and **E** are on hold. Harvard specifically is off the list for this
shelf. Paying ISBNdb for coverage we demonstrably already have would be the
wrong order.
One thing worth deciding separately: 17% of books legitimately have no cover art
anywhere. The placeholder now looks deliberate rather than broken, but if you