Move Open Library lookup to the Editions API; stop asking Google Books for its placeholder
Two user-reported shelf-testing symptoms, both a third party answering
misleadingly and the app believing it.
1. Open Library's /api/books?bibkeys=... now 404s for EVERY ISBN, including
books OL demonstrably still holds, with OL's own x-ol-stats header on the
response while /isbn/, /search.json, /api/volumes/brief and covers.* all
serve normally. OL's docs call it the "Legacy Books API" that "may be phased
out" and it is gone from their API index, so this reads as a retirement
rather than an outage. The app had been effectively single-sourced on Google
Books since it broke: every failure the user saw was a book GB lacks.
Lookup now uses /isbn/{isbn}.json — current, non-legacy, edition-level, and
the only option of the three that carries a description. /api/volumes/brief
is a near drop-in for the old response shape and was rejected precisely
because it is also legacy.
Its costs, all handled: authors are references, so AuthorNameCache resolves
and caches them for the process lifetime (books by one author get scanned in
runs off one shelf); an edition may carry NO authors, in which case they live
on the work — 9780898707168 on the user's own shelf is exactly this, so
without the work fallback the move would have silently dropped its author;
author/work requests are best-effort and can only degrade a record, never
turn Found into Unavailable.
404 on this endpoint is authoritative NotFound. The legacy endpoint reported
a miss as 200 with an empty object, which is why every non-2xx there was a
failure. Every other non-2xx still is.
2. Google Books answers zoom=2 with a grey "image not available" PNG at HTTP
200 — not a 404 — for any volume it holds no full preview of. Coil loads it
as a success, so BookCover's placeholder never fires and the cover pipeline
uploads Google's placeholder to PocketBase as the book's cover. Measured over
18 real volumes: 11 placeholders at zoom=2, 0 at zoom=1&w=400. zoom=0/3/6 are
placeholders too. normalizeCoverUrl now pins zoom=1, adds w=400 and strips
edge=curl.
SPEC.md's "Book metadata lookup" is rewritten with both rules and the evidence
for them — it was the source of the zoom=2 instruction, and would otherwise be
the reason someone restores it.
Also fixed, because it blocked verification: LibraryViewModelTest never cleared
the view models it built, and LibraryViewModel's eleven WhileSubscribed(5_000)
flows kept running five seconds into later tests, racing resetMain(). It now
cancels each viewModelScope in tearDown.
NOT fixed, reported instead: AddBookViewModel.performSave's in-flight guard is a
check-then-act and two coroutines can both pass it. Unrelated to this change
(that VM has no metadata dependency) and out of scope. See HAZARD #13.
Verified: assembleDebug exit 0; testDebugUnitTest --rerun-tasks 378 tests,
2 skipped, 0 failures (was 355); verifyPaparazziDebug exit 0, no pixels moved;
0 "always 'false'" warnings; no build files touched. LIVE_METADATA=1 live test
ran (not skipped): 9/9 Found with cover art, with the GB key absent, so Open
Library alone answered through the new endpoint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4695731f91
commit
ab83287b5d
25 files changed
+972
-193
No files matched your search
@@ -0,0 +1,41 @@
|
|||||||
|
package org.modg.bookshelf.data.metadata
|
||||||
|
|
||||||
|
import java.util.concurrent.ConcurrentHashMap
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Process-lifetime cache of Open Library author key -> author name.
|
||||||
|
*
|
||||||
|
* Open Library's Editions API gives authors as references (`/authors/OL26320A`),
|
||||||
|
* so every lookup that would otherwise report an author costs an extra request.
|
||||||
|
* The user's access pattern makes that nearly free to avoid: books by one author
|
||||||
|
* arrive in runs, because they sit next to each other on the shelf and get scanned
|
||||||
|
* one after another. The second Tolkien of the evening resolves from memory.
|
||||||
|
*
|
||||||
|
* Deliberately unbounded. Entries are a short key and a short name — a few dozen
|
||||||
|
* bytes each — and the population is the number of distinct authors in one home
|
||||||
|
* library, so the ceiling is hundreds of entries, not a leak. Given that, an
|
||||||
|
* eviction policy would be more code and more risk than the thing it prevents.
|
||||||
|
*
|
||||||
|
* Only successful resolutions are stored. A failed author fetch is left uncached on
|
||||||
|
* purpose: it is usually a transient network failure, and caching it would make one
|
||||||
|
* bad moment cost that author their name for the rest of the session.
|
||||||
|
*
|
||||||
|
* [ConcurrentHashMap] because [MetadataRepository] runs both sources concurrently
|
||||||
|
* and a single lookup may resolve several authors; the worst a race can do is fetch
|
||||||
|
* the same author twice and store the same answer twice.
|
||||||
|
*/
|
||||||
|
class AuthorNameCache {
|
||||||
|
|
||||||
|
private val names = ConcurrentHashMap<String, String>()
|
||||||
|
|
||||||
|
/** The cached name for an author key such as `/authors/OL26320A`, or null. */
|
||||||
|
operator fun get(authorKey: String): String? = names[authorKey]
|
||||||
|
|
||||||
|
/** Records a successfully resolved name. Blank names are ignored, never stored. */
|
||||||
|
fun put(authorKey: String, name: String) {
|
||||||
|
if (name.isNotBlank()) names[authorKey] = name
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Entry count — for tests asserting that a second lookup did not re-fetch. */
|
||||||
|
val size: Int get() = names.size
|
||||||
|
}
|
||||||
@@ -38,7 +38,7 @@ data class GoogleBooksIndustryIdentifierDto(
|
|||||||
val identifier: String? = null,
|
val identifier: String? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
/** Maps the GB DTO to the source-agnostic [BookMetadata], forcing https + zoom=2 on the cover per SPEC. */
|
/** Maps the GB DTO to the source-agnostic [BookMetadata], normalizing the cover per [normalizeCoverUrl]. */
|
||||||
fun GoogleBooksVolumeInfoDto.toBookMetadata(): BookMetadata = BookMetadata(
|
fun GoogleBooksVolumeInfoDto.toBookMetadata(): BookMetadata = BookMetadata(
|
||||||
isbn13 = industryIdentifiers.firstOrNull { it.type == "ISBN_13" }?.identifier,
|
isbn13 = industryIdentifiers.firstOrNull { it.type == "ISBN_13" }?.identifier,
|
||||||
isbn10 = industryIdentifiers.firstOrNull { it.type == "ISBN_10" }?.identifier,
|
isbn10 = industryIdentifiers.firstOrNull { it.type == "ISBN_10" }?.identifier,
|
||||||
@@ -52,13 +52,48 @@ fun GoogleBooksVolumeInfoDto.toBookMetadata(): BookMetadata = BookMetadata(
|
|||||||
coverUrl = normalizeCoverUrl(imageLinks?.thumbnail ?: imageLinks?.smallThumbnail),
|
coverUrl = normalizeCoverUrl(imageLinks?.thumbnail ?: imageLinks?.smallThumbnail),
|
||||||
)
|
)
|
||||||
|
|
||||||
/** Forces https + zoom=2, per SPEC's cover chain. Internal (not private) so [GoogleBooksSearchClient] reuses it too. */
|
/**
|
||||||
|
* Forces https, pins `zoom=1`, requests a wider rendition with `w`, and drops
|
||||||
|
* `edge=curl`. Internal (not private) so [GoogleBooksSearchClient] reuses it too.
|
||||||
|
*
|
||||||
|
* **Do not restore `zoom=2`.** SPEC.md said to force it and that was wrong: for any
|
||||||
|
* volume Google holds no full preview of — the metadata-only `...AAAACAAJ` records,
|
||||||
|
* which is most small-press and older material — `zoom=2` is not a valid rendition,
|
||||||
|
* and Google does not 404. It answers **200 with a placeholder image**: grey italic
|
||||||
|
* "image not available" on white. Coil loads it successfully, so nothing downstream
|
||||||
|
* can tell it from a cover, and [org.modg.bookshelf.ui.components.BookCover]'s own
|
||||||
|
* placeholder never gets a chance to fire. Worse, the cover pipeline then uploads it
|
||||||
|
* to PocketBase as the book's cover, where it sticks.
|
||||||
|
*
|
||||||
|
* Measured 2026-09-20 over 18 volumes with imageLinks: `zoom=2` returned that
|
||||||
|
* placeholder for **11 of 18** (byte-identical, 15,567 bytes), while `zoom=1&w=400`
|
||||||
|
* returned 18 distinct real covers. `zoom=0`, `3` and `6` are placeholders too —
|
||||||
|
* the same artwork at 575x750 and 1280x1670 — so there is no "just use a bigger
|
||||||
|
* zoom" fix, and detecting the placeholder by hash would mean chasing renditions.
|
||||||
|
* Asking for a URL Google will actually serve is the fix.
|
||||||
|
*
|
||||||
|
* `w` is what buys resolution instead: Google honours it up to the source scan's
|
||||||
|
* native width, so a volume with real art yields 400px and one with only a small
|
||||||
|
* scan yields its own 300px rather than a placeholder. `edge=curl` draws a fake
|
||||||
|
* page-curl onto the right edge of the image, which fights a design language whose
|
||||||
|
* covers are meant to be the hero.
|
||||||
|
*/
|
||||||
internal fun normalizeCoverUrl(raw: String?): String? {
|
internal fun normalizeCoverUrl(raw: String?): String? {
|
||||||
if (raw.isNullOrBlank()) return null
|
if (raw.isNullOrBlank()) return null
|
||||||
val https = raw.replaceFirst("http://", "https://")
|
val https = raw.replaceFirst("http://", "https://").replace("&edge=curl", "")
|
||||||
return if ("zoom=" in https) https.replace(Regex("zoom=\\d+"), "zoom=2") else "$https&zoom=2"
|
val zoomed = if ("zoom=" in https) https.replace(Regex("zoom=\\d+"), "zoom=1") else "$https&zoom=1"
|
||||||
|
// Anchored to a parameter boundary so a URL that merely contains "w=" inside
|
||||||
|
// another parameter's value doesn't suppress the width we need.
|
||||||
|
return if (Regex("[?&]w=").containsMatchIn(zoomed)) zoomed else "$zoomed&w=$GoogleBooksCoverWidth"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Requested cover width in pixels. 400 comfortably exceeds the widest slot the app
|
||||||
|
* draws (a detail-screen cover on a large phone) without asking for a rendition
|
||||||
|
* Google has to synthesize.
|
||||||
|
*/
|
||||||
|
internal const val GoogleBooksCoverWidth = 400
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Maps one volume to a [SearchHit] for the library search screen. [isbn13s] collects
|
* Maps one volume to a [SearchHit] for the library search screen. [isbn13s] collects
|
||||||
* every industry identifier this volume reports, normalized to ISBN-13 regardless of
|
* every industry identifier this volume reports, normalized to ISBN-13 regardless of
|
||||||
|
|||||||
@@ -6,23 +6,31 @@ import kotlinx.coroutines.Dispatchers
|
|||||||
import kotlinx.coroutines.withContext
|
import kotlinx.coroutines.withContext
|
||||||
import kotlinx.serialization.SerializationException
|
import kotlinx.serialization.SerializationException
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
import kotlinx.serialization.json.decodeFromJsonElement
|
|
||||||
import kotlinx.serialization.json.jsonObject
|
|
||||||
import okhttp3.OkHttpClient
|
import okhttp3.OkHttpClient
|
||||||
import okhttp3.Request
|
import okhttp3.Request
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Open Library lookup — SPEC.md "Book metadata lookup" primary source.
|
* Open Library lookup — SPEC.md "Book metadata lookup" primary source.
|
||||||
*
|
*
|
||||||
* Genuinely never throws (verified 2026-09-12): every outcome, including
|
* Uses the **Editions API** (`/isbn/{isbn}.json`). The previous endpoint,
|
||||||
* transport failure, comes back as a [SourceResult] rather than a swallowed
|
* `/api/books?bibkeys=...&jscmd=data`, began answering 404 to everything; see
|
||||||
* null or an exception reaching the caller. See [GoogleBooksClient]'s KDoc
|
* [OpenLibraryEditionDto]'s KDoc and docs/METADATA-SOURCES.md for the evidence that
|
||||||
* for why that claim is dated — the same [FailureKind.UNEXPECTED] reasoning
|
* this was a retirement rather than an outage.
|
||||||
* applies here.
|
*
|
||||||
|
* The move changed one thing that matters beyond the URL: **404 now means the book
|
||||||
|
* genuinely is not here.** The legacy endpoint reported a miss as 200 with an empty
|
||||||
|
* object, so every non-2xx was rightly a [SourceResult.Failed]. This endpoint has no
|
||||||
|
* such body to return and 404s instead, so a 404 is an authoritative
|
||||||
|
* [SourceResult.NotFound] — see [classifyEdition], which is the single place that
|
||||||
|
* distinction lives.
|
||||||
|
*
|
||||||
|
* Genuinely never throws: every outcome, including transport failure, comes back as
|
||||||
|
* a [SourceResult] rather than a swallowed null or an exception reaching the caller.
|
||||||
*/
|
*/
|
||||||
class OpenLibraryClient(
|
class OpenLibraryClient(
|
||||||
private val httpClient: OkHttpClient,
|
private val httpClient: OkHttpClient,
|
||||||
json: Json,
|
json: Json,
|
||||||
|
private val authorNames: AuthorNameCache = AuthorNameCache(),
|
||||||
) {
|
) {
|
||||||
// Real responses carry fields this DTO doesn't model; never let an unknown key throw.
|
// Real responses carry fields this DTO doesn't model; never let an unknown key throw.
|
||||||
private val json = Json(from = json) { ignoreUnknownKeys = true }
|
private val json = Json(from = json) { ignoreUnknownKeys = true }
|
||||||
@@ -31,6 +39,11 @@ class OpenLibraryClient(
|
|||||||
* Retries transient failures per [RetryPolicy]. Measured against the live API,
|
* Retries transient failures per [RetryPolicy]. Measured against the live API,
|
||||||
* 13% of requests fail at the TLS stage in well under a second while successful
|
* 13% of requests fail at the TLS stage in well under a second while successful
|
||||||
* ones take seconds — so a retry is nearly free and removes most of that 13%.
|
* ones take seconds — so a retry is nearly free and removes most of that 13%.
|
||||||
|
*
|
||||||
|
* Only the **edition** request is retried. Author and work requests happen after
|
||||||
|
* it has already succeeded and are best-effort by design (see [enrich]), so a
|
||||||
|
* retry can never re-run them: a failure there degrades the record, it does not
|
||||||
|
* fail the lookup.
|
||||||
*/
|
*/
|
||||||
suspend fun lookup(isbn13: String): SourceResult =
|
suspend fun lookup(isbn13: String): SourceResult =
|
||||||
withContext(Dispatchers.IO) { withRetry { fetch(isbn13) } }
|
withContext(Dispatchers.IO) { withRetry { fetch(isbn13) } }
|
||||||
@@ -40,11 +53,10 @@ class OpenLibraryClient(
|
|||||||
withContext(Dispatchers.IO) { fetch(isbn13) }
|
withContext(Dispatchers.IO) { fetch(isbn13) }
|
||||||
|
|
||||||
private fun fetch(isbn13: String): SourceResult = try {
|
private fun fetch(isbn13: String): SourceResult = try {
|
||||||
val request = Request.Builder()
|
val (code, body) = get("https://openlibrary.org/isbn/$isbn13.json")
|
||||||
.url("https://openlibrary.org/api/books?bibkeys=ISBN:$isbn13&format=json&jscmd=data")
|
when (val outcome = classifyEdition(code, body)) {
|
||||||
.build()
|
is EditionOutcome.Terminal -> outcome.result
|
||||||
httpClient.newCall(request).execute().use { response ->
|
is EditionOutcome.Parsed -> SourceResult.Found(enrich(outcome.dto, isbn13))
|
||||||
classify(response.code, response.body.string(), isbn13)
|
|
||||||
}
|
}
|
||||||
} catch (e: CancellationException) {
|
} catch (e: CancellationException) {
|
||||||
// See GoogleBooksClient.fetch's identical arm: cancellation is normal
|
// See GoogleBooksClient.fetch's identical arm: cancellation is normal
|
||||||
@@ -60,26 +72,117 @@ class OpenLibraryClient(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Package-visible pure function — no socket involved — so it's exhaustively
|
* Turns a parsed edition into a complete [BookMetadata], resolving author
|
||||||
* unit-testable offline (2xx-with-record, 2xx-without-record, 404, 429, 500,
|
* references to names.
|
||||||
* malformed body). [parseResponse] is defined in terms of this so the two
|
*
|
||||||
* can never disagree about what a body means.
|
* Best-effort throughout: the edition request already succeeded, so a failure
|
||||||
|
* here must degrade the record rather than turn a found book into a failed
|
||||||
|
* lookup. An author we cannot name is simply omitted. The one thing that is NOT
|
||||||
|
* best-effort is cancellation, which propagates — the user dismissing the scan
|
||||||
|
* sheet mid-lookup is normal control flow.
|
||||||
*/
|
*/
|
||||||
internal fun classify(httpCode: Int, body: String?, isbn13: String): SourceResult {
|
private fun enrich(dto: OpenLibraryEditionDto, isbn13: String): BookMetadata {
|
||||||
if (httpCode !in 200..299) return SourceResult.fromHttpCode(httpCode)
|
val workKey = dto.works.firstOrNull()?.key?.takeIf { it.isNotBlank() }
|
||||||
if (body.isNullOrBlank()) return SourceResult.Failed("empty body", FailureKind.MALFORMED)
|
// Fetched only when the edition itself names no authors, so the common case
|
||||||
|
// costs nothing; when we do fetch it, its description is free to take.
|
||||||
|
val work = if (dto.authors.isEmpty() && workKey != null) fetchWork(workKey) else null
|
||||||
|
val authorKeys = authorKeysFor(dto, work)
|
||||||
|
return dto.toBookMetadata(
|
||||||
|
lookupIsbn13 = isbn13,
|
||||||
|
authors = authorKeys.mapNotNull { authorName(it) },
|
||||||
|
description = work?.description.text(),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Author references for an edition: its own if it has any, otherwise the work's.
|
||||||
|
*
|
||||||
|
* The fallback is not hypothetical. ISBN 9780898707168 — *The Harp and Laurel
|
||||||
|
* Wreath*, on the user's shelf — has no `authors` on its edition record at all,
|
||||||
|
* only on its work. The legacy endpoint resolved that for us and returned
|
||||||
|
* "Laura M. Berquist"; without this fallback the move to the Editions API would
|
||||||
|
* have silently dropped the author for books like it.
|
||||||
|
*/
|
||||||
|
internal fun authorKeysFor(dto: OpenLibraryEditionDto, work: OpenLibraryWorkDto?): List<String> {
|
||||||
|
val editionKeys = dto.authors.mapNotNull { it.key?.takeIf(String::isNotBlank) }
|
||||||
|
if (editionKeys.isNotEmpty()) return editionKeys
|
||||||
|
return work?.authors?.mapNotNull { it.author?.key?.takeIf(String::isNotBlank) }.orEmpty()
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolves one author reference to a name, via [AuthorNameCache]. Null if it can't. */
|
||||||
|
private fun authorName(authorKey: String): String? {
|
||||||
|
authorNames[authorKey]?.let { return it }
|
||||||
|
val dto = getAndDecode<OpenLibraryAuthorDto>("https://openlibrary.org$authorKey.json")
|
||||||
|
val name = dto?.name?.takeIf { it.isNotBlank() } ?: return null
|
||||||
|
authorNames.put(authorKey, name)
|
||||||
|
return name
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun fetchWork(workKey: String): OpenLibraryWorkDto? =
|
||||||
|
getAndDecode("https://openlibrary.org$workKey.json")
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A supporting GET whose failure is not the lookup's failure. Swallows everything
|
||||||
|
* except cancellation, which is normal control flow and must propagate.
|
||||||
|
*/
|
||||||
|
private inline fun <reified T> getAndDecode(url: String): T? = try {
|
||||||
|
val (code, body) = get(url)
|
||||||
|
if (code in 200..299 && !body.isNullOrBlank()) json.decodeFromString<T>(body) else null
|
||||||
|
} catch (e: CancellationException) {
|
||||||
|
throw e
|
||||||
|
} catch (e: Throwable) {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun get(url: String): Pair<Int, String?> =
|
||||||
|
httpClient.newCall(Request.Builder().url(url).build()).execute().use { response ->
|
||||||
|
response.code to response.body.string()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Package-visible pure function — no socket involved — so it's exhaustively
|
||||||
|
* unit-testable offline (2xx-with-record, 404, 429, 500, malformed body).
|
||||||
|
*
|
||||||
|
* **404 is [SourceResult.NotFound], not a failure.** On this endpoint a 404 is
|
||||||
|
* Open Library answering authoritatively that it holds no edition with that
|
||||||
|
* ISBN, which is exactly what SPEC.md's NotFound means; its body is an HTML
|
||||||
|
* error page, so nothing tries to parse it. Every other non-2xx remains a
|
||||||
|
* failure, because a source we could not reach must never be reported to the
|
||||||
|
* user as a book that does not exist.
|
||||||
|
*/
|
||||||
|
internal fun classifyEdition(httpCode: Int, body: String?): EditionOutcome {
|
||||||
|
if (httpCode == 404) return EditionOutcome.Terminal(SourceResult.NotFound)
|
||||||
|
if (httpCode !in 200..299) return EditionOutcome.Terminal(SourceResult.fromHttpCode(httpCode))
|
||||||
|
if (body.isNullOrBlank()) {
|
||||||
|
return EditionOutcome.Terminal(SourceResult.Failed("empty body", FailureKind.MALFORMED))
|
||||||
|
}
|
||||||
return try {
|
return try {
|
||||||
val root = json.parseToJsonElement(body).jsonObject
|
EditionOutcome.Parsed(json.decodeFromString<OpenLibraryEditionDto>(body))
|
||||||
val entry = root["ISBN:$isbn13"]?.jsonObject ?: return SourceResult.NotFound
|
|
||||||
SourceResult.Found(json.decodeFromJsonElement<OpenLibraryBookDto>(entry).toBookMetadata(isbn13))
|
|
||||||
} catch (e: SerializationException) {
|
} catch (e: SerializationException) {
|
||||||
SourceResult.Failed("malformed json", FailureKind.MALFORMED)
|
EditionOutcome.Terminal(SourceResult.Failed("malformed json", FailureKind.MALFORMED))
|
||||||
} catch (e: IllegalArgumentException) {
|
} catch (e: IllegalArgumentException) {
|
||||||
SourceResult.Failed("malformed json", FailureKind.MALFORMED)
|
EditionOutcome.Terminal(SourceResult.Failed("malformed json", FailureKind.MALFORMED))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Package-visible for offline fixture tests — parses a raw response body with no network involved. */
|
/**
|
||||||
|
* Status + body -> a [SourceResult], for tests covering the status mapping alone.
|
||||||
|
* Authors stay unresolved here because resolving them needs the network; the full
|
||||||
|
* path including author resolution is covered by the fixture-serving tests.
|
||||||
|
*/
|
||||||
|
internal fun classify(httpCode: Int, body: String?, isbn13: String): SourceResult =
|
||||||
|
when (val outcome = classifyEdition(httpCode, body)) {
|
||||||
|
is EditionOutcome.Terminal -> outcome.result
|
||||||
|
is EditionOutcome.Parsed -> SourceResult.Found(outcome.dto.toBookMetadata(isbn13))
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Package-visible for offline fixture tests — parses a raw edition body, no network. */
|
||||||
internal fun parseResponse(body: String, isbn13: String): BookMetadata? =
|
internal fun parseResponse(body: String, isbn13: String): BookMetadata? =
|
||||||
(classify(200, body, isbn13) as? SourceResult.Found)?.metadata
|
(classify(200, body, isbn13) as? SourceResult.Found)?.metadata
|
||||||
|
|
||||||
|
/** What a status + body amounted to: a record to enrich, or a final answer. */
|
||||||
|
internal sealed interface EditionOutcome {
|
||||||
|
data class Parsed(val dto: OpenLibraryEditionDto) : EditionOutcome
|
||||||
|
data class Terminal(val result: SourceResult) : EditionOutcome
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -2,67 +2,133 @@ package org.modg.bookshelf.data.metadata
|
|||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
import kotlinx.serialization.SerialName
|
||||||
import kotlinx.serialization.Serializable
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.JsonElement
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.JsonPrimitive
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* https://openlibrary.org/api/books?bibkeys=ISBN:{isbn}&format=json&jscmd=data
|
* https://openlibrary.org/isbn/{isbn}.json — Open Library's **Editions API**
|
||||||
* Response is a JSON object keyed by "ISBN:{isbn}"; this DTO models one value.
|
* (documented under "Work & Edition APIs"), which answers 302 -> 200 with a single
|
||||||
* Real responses carry many more fields (excerpts, ebooks, key, url, ...) that
|
* edition record.
|
||||||
* we don't need — the caller's Json must have ignoreUnknownKeys = true.
|
*
|
||||||
|
* Replaced `/api/books?bibkeys=...&jscmd=data` on 2026-09-20. That endpoint had
|
||||||
|
* started answering **404 to every bibkey form, for every ISBN**, including books
|
||||||
|
* Open Library demonstrably still holds, while every other OL endpoint served
|
||||||
|
* normally. Its own documentation calls it the "Legacy Books API" and says "this is
|
||||||
|
* a legacy endpoint and may be phased out in the future", and it is absent from
|
||||||
|
* OL's API index — so this reads as a retirement, not an outage, and waiting for it
|
||||||
|
* to come back was not a plan. See docs/METADATA-SOURCES.md § "The Open Library
|
||||||
|
* endpoint move".
|
||||||
|
*
|
||||||
|
* Real responses carry far more fields than this (classifications, source_records,
|
||||||
|
* lc_classifications, ...); the caller's Json must have ignoreUnknownKeys = true.
|
||||||
*/
|
*/
|
||||||
@Serializable
|
@Serializable
|
||||||
data class OpenLibraryBookDto(
|
data class OpenLibraryEditionDto(
|
||||||
val title: String? = null,
|
val title: String? = null,
|
||||||
val subtitle: String? = null,
|
val subtitle: String? = null,
|
||||||
val authors: List<OpenLibraryAuthorDto> = emptyList(),
|
/**
|
||||||
val publishers: List<OpenLibraryPublisherDto> = emptyList(),
|
* Edition authors are **references, not names** — `[{"key": "/authors/OL26320A"}]`.
|
||||||
|
* Resolving them costs a request each, which is what [AuthorNameCache] exists to
|
||||||
|
* amortize. This list is also legitimately absent on real records (verified: ISBN
|
||||||
|
* 9780898707168, a book on the user's own shelf), in which case the authors live
|
||||||
|
* on the work — see [OpenLibraryWorkDto] and [OpenLibraryClient.authorKeysFor].
|
||||||
|
*/
|
||||||
|
val authors: List<OpenLibraryKeyRefDto> = emptyList(),
|
||||||
|
/** Plain strings on this endpoint, unlike the legacy API's `[{"name": ...}]` objects. */
|
||||||
|
val publishers: List<String> = emptyList(),
|
||||||
|
val works: List<OpenLibraryKeyRefDto> = emptyList(),
|
||||||
@SerialName("publish_date") val publishDate: String? = null,
|
@SerialName("publish_date") val publishDate: String? = null,
|
||||||
@SerialName("number_of_pages") val numberOfPages: Int? = null,
|
@SerialName("number_of_pages") val numberOfPages: Int? = null,
|
||||||
val identifiers: OpenLibraryIdentifiersDto? = null,
|
@SerialName("isbn_10") val isbn10: List<String> = emptyList(),
|
||||||
val cover: OpenLibraryCoverDto? = null,
|
@SerialName("isbn_13") val isbn13: List<String> = emptyList(),
|
||||||
|
/**
|
||||||
|
* Cover IDs for the Covers API. **May contain -1**, Open Library's "no cover here"
|
||||||
|
* sentinel (verified: 9780898707168 -> `[698345, -1]`, 9780140328721 ->
|
||||||
|
* `[15152634, 8739161, -1]`). Taking `.first()` blindly would eventually build
|
||||||
|
* `covers.openlibrary.org/b/id/-1-L.jpg`; [coverUrl] filters to positive ids.
|
||||||
|
*/
|
||||||
|
val covers: List<Int> = emptyList(),
|
||||||
|
/** String on some records, `{"type": ..., "value": ...}` on others — see [text]. */
|
||||||
|
val description: JsonElement? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
/** `{"key": "/authors/OL26320A"}` / `{"key": "/works/OL8997784W"}`. */
|
||||||
|
@Serializable
|
||||||
|
data class OpenLibraryKeyRefDto(val key: String? = null)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* https://openlibrary.org/authors/{id}.json — fetched only to turn an author
|
||||||
|
* reference into a name. Tiny record; [AuthorNameCache] keeps each one for the
|
||||||
|
* process lifetime.
|
||||||
|
*/
|
||||||
@Serializable
|
@Serializable
|
||||||
data class OpenLibraryAuthorDto(val name: String? = null)
|
data class OpenLibraryAuthorDto(val name: String? = null)
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class OpenLibraryPublisherDto(val name: String? = null)
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Present only when Open Library actually holds cover art for the edition, and so
|
* https://openlibrary.org/works/{id}.json — fetched only when the edition record
|
||||||
* the only trustworthy "has a cover" signal from this API. A synthesized by-ISBN
|
* carries no authors of its own. Work authors nest one level deeper than edition
|
||||||
* covers.openlibrary.org URL is NOT evidence of one: for an edition with no art it
|
* authors: `[{"author": {"key": "/authors/OL335577A"}}]`.
|
||||||
* answers 200 with a 43-byte 1x1 transparent GIF (verified 2026-09-09), which any
|
|
||||||
* image loader reports as a successful load — the cover then renders as nothing at
|
|
||||||
* all and no error placeholder ever fires. Only `?default=false` turns that into a
|
|
||||||
* 404; see [MetadataRepository] for the last-resort URL that uses it.
|
|
||||||
*/
|
*/
|
||||||
@Serializable
|
@Serializable
|
||||||
data class OpenLibraryCoverDto(
|
data class OpenLibraryWorkDto(
|
||||||
val small: String? = null,
|
val authors: List<OpenLibraryWorkAuthorDto> = emptyList(),
|
||||||
val medium: String? = null,
|
val description: JsonElement? = null,
|
||||||
val large: String? = null,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
@Serializable
|
@Serializable
|
||||||
data class OpenLibraryIdentifiersDto(
|
data class OpenLibraryWorkAuthorDto(val author: OpenLibraryKeyRefDto? = null)
|
||||||
@SerialName("isbn_10") val isbn10: List<String> = emptyList(),
|
|
||||||
@SerialName("isbn_13") val isbn13: List<String> = emptyList(),
|
|
||||||
)
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Maps the OL DTO to the source-agnostic [BookMetadata]. [coverUrl] stays null unless
|
* Open Library writes a description either as a bare string or as a typed object
|
||||||
* OL reports real cover art, so that the SPEC merge rule can fall through to Google
|
* `{"type": "/type/text", "value": "..."}`, and both shapes occur in the wild — of
|
||||||
* Books' thumbnail instead of pinning a URL that resolves to a blank image.
|
* 15 real editions sampled on 2026-09-20, 1 was a string, 5 were objects and 9 had
|
||||||
|
* no description at all. Modelling it as [JsonElement] and unwrapping here is
|
||||||
|
* cheaper than a custom serializer and cannot throw on a third shape.
|
||||||
*/
|
*/
|
||||||
fun OpenLibraryBookDto.toBookMetadata(lookupIsbn13: String): BookMetadata = BookMetadata(
|
internal fun JsonElement?.text(): String? = when (this) {
|
||||||
isbn13 = identifiers?.isbn13?.firstOrNull() ?: lookupIsbn13,
|
null -> null
|
||||||
isbn10 = identifiers?.isbn10?.firstOrNull(),
|
is JsonPrimitive -> contentOrNullIfNotString()
|
||||||
|
is JsonObject -> this["value"]?.jsonPrimitive?.contentOrNullIfNotString()
|
||||||
|
else -> null
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun JsonPrimitive.contentOrNullIfNotString(): String? =
|
||||||
|
if (isString) content.takeIf { it.isNotBlank() } else null
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The cover URL, or null when Open Library reports no art for this edition.
|
||||||
|
*
|
||||||
|
* Built from a cover **ID**, which is real evidence that art exists. The by-ISBN
|
||||||
|
* form (`/b/isbn/{isbn}-L.jpg`) is NOT evidence: for an edition with no art it
|
||||||
|
* answers 200 with a 43-byte 1x1 transparent GIF (verified 2026-09-09) that any
|
||||||
|
* image loader reports as a successful load and then paints as nothing at all.
|
||||||
|
* SPEC.md's cover chain keeps that URL only as a last resort, with `?default=false`
|
||||||
|
* so a miss becomes a 404 — see [MetadataRepository.byIsbnCoverUrl].
|
||||||
|
*/
|
||||||
|
internal fun OpenLibraryEditionDto.coverUrl(): String? =
|
||||||
|
covers.firstOrNull { it > 0 }?.let { "https://covers.openlibrary.org/b/id/$it-L.jpg" }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Maps the edition record to the source-agnostic [BookMetadata]. [authors] is passed
|
||||||
|
* in already resolved because resolving it needs the network and this function must
|
||||||
|
* stay pure; [description] likewise may come from the work when the edition has none
|
||||||
|
* and [OpenLibraryClient] fetched the work anyway.
|
||||||
|
*/
|
||||||
|
fun OpenLibraryEditionDto.toBookMetadata(
|
||||||
|
lookupIsbn13: String,
|
||||||
|
authors: List<String> = emptyList(),
|
||||||
|
description: String? = null,
|
||||||
|
): BookMetadata = BookMetadata(
|
||||||
|
isbn13 = isbn13.firstOrNull() ?: lookupIsbn13,
|
||||||
|
isbn10 = isbn10.firstOrNull(),
|
||||||
title = title,
|
title = title,
|
||||||
subtitle = subtitle,
|
subtitle = subtitle,
|
||||||
authors = authors.mapNotNull { it.name },
|
authors = authors,
|
||||||
publisher = publishers.firstOrNull()?.name,
|
publisher = publishers.firstOrNull()?.takeIf { it.isNotBlank() },
|
||||||
publishedDate = publishDate,
|
publishedDate = publishDate,
|
||||||
pageCount = numberOfPages,
|
pageCount = numberOfPages,
|
||||||
description = null,
|
description = this.description.text() ?: description,
|
||||||
coverUrl = cover?.large?.takeIf { it.isNotBlank() } ?: cover?.medium?.takeIf { it.isNotBlank() },
|
coverUrl = coverUrl(),
|
||||||
)
|
)
|
||||||
@@ -22,7 +22,7 @@ class GoogleBooksClientTest {
|
|||||||
private val client = GoogleBooksClient(OkHttpClient(), Json)
|
private val client = GoogleBooksClient(OkHttpClient(), Json)
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `parses a successful response into BookMetadata and forces https zoom=2 on the cover`() {
|
fun `parses a successful response into BookMetadata and normalizes the cover url`() {
|
||||||
val body = fixture("googlebooks_success.json")
|
val body = fixture("googlebooks_success.json")
|
||||||
val result = client.parseResponse(body)
|
val result = client.parseResponse(body)
|
||||||
|
|
||||||
@@ -35,11 +35,92 @@ class GoogleBooksClientTest {
|
|||||||
assertEquals("9780134685991", result.isbn13)
|
assertEquals("9780134685991", result.isbn13)
|
||||||
assertEquals("0134685997", result.isbn10)
|
assertEquals("0134685997", result.isbn10)
|
||||||
assertEquals(
|
assertEquals(
|
||||||
"https://books.google.com/books/content?id=ABC123XYZ&printsec=frontcover&img=1&zoom=2",
|
"https://books.google.com/books/content?id=ABC123XYZ&printsec=frontcover&img=1&zoom=1&w=400",
|
||||||
result.coverUrl,
|
result.coverUrl,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- normalizeCoverUrl: the 2026-09-20 "image not available" fix. ---
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The defect this rule exists to prevent. Google answers `zoom=2` for a volume
|
||||||
|
* it has no full preview of with **200 and a grey "image not available" PNG**,
|
||||||
|
* not a 404 — so Coil loads it happily, BookCover's own placeholder never fires,
|
||||||
|
* and the cover pipeline uploads Google's placeholder to PocketBase as the
|
||||||
|
* book's cover. Measured 2026-09-20: 11 of 18 real volumes did this at zoom=2,
|
||||||
|
* 0 of 18 at zoom=1 with a width. See normalizeCoverUrl's KDoc.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `pins zoom to 1 so google cannot substitute its image-not-available placeholder`() {
|
||||||
|
assertEquals(
|
||||||
|
"https://books.google.com/books/content?id=X&printsec=frontcover&img=1&zoom=1&w=400",
|
||||||
|
normalizeCoverUrl("http://books.google.com/books/content?id=X&printsec=frontcover&img=1&zoom=2"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `rewrites every zoom level, not just the one google usually sends`() {
|
||||||
|
listOf(0, 1, 2, 3, 5, 6).forEach { zoom ->
|
||||||
|
val normalized = normalizeCoverUrl("https://books.google.com/books/content?id=X&zoom=$zoom")
|
||||||
|
assertEquals(
|
||||||
|
"zoom=$zoom should have been pinned to 1",
|
||||||
|
"https://books.google.com/books/content?id=X&zoom=1&w=400",
|
||||||
|
normalized,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `adds a zoom and a width when the url carries neither`() {
|
||||||
|
assertEquals(
|
||||||
|
"https://books.google.com/books/content?id=X&zoom=1&w=400",
|
||||||
|
normalizeCoverUrl("https://books.google.com/books/content?id=X"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `edge=curl` paints a fake page-curl over the right edge of the cover. */
|
||||||
|
@Test
|
||||||
|
fun `strips the edge curl treatment`() {
|
||||||
|
val normalized = normalizeCoverUrl(
|
||||||
|
"http://books.google.com/books/content?id=X&img=1&zoom=1&edge=curl&source=gbs_api",
|
||||||
|
)
|
||||||
|
assertEquals(
|
||||||
|
"https://books.google.com/books/content?id=X&img=1&zoom=1&source=gbs_api&w=400",
|
||||||
|
normalized,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `forces https so the cover is not fetched in the clear`() {
|
||||||
|
val normalized = checkNotNull(normalizeCoverUrl("http://books.google.com/books/content?id=X"))
|
||||||
|
assertTrue(normalized.startsWith("https://"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/** An explicit width already on the URL is the caller's business, not ours to override. */
|
||||||
|
@Test
|
||||||
|
fun `keeps a width the url already specifies`() {
|
||||||
|
assertEquals(
|
||||||
|
"https://books.google.com/books/content?id=X&zoom=1&w=800",
|
||||||
|
normalizeCoverUrl("https://books.google.com/books/content?id=X&zoom=1&w=800"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** ...but "w=" buried in another parameter's value is not a width. */
|
||||||
|
@Test
|
||||||
|
fun `still adds a width when w= appears only inside another parameter value`() {
|
||||||
|
val normalized = checkNotNull(
|
||||||
|
normalizeCoverUrl("https://books.google.com/books/content?id=X&sig=aw=bc&zoom=1"),
|
||||||
|
)
|
||||||
|
assertTrue("expected a width to be appended, got $normalized", normalized.endsWith("&w=400"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `returns null for a missing or blank cover url`() {
|
||||||
|
assertNull(normalizeCoverUrl(null))
|
||||||
|
assertNull(normalizeCoverUrl(""))
|
||||||
|
assertNull(normalizeCoverUrl(" "))
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `returns null when there are no items`() {
|
fun `returns null when there are no items`() {
|
||||||
val body = fixture("googlebooks_no_items.json")
|
val body = fixture("googlebooks_no_items.json")
|
||||||
|
|||||||
+2
-2
@@ -35,13 +35,13 @@ class GoogleBooksSearchClientTest {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `parses the real success fixture and forces https zoom=2 on the cover`() {
|
fun `parses the real success fixture and normalizes the cover url`() {
|
||||||
val hits = client.parseResponse(fixture("googlebooks_success.json"))
|
val hits = client.parseResponse(fixture("googlebooks_success.json"))
|
||||||
assertEquals(1, hits.size)
|
assertEquals(1, hits.size)
|
||||||
assertEquals("Effective Java", hits[0].title)
|
assertEquals("Effective Java", hits[0].title)
|
||||||
assertEquals(listOf("9780134685991"), hits[0].isbn13s)
|
assertEquals(listOf("9780134685991"), hits[0].isbn13s)
|
||||||
assertEquals(
|
assertEquals(
|
||||||
"https://books.google.com/books/content?id=ABC123XYZ&printsec=frontcover&img=1&zoom=2",
|
"https://books.google.com/books/content?id=ABC123XYZ&printsec=frontcover&img=1&zoom=1&w=400",
|
||||||
hits[0].coverUrl,
|
hits[0].coverUrl,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,7 +4,11 @@ import kotlinx.coroutines.CancellationException
|
|||||||
import kotlinx.coroutines.test.runTest
|
import kotlinx.coroutines.test.runTest
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
import okhttp3.Interceptor
|
import okhttp3.Interceptor
|
||||||
|
import okhttp3.MediaType.Companion.toMediaType
|
||||||
import okhttp3.OkHttpClient
|
import okhttp3.OkHttpClient
|
||||||
|
import okhttp3.Protocol
|
||||||
|
import okhttp3.Response
|
||||||
|
import okhttp3.ResponseBody.Companion.toResponseBody
|
||||||
import org.junit.Assert.assertEquals
|
import org.junit.Assert.assertEquals
|
||||||
import org.junit.Assert.assertFalse
|
import org.junit.Assert.assertFalse
|
||||||
import org.junit.Assert.assertNull
|
import org.junit.Assert.assertNull
|
||||||
@@ -13,143 +17,316 @@ import org.junit.Assert.fail
|
|||||||
import org.junit.Test
|
import org.junit.Test
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Parses checked-in sample JSON fixtures (src/test/resources/fixtures) — no network
|
* Parses checked-in sample JSON fixtures (src/test/resources/fixtures) captured from
|
||||||
* involved, offline-safe. [OpenLibraryClient.parseResponse] is the unit under test;
|
* the live Editions API on 2026-09-20 — no network involved, offline-safe.
|
||||||
* [OpenLibraryClient.lookup] itself is untested here because it requires a live socket.
|
*
|
||||||
|
* Two layers are tested. The pure parse/classify functions take a body directly. The
|
||||||
|
* multi-request paths — author resolution, the work fallback, the [AuthorNameCache] —
|
||||||
|
* need a client that answers several URLs, so they run against [servingFixtures],
|
||||||
|
* an OkHttp [Interceptor] that serves fixtures by URL without opening a socket.
|
||||||
*/
|
*/
|
||||||
class OpenLibraryClientTest {
|
class OpenLibraryClientTest {
|
||||||
|
|
||||||
private val client = OpenLibraryClient(OkHttpClient(), Json)
|
private val client = OpenLibraryClient(OkHttpClient(), Json)
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `parses a successful response into BookMetadata`() {
|
fun `parses an edition response into BookMetadata`() {
|
||||||
val body = fixture("openlibrary_success.json")
|
val result = client.parseResponse(fixture("openlibrary_edition_gondolin.json"), "9781328613042")
|
||||||
val result = client.parseResponse(body, "9780201558029")
|
|
||||||
|
|
||||||
checkNotNull(result)
|
checkNotNull(result)
|
||||||
assertEquals("Concrete Mathematics", result.title)
|
assertEquals("The fall of Gondolin", result.title)
|
||||||
assertEquals("A Foundation for Computer Science", result.subtitle)
|
assertEquals("Houghton Mifflin Harcourt", result.publisher)
|
||||||
assertEquals(listOf("Ronald L. Graham", "Donald E. Knuth"), result.authors)
|
assertEquals("2018", result.publishedDate)
|
||||||
assertEquals("Addison-Wesley Professional", result.publisher)
|
assertEquals(302, result.pageCount)
|
||||||
assertEquals("1994", result.publishedDate)
|
assertEquals("9781328613042", result.isbn13)
|
||||||
assertEquals(672, result.pageCount)
|
assertEquals("1328613046", result.isbn10)
|
||||||
assertEquals("9780201558029", result.isbn13)
|
assertEquals("https://covers.openlibrary.org/b/id/12451486-L.jpg", result.coverUrl)
|
||||||
assertEquals("0201558025", result.isbn10)
|
|
||||||
assertEquals("https://covers.openlibrary.org/b/id/675832-L.jpg", result.coverUrl)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The by-ISBN cover endpoint answers 200 with a 1x1 transparent GIF for editions
|
* The legacy endpoint never returned a description, so this is new information —
|
||||||
* with no art, so synthesizing that URL here would hand the UI a cover that loads
|
* on this record it arrives as a bare string.
|
||||||
* "successfully" and paints nothing. No `cover` object means no cover URL, which
|
|
||||||
* is what lets [MetadataMerger] fall through to Google Books' thumbnail.
|
|
||||||
*/
|
*/
|
||||||
@Test
|
@Test
|
||||||
fun `leaves coverUrl null when the response reports no cover art`() {
|
fun `reads a description given as a bare string`() {
|
||||||
val body = """
|
val result = checkNotNull(
|
||||||
{"ISBN:9780201558029": {"title": "Concrete Mathematics", "publishers": [{"name": "AW"}]}}
|
client.parseResponse(fixture("openlibrary_edition_gondolin.json"), "9781328613042"),
|
||||||
""".trimIndent()
|
|
||||||
val result = checkNotNull(client.parseResponse(body, "9780201558029"))
|
|
||||||
|
|
||||||
assertEquals("Concrete Mathematics", result.title)
|
|
||||||
assertNull(result.coverUrl)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun `falls back to the medium cover when no large one is offered`() {
|
|
||||||
val body = """
|
|
||||||
{"ISBN:9780201558029": {"title": "Concrete Mathematics",
|
|
||||||
"cover": {"small": "https://covers.openlibrary.org/b/id/675832-S.jpg",
|
|
||||||
"medium": "https://covers.openlibrary.org/b/id/675832-M.jpg"}}}
|
|
||||||
""".trimIndent()
|
|
||||||
val result = checkNotNull(client.parseResponse(body, "9780201558029"))
|
|
||||||
|
|
||||||
assertEquals("https://covers.openlibrary.org/b/id/675832-M.jpg", result.coverUrl)
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Two real books off the user's shelf that the app failed to identify on its
|
|
||||||
* first on-device run, captured verbatim from the live API on 2026-09-09. Both
|
|
||||||
* are small-press (Bethlehem Books) children's historical fiction — the exact
|
|
||||||
* profile we assumed Open Library would be thin on. It isn't: both carry title,
|
|
||||||
* author, publisher and cover art, and neither exposes an `isbn_13` identifier,
|
|
||||||
* which is why the lookup ISBN has to survive as the fallback. Whatever went
|
|
||||||
* wrong on the phone, it was NOT this parser and NOT Open Library's coverage.
|
|
||||||
*/
|
|
||||||
@Test
|
|
||||||
fun `parses the real responses for the two books the app failed to identify`() {
|
|
||||||
val hittite = checkNotNull(
|
|
||||||
client.parseResponse(fixture("openlibrary_hittite_warrior.json"), "9781883937386"),
|
|
||||||
)
|
)
|
||||||
|
assertTrue(
|
||||||
|
"expected the Gondolin blurb, got ${result.description}",
|
||||||
|
result.description.orEmpty().contains("Morgoth"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** ...and on this one as a typed object, `{"type": ..., "value": ...}`. Both are real. */
|
||||||
|
@Test
|
||||||
|
fun `reads a description given as a typed object`() {
|
||||||
|
val result = checkNotNull(
|
||||||
|
client.parseResponse(fixture("openlibrary_edition_hittite.json"), "9781883937386"),
|
||||||
|
)
|
||||||
|
assertEquals("Hittite warrior", result.title)
|
||||||
|
val description = checkNotNull(result.description) { "expected the object-form description" }
|
||||||
|
assertTrue("description should be unwrapped text, got: $description", description.isNotBlank())
|
||||||
|
assertFalse("the JSON wrapper leaked into the text", description.contains("/type/text"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `leaves description null when the edition has none`() {
|
||||||
|
val result = checkNotNull(
|
||||||
|
client.parseResponse(fixture("openlibrary_edition_harp.json"), "9780898707168"),
|
||||||
|
)
|
||||||
|
assertNull(result.description)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Open Library uses -1 in `covers` to mean "no cover here". This real record is
|
||||||
|
* `[698345, -1]`; taking the list's first positive entry is what keeps us from
|
||||||
|
* ever building covers.openlibrary.org/b/id/-1-L.jpg.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `skips the -1 no-cover sentinel when choosing a cover id`() {
|
||||||
|
val result = checkNotNull(
|
||||||
|
client.parseResponse(fixture("openlibrary_edition_harp.json"), "9780898707168"),
|
||||||
|
)
|
||||||
|
assertEquals("https://covers.openlibrary.org/b/id/698345-L.jpg", result.coverUrl)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `leaves coverUrl null when every cover id is a sentinel`() {
|
||||||
|
val body = """{"title": "No Art", "covers": [-1]}"""
|
||||||
|
assertNull(checkNotNull(client.parseResponse(body, "9780201558029")).coverUrl)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `leaves coverUrl null when the edition reports no covers at all`() {
|
||||||
|
val body = """{"title": "No Art"}"""
|
||||||
|
assertNull(checkNotNull(client.parseResponse(body, "9780201558029")).coverUrl)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** No isbn_13 in the record — the ISBN we looked up has to carry through. */
|
||||||
|
@Test
|
||||||
|
fun `falls back to the looked-up isbn when the record carries no isbn_13`() {
|
||||||
|
val body = """{"title": "Hittite warrior", "isbn_10": ["1883937388"]}"""
|
||||||
|
val result = checkNotNull(client.parseResponse(body, "9781883937386"))
|
||||||
|
|
||||||
|
assertEquals("9781883937386", result.isbn13)
|
||||||
|
assertEquals("1883937388", result.isbn10)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `fails soft on malformed json instead of throwing`() {
|
||||||
|
assertNull(client.parseResponse(fixture("malformed.json"), "9780201558029"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Carried over from the legacy-endpoint tests: these are the two books that
|
||||||
|
* failed on the user's phone in waves 5/6, and the reason this project knows the
|
||||||
|
* coverage hypothesis was wrong. Re-pinned against the Editions API so the
|
||||||
|
* endpoint move can't quietly regress them.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `parses the real responses for the two books the app once failed to identify`() = runTest {
|
||||||
|
val client = servingFixtures()
|
||||||
|
|
||||||
|
val hittite = (client.lookupOnce("9781883937386") as? SourceResult.Found)?.metadata
|
||||||
|
checkNotNull(hittite) { "expected Found for Hittite Warrior" }
|
||||||
assertEquals("Hittite warrior", hittite.title)
|
assertEquals("Hittite warrior", hittite.title)
|
||||||
assertEquals(listOf("Joanne S. Williamson"), hittite.authors)
|
assertEquals(listOf("Joanne S. Williamson"), hittite.authors)
|
||||||
assertEquals("Bethlehem Books", hittite.publisher)
|
assertEquals("Bethlehem Books", hittite.publisher)
|
||||||
assertEquals(237, hittite.pageCount)
|
assertEquals(237, hittite.pageCount)
|
||||||
assertEquals("1883937388", hittite.isbn10)
|
assertEquals("1883937388", hittite.isbn10)
|
||||||
// No isbn_13 in the record — the ISBN we looked up has to carry through.
|
|
||||||
assertEquals("9781883937386", hittite.isbn13)
|
assertEquals("9781883937386", hittite.isbn13)
|
||||||
assertEquals("https://covers.openlibrary.org/b/id/930599-L.jpg", hittite.coverUrl)
|
assertEquals("https://covers.openlibrary.org/b/id/930599-L.jpg", hittite.coverUrl)
|
||||||
|
|
||||||
val shadowHawk = checkNotNull(
|
val shadowHawk = (client.lookupOnce("9781883937676") as? SourceResult.Found)?.metadata
|
||||||
client.parseResponse(fixture("openlibrary_shadow_hawk.json"), "9781883937676"),
|
checkNotNull(shadowHawk) { "expected Found for Shadow Hawk" }
|
||||||
)
|
|
||||||
assertEquals("Shadow hawk", shadowHawk.title)
|
assertEquals("Shadow hawk", shadowHawk.title)
|
||||||
assertEquals(listOf("Andre Norton"), shadowHawk.authors)
|
assertEquals(listOf("Andre Norton"), shadowHawk.authors)
|
||||||
assertEquals("9781883937676", shadowHawk.isbn13)
|
assertEquals("9781883937676", shadowHawk.isbn13)
|
||||||
checkNotNull(shadowHawk.coverUrl)
|
assertEquals("https://covers.openlibrary.org/b/id/930617-L.jpg", shadowHawk.coverUrl)
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun `returns null when the isbn key is absent from the response`() {
|
|
||||||
val body = fixture("openlibrary_not_found.json")
|
|
||||||
assertNull(client.parseResponse(body, "9780201558029"))
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun `fails soft on malformed json instead of throwing`() {
|
|
||||||
val body = fixture("malformed.json")
|
|
||||||
assertNull(client.parseResponse(body, "9780201558029"))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- classify(): the three-way per-source outcome (SPEC.md "Book metadata lookup"). ---
|
// --- classify(): the three-way per-source outcome (SPEC.md "Book metadata lookup"). ---
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports Found for a 2xx response with a record`() {
|
fun `classify reports Found for a 2xx response with a record`() {
|
||||||
val result = client.classify(200, fixture("openlibrary_success.json"), "9780201558029")
|
val result = client.classify(200, fixture("openlibrary_edition_gondolin.json"), "9781328613042")
|
||||||
|
|
||||||
val found = result as? SourceResult.Found
|
val found = result as? SourceResult.Found
|
||||||
checkNotNull(found) { "expected Found, got $result" }
|
checkNotNull(found) { "expected Found, got $result" }
|
||||||
assertEquals("Concrete Mathematics", found.metadata.title)
|
assertEquals("The fall of Gondolin", found.metadata.title)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The semantic change that came with the endpoint move, and the one most worth
|
||||||
|
* pinning down. The legacy `/api/books` reported a miss as 200 with an empty
|
||||||
|
* object, so every non-2xx there was a failure. `/isbn/{isbn}.json` has no such
|
||||||
|
* body to return and answers 404, which IS Open Library saying authoritatively
|
||||||
|
* that it holds no such edition. Reporting that as Unavailable would put a retry
|
||||||
|
* affordance in front of the user for a book no amount of retrying will find.
|
||||||
|
*/
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports NotFound for a 2xx response with no record for that isbn`() {
|
fun `classify reports NotFound for a 404, because on this endpoint 404 is authoritative`() {
|
||||||
val result = client.classify(200, fixture("openlibrary_not_found.json"), "9780201558029")
|
assertEquals(SourceResult.NotFound, client.classify(404, "<!DOCTYPE html><html>…", "9780201558029"))
|
||||||
assertEquals(SourceResult.NotFound, result)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ...but only 404. Every other 4xx is still a failure: SPEC.md's rule is that a
|
||||||
|
* source we could not reach must never reach the user as a book that isn't there.
|
||||||
|
*/
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports Failed for a 404`() {
|
fun `classify reports Failed for a 403, distinctly from NotFound`() {
|
||||||
val result = client.classify(404, null, "9780201558029")
|
assertEquals(
|
||||||
assertEquals(SourceResult.Failed("http 404", FailureKind.CLIENT_ERROR), result)
|
SourceResult.Failed("http 403", FailureKind.CLIENT_ERROR),
|
||||||
|
client.classify(403, null, "9780201558029"),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports Failed for a 429, distinctly from NotFound`() {
|
fun `classify reports Failed for a 429, distinctly from NotFound`() {
|
||||||
val result = client.classify(429, null, "9780201558029")
|
assertEquals(
|
||||||
assertEquals(SourceResult.Failed("http 429 (rate limited)", FailureKind.RATE_LIMITED), result)
|
SourceResult.Failed("http 429 (rate limited)", FailureKind.RATE_LIMITED),
|
||||||
|
client.classify(429, null, "9780201558029"),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports Failed for a 500`() {
|
fun `classify reports Failed for a 500`() {
|
||||||
val result = client.classify(500, "Internal Server Error", "9780201558029")
|
assertEquals(
|
||||||
assertEquals(SourceResult.Failed("http 500 (server error)", FailureKind.SERVER_ERROR), result)
|
SourceResult.Failed("http 500 (server error)", FailureKind.SERVER_ERROR),
|
||||||
|
client.classify(500, "Internal Server Error", "9780201558029"),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `classify reports Failed for a malformed body even on a 2xx status`() {
|
fun `classify reports Failed for a malformed body even on a 2xx status`() {
|
||||||
val result = client.classify(200, fixture("malformed.json"), "9780201558029")
|
assertEquals(
|
||||||
assertEquals(SourceResult.Failed("malformed json", FailureKind.MALFORMED), result)
|
SourceResult.Failed("malformed json", FailureKind.MALFORMED),
|
||||||
|
client.classify(200, fixture("malformed.json"), "9780201558029"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `classify reports Failed for an empty body on a 2xx status`() {
|
||||||
|
assertEquals(
|
||||||
|
SourceResult.Failed("empty body", FailureKind.MALFORMED),
|
||||||
|
client.classify(200, "", "9780201558029"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- author resolution: the cost the Editions API adds, and the cache that pays it. ---
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `resolves author references to names`() = runTest {
|
||||||
|
val client = servingFixtures()
|
||||||
|
|
||||||
|
val result = client.lookupOnce("9781328613042") as? SourceResult.Found
|
||||||
|
checkNotNull(result) { "expected Found" }
|
||||||
|
assertEquals(listOf("J.R.R. Tolkien"), result.metadata.authors)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ISBN 9780898707168 is on the user's own shelf and its edition record has NO
|
||||||
|
* authors — they exist only on the work. The legacy endpoint resolved that for
|
||||||
|
* us and returned "Laura M. Berquist", so without this fallback the endpoint
|
||||||
|
* move would have silently dropped the author for books like it.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `falls back to the work when the edition record names no authors`() = runTest {
|
||||||
|
val client = servingFixtures()
|
||||||
|
|
||||||
|
val result = client.lookupOnce("9780898707168") as? SourceResult.Found
|
||||||
|
checkNotNull(result) { "expected Found" }
|
||||||
|
assertEquals("The harp and laurel wreath", result.metadata.title)
|
||||||
|
assertEquals(listOf("Laura M. Berquist"), result.metadata.authors)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `does not fetch the work when the edition already names its authors`() = runTest {
|
||||||
|
val requests = mutableListOf<String>()
|
||||||
|
val client = servingFixtures(log = requests)
|
||||||
|
|
||||||
|
client.lookupOnce("9781328613042")
|
||||||
|
|
||||||
|
assertTrue(
|
||||||
|
"the work should not be fetched when the edition has authors; got $requests",
|
||||||
|
requests.none { it.contains("/works/") },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The point of the cache. A second book by an author already seen resolves from
|
||||||
|
* memory — which is the shape of the user's scanning session, where books by one
|
||||||
|
* author sit together on the shelf and get scanned one after another.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `caches author names across lookups instead of re-fetching`() = runTest {
|
||||||
|
val requests = mutableListOf<String>()
|
||||||
|
val cache = AuthorNameCache()
|
||||||
|
val client = servingFixtures(log = requests, cache = cache)
|
||||||
|
|
||||||
|
client.lookupOnce("9781328613042")
|
||||||
|
val afterFirst = requests.count { it.contains("/authors/") }
|
||||||
|
client.lookupOnce("9781328613042")
|
||||||
|
val afterSecond = requests.count { it.contains("/authors/") }
|
||||||
|
|
||||||
|
assertEquals("the first lookup should resolve the author once", 1, afterFirst)
|
||||||
|
assertEquals("the second lookup should not re-fetch the author", 1, afterSecond)
|
||||||
|
assertEquals(1, cache.size)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `a cache shared between clients is reused by both`() = runTest {
|
||||||
|
val requests = mutableListOf<String>()
|
||||||
|
val cache = AuthorNameCache()
|
||||||
|
|
||||||
|
servingFixtures(log = requests, cache = cache).lookupOnce("9781328613042")
|
||||||
|
servingFixtures(log = requests, cache = cache).lookupOnce("9781328613042")
|
||||||
|
|
||||||
|
assertEquals(1, requests.count { it.contains("/authors/") })
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Author resolution is best-effort: the edition request already succeeded, so a
|
||||||
|
* failure resolving a name must degrade the record rather than turn a book we
|
||||||
|
* found into a lookup the user is told to retry.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `a failing author request degrades the record instead of failing the lookup`() = runTest {
|
||||||
|
val client = servingFixtures(failUrlsContaining = "/authors/")
|
||||||
|
|
||||||
|
val result = client.lookupOnce("9781328613042") as? SourceResult.Found
|
||||||
|
checkNotNull(result) { "expected Found despite the author request failing, got it as a failure" }
|
||||||
|
assertEquals("The fall of Gondolin", result.metadata.title)
|
||||||
|
assertEquals(emptyList<String>(), result.metadata.authors)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `a failing work request degrades the record instead of failing the lookup`() = runTest {
|
||||||
|
val client = servingFixtures(failUrlsContaining = "/works/")
|
||||||
|
|
||||||
|
val result = client.lookupOnce("9780898707168") as? SourceResult.Found
|
||||||
|
checkNotNull(result) { "expected Found despite the work request failing" }
|
||||||
|
assertEquals("The harp and laurel wreath", result.metadata.title)
|
||||||
|
assertEquals(emptyList<String>(), result.metadata.authors)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A failed resolution must not be remembered — one bad moment shouldn't cost a name all session. */
|
||||||
|
@Test
|
||||||
|
fun `does not cache a failed author resolution`() = runTest {
|
||||||
|
val cache = AuthorNameCache()
|
||||||
|
servingFixtures(cache = cache, failUrlsContaining = "/authors/").lookupOnce("9781328613042")
|
||||||
|
assertEquals(0, cache.size)
|
||||||
|
|
||||||
|
val result = servingFixtures(cache = cache).lookupOnce("9781328613042") as? SourceResult.Found
|
||||||
|
assertEquals(listOf("J.R.R. Tolkien"), checkNotNull(result).metadata.authors)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun `a 404 from the edition endpoint reaches the caller as NotFound`() = runTest {
|
||||||
|
val client = servingFixtures()
|
||||||
|
|
||||||
|
assertEquals(SourceResult.NotFound, client.lookupOnce("9780000000000"))
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- fetch(): §1 of the 2026-09-12 crash fix -- same coverage as GoogleBooksClientTest.
|
// --- fetch(): §1 of the 2026-09-12 crash fix -- same coverage as GoogleBooksClientTest.
|
||||||
@@ -189,6 +366,23 @@ class OpenLibraryClientTest {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cancellation must survive the best-effort author path too, which swallows
|
||||||
|
* Throwable by design. If it were swallowed there, dismissing the scan sheet
|
||||||
|
* mid-lookup would leave the request running instead of unwinding.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun `CancellationException from a supporting request propagates rather than being swallowed`() = runTest {
|
||||||
|
val client = servingFixtures(throwOnUrlsContaining = "/authors/" to CancellationException("dismissed"))
|
||||||
|
|
||||||
|
try {
|
||||||
|
client.lookupOnce("9781328613042")
|
||||||
|
fail("expected CancellationException to propagate out of author resolution")
|
||||||
|
} catch (e: CancellationException) {
|
||||||
|
// expected
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private fun clientThrowing(t: Throwable): OpenLibraryClient {
|
private fun clientThrowing(t: Throwable): OpenLibraryClient {
|
||||||
val httpClient = OkHttpClient.Builder()
|
val httpClient = OkHttpClient.Builder()
|
||||||
.addInterceptor(Interceptor { throw t })
|
.addInterceptor(Interceptor { throw t })
|
||||||
@@ -196,6 +390,49 @@ class OpenLibraryClientTest {
|
|||||||
return OpenLibraryClient(httpClient, Json)
|
return OpenLibraryClient(httpClient, Json)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A client whose HTTP layer answers from the fixture set instead of the network,
|
||||||
|
* so the multi-request paths (edition -> work -> author) are exercised end to end
|
||||||
|
* offline. Unknown URLs answer 404, which is also how the not-found case is driven.
|
||||||
|
*/
|
||||||
|
private fun servingFixtures(
|
||||||
|
log: MutableList<String>? = null,
|
||||||
|
cache: AuthorNameCache = AuthorNameCache(),
|
||||||
|
failUrlsContaining: String? = null,
|
||||||
|
throwOnUrlsContaining: Pair<String, Throwable>? = null,
|
||||||
|
): OpenLibraryClient {
|
||||||
|
val bodies = mapOf(
|
||||||
|
"https://openlibrary.org/isbn/9781328613042.json" to "openlibrary_edition_gondolin.json",
|
||||||
|
"https://openlibrary.org/isbn/9780898707168.json" to "openlibrary_edition_harp.json",
|
||||||
|
"https://openlibrary.org/isbn/9781883937386.json" to "openlibrary_edition_hittite.json",
|
||||||
|
"https://openlibrary.org/isbn/9781883937676.json" to "openlibrary_edition_shadowhawk.json",
|
||||||
|
"https://openlibrary.org/works/OL19479056W.json" to "openlibrary_work_harp.json",
|
||||||
|
"https://openlibrary.org/authors/OL335577A.json" to "openlibrary_author_berquist.json",
|
||||||
|
"https://openlibrary.org/authors/OL26320A.json" to "openlibrary_author_tolkien.json",
|
||||||
|
"https://openlibrary.org/authors/OL244960A.json" to "openlibrary_author_williamson.json",
|
||||||
|
"https://openlibrary.org/authors/OL27951A.json" to "openlibrary_author_norton.json",
|
||||||
|
)
|
||||||
|
val httpClient = OkHttpClient.Builder()
|
||||||
|
.addInterceptor { chain ->
|
||||||
|
val url = chain.request().url.toString()
|
||||||
|
log?.add(url)
|
||||||
|
throwOnUrlsContaining?.let { (needle, t) -> if (needle in url) throw t }
|
||||||
|
val failed = failUrlsContaining != null && failUrlsContaining in url
|
||||||
|
val fixtureName = bodies[url]
|
||||||
|
val code = if (failed || fixtureName == null) 404 else 200
|
||||||
|
val body = if (code == 200) fixture(fixtureName!!) else "<!DOCTYPE html><html>not found</html>"
|
||||||
|
Response.Builder()
|
||||||
|
.request(chain.request())
|
||||||
|
.protocol(Protocol.HTTP_1_1)
|
||||||
|
.code(code)
|
||||||
|
.message(if (code == 200) "OK" else "Not Found")
|
||||||
|
.body(body.toResponseBody("application/json".toMediaType()))
|
||||||
|
.build()
|
||||||
|
}
|
||||||
|
.build()
|
||||||
|
return OpenLibraryClient(httpClient, Json, cache)
|
||||||
|
}
|
||||||
|
|
||||||
private fun fixture(name: String): String =
|
private fun fixture(name: String): String =
|
||||||
checkNotNull(javaClass.classLoader.getResourceAsStream("fixtures/$name")) { "missing fixture $name" }
|
checkNotNull(javaClass.classLoader.getResourceAsStream("fixtures/$name")) { "missing fixture $name" }
|
||||||
.bufferedReader()
|
.bufferedReader()
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
package org.modg.bookshelf.ui.library
|
package org.modg.bookshelf.ui.library
|
||||||
|
|
||||||
|
import androidx.lifecycle.viewModelScope
|
||||||
import androidx.room.Room
|
import androidx.room.Room
|
||||||
import androidx.test.core.app.ApplicationProvider
|
import androidx.test.core.app.ApplicationProvider
|
||||||
import java.util.concurrent.CountDownLatch
|
import java.util.concurrent.CountDownLatch
|
||||||
import kotlinx.coroutines.CancellationException
|
import kotlinx.coroutines.CancellationException
|
||||||
import kotlinx.coroutines.Dispatchers
|
import kotlinx.coroutines.Dispatchers
|
||||||
|
import kotlinx.coroutines.cancel
|
||||||
import kotlinx.coroutines.ExperimentalCoroutinesApi
|
import kotlinx.coroutines.ExperimentalCoroutinesApi
|
||||||
import kotlinx.coroutines.flow.first
|
import kotlinx.coroutines.flow.first
|
||||||
import kotlinx.coroutines.launch
|
import kotlinx.coroutines.launch
|
||||||
@@ -64,6 +66,9 @@ class LibraryViewModelTest {
|
|||||||
private lateinit var context: android.content.Context
|
private lateinit var context: android.content.Context
|
||||||
private val coverHost = FakeCoverHost()
|
private val coverHost = FakeCoverHost()
|
||||||
|
|
||||||
|
/** Every view model built by [viewModel], so [tearDown] can cancel each one. */
|
||||||
|
private val viewModels = mutableListOf<LibraryViewModel>()
|
||||||
|
|
||||||
@Before
|
@Before
|
||||||
fun setUp() {
|
fun setUp() {
|
||||||
Dispatchers.setMain(UnconfinedTestDispatcher())
|
Dispatchers.setMain(UnconfinedTestDispatcher())
|
||||||
@@ -76,9 +81,25 @@ class LibraryViewModelTest {
|
|||||||
bookRepository = BookRepository(db.bookDao(), context)
|
bookRepository = BookRepository(db.bookDao(), context)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Every view model this test built has to be cancelled before [Dispatchers.resetMain].
|
||||||
|
*
|
||||||
|
* [LibraryViewModel] exposes eleven `stateIn(viewModelScope, WhileSubscribed(5_000), ...)`
|
||||||
|
* flows, so its upstreams keep running for five seconds after their last collector
|
||||||
|
* goes away. Nothing cleared the view model, so those coroutines outlived the test
|
||||||
|
* that made them and were still touching `Dispatchers.Main` while the NEXT test's
|
||||||
|
* setUp/tearDown reset it — which surfaces as a flaky
|
||||||
|
* "Dispatchers.Main is used concurrently with setting it" in whichever test
|
||||||
|
* happens to be running when the window expires, not in the one that leaked.
|
||||||
|
* Found 2026-09-20 when unrelated new tests shifted the suite's timing.
|
||||||
|
*/
|
||||||
@After
|
@After
|
||||||
fun tearDown() {
|
fun tearDown() {
|
||||||
db.close()
|
viewModels.forEach { it.viewModelScope.cancel() }
|
||||||
|
viewModels.clear()
|
||||||
|
// Guarded because a failure in setUp leaves this uninitialized, and an
|
||||||
|
// UninitializedPropertyAccessException here would mask the real failure.
|
||||||
|
if (::db.isInitialized) db.close()
|
||||||
Dispatchers.resetMain()
|
Dispatchers.resetMain()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -95,7 +116,7 @@ class LibraryViewModelTest {
|
|||||||
bookSearchRepository = BookSearchRepository(searchHttpClient, Json { ignoreUnknownKeys = true }),
|
bookSearchRepository = BookSearchRepository(searchHttpClient, Json { ignoreUnknownKeys = true }),
|
||||||
metadataRepository = MetadataRepository(OkHttpClient(), Json { ignoreUnknownKeys = true }),
|
metadataRepository = MetadataRepository(OkHttpClient(), Json { ignoreUnknownKeys = true }),
|
||||||
initialShelfId = null,
|
initialShelfId = null,
|
||||||
)
|
).also { viewModels += it }
|
||||||
|
|
||||||
// --- sync (2026-09-17 auth-expiry incident) ---
|
// --- sync (2026-09-17 auth-expiry incident) ---
|
||||||
|
|
||||||
|
|||||||
@@ -151,8 +151,8 @@ class ScanViewModelTest {
|
|||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun `a successful lookup preloads its cover, and save stores it as the book's cover file`() = runTest {
|
fun `a successful lookup preloads its cover, and save stores it as the book's cover file`() = runTest {
|
||||||
val olFixture = fixture("openlibrary_success.json")
|
val olFixture = fixture("openlibrary_edition_gondolin.json")
|
||||||
val coverUrl = "https://covers.openlibrary.org/b/id/675832-L.jpg" // the fixture's reported cover.large
|
val coverUrl = "https://covers.openlibrary.org/b/id/12451486-L.jpg" // built from the fixture's cover id
|
||||||
coverHost.serve(coverUrl, byteArrayOf(5, 5, 5))
|
coverHost.serve(coverUrl, byteArrayOf(5, 5, 5))
|
||||||
val vm = ScanViewModel(
|
val vm = ScanViewModel(
|
||||||
bookRepository = BookRepository(db.bookDao(), context),
|
bookRepository = BookRepository(db.bookDao(), context),
|
||||||
@@ -162,12 +162,12 @@ class ScanViewModelTest {
|
|||||||
coverDownloader = coverHost.downloader(java.io.File(context.cacheDir, "cover-preload")),
|
coverDownloader = coverHost.downloader(java.io.File(context.cacheDir, "cover-preload")),
|
||||||
)
|
)
|
||||||
|
|
||||||
vm.runLookup("9780201558029")
|
vm.runLookup("9781328613042")
|
||||||
val found = vm.sheetState.value as ScanSheetState.Found
|
val found = vm.sheetState.value as ScanSheetState.Found
|
||||||
val preloaded = (vm.cover.first { it is CoverPreloadState.Ready } as CoverPreloadState.Ready).file
|
val preloaded = (vm.cover.first { it is CoverPreloadState.Ready } as CoverPreloadState.Ready).file
|
||||||
assertTrue(vm.performSave(found.isbn13, found.metadata))
|
assertTrue(vm.performSave(found.isbn13, found.metadata))
|
||||||
|
|
||||||
val saved = db.bookDao().findByIsbn13("9780201558029")
|
val saved = db.bookDao().findByIsbn13("9781328613042")
|
||||||
checkNotNull(saved)
|
checkNotNull(saved)
|
||||||
assertTrue(java.io.File(checkNotNull(saved.localCoverPath)).readBytes().contentEquals(byteArrayOf(5, 5, 5)))
|
assertTrue(java.io.File(checkNotNull(saved.localCoverPath)).readBytes().contentEquals(byteArrayOf(5, 5, 5)))
|
||||||
assertTrue("the preload is discarded once saved", !preloaded.exists())
|
assertTrue("the preload is discarded once saved", !preloaded.exists())
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
{"name": "Laura M. Berquist", "personal_name": "Laura M. Berquist", "last_modified": {"type": "/type/datetime", "value": "2008-09-05 16:35:29.434581"}, "key": "/authors/OL335577A", "type": {"key": "/type/author"}, "id": 898991, "revision": 2}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"name": "Andre Norton", "bio": {"type": "/type/text", "value": "Andre Norton was born Alice Mary Norton in Cleveland, Ohio, the daughter of rug company owner and his wife. She began writing while she was in high school, and she was the editor of a literary page in the school's paper. She also wrote her first novel, Ralestone Luck, which was published in 1938. Her first published novel was The Prince Commands (1934). She graduated from high school in 1930 and began studying teaching at Flora Stone Mather College of Western Reserve University. In 1932 she dropped out early due to economic conditions and began working for the Cleveland Library System. In 1934, she legally changed her name to Andre Alice Norton, the pen name she had adopted to increase her marketability since boys were the main audience for fantasy. In 1941, she bought a bookstore called the Mystery House in Mount Rainier, Maryland, but the business failed and she returned to the Cleveland Public Library. In 1950 she became a reader for the Gnome Press Co. In 1958 she became a full-time author. In 1966 she moved to Florida for health reasons, and then to Murfreesboro, Tennessee. In 1977, she received the Gandalf Grand Master Award from the World Science Fiction Society, and in 1983 she received the Damon Knight Memorial Grand Master Award from the Science Fiction and Fantasy Writers of America. She died in March of 2005 of congestive heart failure.\r\n\r\nShe has been called the Grande Dame of Science Fiction and Fantasy. Over the course of her career, she published over 300 published titles read by four generations. Shortly after her death, the Science Fiction and Fantasy Writers of America created the Andre Norton Award for outstanding work of fantasy or science fiction for Young Adults."}, "remote_ids": {"viaf": "121640202", "wikidata": "Q235899", "bookbrainz": "c3dc7911-ce39-41bf-ad75-07dbc347e721", "musicbrainz": "3bb1d81c-9497-45e1-b257-f1635755ba28", "goodreads": "4766", "isni": "0000000114809327", "imdb": "nm0636157", "lc_naf": "n80126299", "librarything": "nortonandre", "librivox": "358", "project_gutenberg": "7021", "opac_sbn": "CFIV037867"}, "alternate_names": ["Andre Alice Norton", "Andrew North", "Alice Mary Norton", "Alice M. Norton", "Andr\u00e9 Norton", "Alice Andre Norton", "Andre NORTON", "Andre\u0301 Norton", "Norton Andr\u00e9", "Andre\u0301 NORTON", "Mar\u00eda del Carmen Guti\u00e9rrez S\u00e1nchez", "Carmela Guti\u00e9rrez de Gambra", "Miguel Arazuri", "Andr\u00e9 Ronsac", "Clara San Miguel", "Alice Norton", "Enid Colman", "Andre\u0307 Norton", "Andre Norton Mercedes Lackey", "Norton Andre", "Andre Norton and Mercedes Lackey", "Andre Norton-", "Norton, Andre, Griffin, P. M.", "Andre Norton -", "Andrew [aka Andre Norton North", "Andre Andre Norton", "Andre Alice Andre Alice Norton", "Norton Andre Alice", "andre norton", "Andre norton", "Andrew, North"], "death_date": "17 March 2005", "personal_name": "Andre Norton", "type": {"key": "/type/author"}, "links": [{"title": "andre-norton-books.com", "url": "http://www.andre-norton-books.com/", "type": {"key": "/type/link"}}], "fuller_name": "Andre Alice Norton", "photos": [14612221, 5542283, 7412718, 7390867, 14613606], "birth_date": "17 February 1912", "source_records": ["amazon:1374845965", "bwb:9780099100218", "bwb:9781437804874", "bwb:9781520267586", "amazon:0575005254", "bwb:9798736238385", "amazon:042605055X", "bwb:9798729436309", "bwb:9781973137955", "promise:bwb_daily_pallets_2021-03-30", "promise:bwb_daily_pallets_2022-08-13", "promise:bwb_daily_pallets_2021-01-13", "promise:bwb_daily_pallets_2022-06-06:O8-ANT-625", "promise:bwb_daily_pallets_2023-04-19:W8-BLD-821", "promise:bwb_daily_pallets_2023-05-22:292-BAB-432", "promise:bwb_daily_pallets_2023-05-24:P8-CKJ-015", "bwb:9788184566833", "bwb:9781976559105", "bwb:9781981935871", "bwb:9781497390560", "bwb:9781721068869", "ia:andrenortonexile0000andr", "ia:bwb_P9-DFF-436", "ia:bwb_S0-ASU-251", "ia:merlinsmirror0000andr", "bwb:9781986788106"], "key": "/authors/OL27951A", "latest_revision": 29, "revision": 29, "created": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "last_modified": {"type": "/type/datetime", "value": "2025-10-30T20:32:14.601101"}}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"alternate_names": ["Dzhon R. R. Tolkin", "J. R. R. Tolkien", "J. R. R. Tolki\u00ebn", "J. R.R. Tolkien", "John R. R. Tolkien", "John Ronald Reuel Tolkien", "Tolkien", "Yue Han Luo Na De Rui Er Tuo Er Jin", "J.R.R Tolkien", "J.R.R. TOLKIEN", "Tolkien, J. R. R. (John Ronald Reuel)"], "personal_name": "J. R. R. Tolkien", "type": {"key": "/type/author"}, "death_date": "2 September 1973", "source_records": ["ia:herrabliss0000tolk", "amazon:0007557272", "amazon:0007637675", "amazon:8845927555", "bwb:9780008360108", "amazon:3867170355", "amazon:8830102717", "amazon:2266017519", "amazon:0007270615", "ia:futatsunoto0000unse", "amazon:0342851705", "ia:leseigneurdesann0000tolk_k6z7", "amazon:3423590610", "amazon:1865035955", "amazon:3608939849", "amazon:0007317255", "amazon:902254771X", "amazon:0344895459", "ia:twotowers0002tolk_r5g9", "ia:returnofkingbein0003tolk_m6s4", "amazon:226700092X", "ia:lordofrings0000tolk_j2h8", "ia:lacomunidaddelan0000tolk", "amazon:0618260293", "amazon:1705047610", "amazon:2267017415", "amazon:0007635559", "amazon:360893829X", "bwb:9781782012696", "marc:marc_uic/UIC_2022.mrc:9810667:974", "amazon:1483034577", "amazon:8845293890", "bwb:9786070791390", "promise:bwb_daily_pallets_2022-07-28", "promise:bwb_daily_pallets_2023-02-22:W8-BNK-802", "promise:bwb_daily_pallets_2020-04-30", "amazon:9022531953", "promise:bwb_daily_pallets_2022-06-06", "amazon:9600403678", "bwb:9781693418167", "amazon:8481362581", "amazon:8467909234", "amazon:2356414916", "amazon:8377856816", "amazon:2267030519", "amazon:8377856824", "promise:bwb_daily_pallets_2022-05-16", "promise:bwb_daily_pallets_2022-03-17", "amazon:5170075308", "promise:bwb_daily_pallets_2022-12-12", "ia:lesaventuresdeto0000tolk_t0n6", "promise:bwb_daily_pallets_2022-08-30:KR-117-775", "amazon:0007927045", "amazon:8412076540", "promise:bwb_daily_pallets_2022-09-07:W7-CXN-847", "amazon:0261103687", "amazon:912409546X", "bwb:9780344895463", "promise:bwb_daily_pallets_2022-08-02", "promise:bwb_daily_pallets_2022-04-01:O8-ADR-484", "amazon:8381164842", "promise:bwb_daily_pallets_2023-04-19:P8-BNX-316", "promise:bwb_daily_pallets_2023-04-11:W8-BRW-606", "promise:bwb_daily_pallets_2023-04-07:KS-199-194", "promise:bwb_daily_pallets_2023-03-31:KS-198-493", "promise:bwb_daily_pallets_2023-03-29:KS-198-041", "promise:bwb_daily_pallets_2023-05-15:KN-103-569", "marc:harvard_bibliographic_metadata/ab.bib.11.20150123.full.mrc:381341198:1041", "promise:bwb_daily_pallets_2023-06-01:KS-537-871", "promise:bwb_daily_pallets_2023-07-05:W8-BAY-330", "bwb:9780261103962", "promise:bwb_daily_pallets_2023-07-24:KS-589-214", "promise:bwb_daily_pallets_2023-08-21:W8-BVD-810", "promise:bwb_daily_pallets_2023-09-21:W8-BVW-542", "promise:bwb_daily_pallets_2023-10-10:W8-ADE-653", "amazon:505002255X", "bwb:9780008663162", "promise:bwb_daily_pallets_2023-10-19:O9-ASF-492", "marc:harvard_bibliographic_metadata/ab.bib.11.20150123.full.mrc:653596550:803", "promise:bwb_daily_pallets_2023-12-08:O9-CDZ-574", "promise:bwb_daily_pallets_2023-12-15:KT-657-103", "bwb:9780008669232", "promise:bwb_daily_pallets_2024-02-13:KT-307-140", "promise:bwb_daily_pallets_2024-03-19:KT-278-709", "promise:bwb_daily_pallets_2024-03-26:P9-DQU-862", "promise:bwb_daily_pallets_2022-04-20:KQ-937-250", "bwb:9780008705312", "marc:marc_columbia/Columbia-extract-20221130-010.mrc:238301614:1243", "bwb:9798855136401", "bwb:9781548547455", "bwb:9781530308248", "bwb:9781548547295", "ia:bwb_KQ-385-673", "ia:bwb_S0-AWJ-105", "ia:bwb_W9-DFW-842", "ia:foliosociety3vol0000unse", "ia:habbit0000jrrt", "ia:twotowersbiengse0000jrrt", "promise:bwb_daily_pallets_2021-04-01", "promise:bwb_daily_pallets_2024-12-03:KU-143-063", "bwb:9798786358774", "amazon:853361554X", "promise:bwb_daily_pallets_2024-11-26:KU-427-215", "promise:bwb_daily_pallets_2024-11-01:S0-ACE-347", "marc:harvard_bibliographic_metadata/20220215_036.bib.mrc:285447599:1475", "marc:UniversityOfMichiganMarcCatalogRecords/umich_bib.mrc:403884050:601"], "photos": [6155606, 6433524, 6433525, 6433526, 5731377, -1], "links": [{"title": "The Tolkien Society", "url": "http://www.tolkiensociety.org/index.html", "type": {"key": "/type/link"}}, {"title": "The Official Website of the JRR Tolkien Estate", "url": "http://www.tolkienestate.com/", "type": {"key": "/type/link"}}, {"title": "TheOneRing.com", "url": "http://www.theonering.com/", "type": {"key": "/type/link"}}, {"title": "Planet Tolkien", "url": "http://www.planet-tolkien.com/", "type": {"key": "/type/link"}}, {"title": "The Tolkien Library: Collecting Tolkien books", "url": "http://www.tolkienlibrary.com/", "type": {"key": "/type/link"}}, {"title": "Tolkien Gateway - The J.R.R. Tolkien encyclopedia", "url": "https://tolkiengateway.net", "type": {"key": "/type/link"}}], "name": "J.R.R. Tolkien", "key": "/authors/OL26320A", "entity_type": "person", "bio": "John Ronald Reuel Tolkien (1892-1973) was a major scholar of the English language, specialising in Old and Middle English. Twice Professor of Anglo-Saxon (Old EnLine truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"name": "Joanne S. Williamson", "personal_name": "Joanne S. Williamson", "last_modified": {"type": "/type/datetime", "value": "2008-09-07 10:49:48.04627"}, "key": "/authors/OL244960A", "type": {"key": "/type/author"}, "id": 640059, "revision": 2}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"description": "\"In the Tale of The Fall of Gondolin are two of the greatest powers in the world. There is Morgoth of the uttermost evil, unseen in this story but ruling over a vast military power from his fortress of Angband. Deeply opposed to Morgoth is Ulmo, second in might only to Manw\u00eb, chief of the Valar: he is called the Lord of Waters, of all seas, lakes, and rivers under the sky. But he works in secret in Middle-earth to support the Noldor, the kindred of the Elves among whom were numbered H\u00farin and T\u00farin Turambar. Central to this enmity of the gods is the city of Gondolin, beautiful but undiscoverable. It was built and peopled by Noldorin Elves who, when they dwelt in Valinor, the land of the gods, rebelled against their rule and fled to Middle-earth. Turgon King of Gondolin is hated and feared above all his enemies by Morgoth, who seeks in vain to discover the marvellously hidden city, while the gods in Valinor in heated debate largely refuse to intervene in support of Ulmo's desires and designs. Into this world comes Tuor, cousin of T\u00farin, the instrument of Ulmo's designs. Guided unseen by him Tuor sets out from the land of his birth on the fearful journey to Gondolin, and in one of the most arresting moments in the history of Middle-earth the sea-god himself appears to him, rising out of the ocean in the midst of a storm. In Gondolin he becomes great; he is wedded to Idril, Turgon's daughter, and their son is E\u00e4rendel, whose birth and profound importance in days to come is foreseen by Ulmo. At last comes the terrible ending. Morgoth learns through an act of supreme treachery all that he needs to mount a devastating attack on the city, with Balrogs and dragons and numberless Orcs. After a minutely observed account of the fall of Gondolin, the tale ends with the escape of T\u00farin and Idril, with the child E\u00e4rendel, looking back from a cleft in the mountains as they flee southward, at the blazing wreckage of their city. They were journeying into a new story, the Tale of E\u00e4rendel, which Tolkien never wrote, but which is sketched out in this book from other sources. Following his presentation of Beren and L\u00fathien Christopher Tolkien has used the same 'history in sequence' mode in the writing of this edition of The Fall of Gondolin. In the words of J.R.R. Tolkien, it was 'the first real story of this imaginary world' and, together with Beren and L\u00fathien and The Children of H\u00farin, he regarded it as one of the three 'Great Tales' of the Elder Days.\" - Amazon.com", "local_id": ["urn:sfpl:31223125471871", "urn:sfpl:31223125472390", "urn:sfpl:31223125472317", "urn:sfpl:31223125471996", "urn:sfpl:31223125471921", "urn:sfpl:31223125472143", "urn:sfpl:31223125472069", "urn:sfpl:31223125472473", "urn:sfpl:31223125471970", "urn:sfpl:31223125471988", "urn:sfpl:31223124987539", "urn:sfpl:31223124987562", "urn:sfpl:31223124987554", "urn:sfpl:31223125471947", "urn:sfpl:31223125471954", "urn:sfpl:31223125472234", "urn:sfpl:31223125472077", "urn:sfpl:31223125472051", "urn:sfpl:31223125472382", "urn:sfpl:31223125472465", "urn:sfpl:31223125472150", "urn:sfpl:31223125471939", "urn:sfpl:31223125472309", "urn:sfpl:31223125472226", "urn:sfpl:31223125471855", "urn:sfpl:31223125471863", "urn:sfpl:31223125471962", "urn:sfpl:31223125471848", "urn:bwbsku:O7-BSQ-211"], "key": "/books/OL26961988M", "authors": [{"key": "/authors/OL26320A"}], "contributions": ["Tolkien, Christopher, editor", "Lee, Alan, illustrator"], "languages": [{"key": "/languages/eng"}], "pagination": "302 pages", "source_records": ["marc:marc_openlibraries_sanfranciscopubliclibrary/sfpl_chq_2018_12_24_run06.mrc:160111339:9415", "bwb:9781328613042", "promise:bwb_daily_pallets_2021-07-13", "marc:marc_columbia/Columbia-extract-20221130-028.mrc:9422547:1219", "marc:marc_nuls/NULS_PHC_180925.mrc:29920693:1162", "marc:harvard_bibliographic_metadata/20220215_034.bib.mrc:169983458:3735"], "title": "The fall of Gondolin", "notes": "Includes bibliographical references (pages 287-300).", "number_of_pages": 302, "edition_name": "First U.S. edition.", "subjects": ["Middle Earth (Imaginary place)", "Elves", "Fiction"], "publish_date": "2018", "publish_country": "mau", "by_statement": "by J.R.R. Tolkien ; edited by Christopher Tolkien ; with illustrations by Alan Lee", "works": [{"key": "/works/OL8997784W"}], "type": {"key": "/type/edition"}, "identifiers": {}, "isbn_10": ["1328613046"], "isbn_13": ["9781328613042"], "lccn": ["2018145861", "2018037911"], "oclc_numbers": ["1048595707", "1031309781"], "ocaid": "fallofgondolin0000tolk", "classifications": {}, "dewey_decimal_class": ["823/.912"], "lc_classifications": ["PR6039.O32 F36 2018", "PR6039.O32F354 2018", "PR6039.O32 F35 2018", "PR6039.O32 F354 2018"], "publishers": ["Houghton Mifflin Harcourt"], "covers": [12451486], "latest_revision": 10, "revision": 10, "created": {"type": "/type/datetime", "value": "2019-05-24T08:52:24.747977"}, "last_modified": {"type": "/tLine truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"notes": {"type": "/type/text", "value": "Includes indexes."}, "identifiers": {"goodreads": ["616823"], "librarything": ["34410"]}, "title": "The harp and laurel wreath", "subtitle": "poetry and dictation for the classical curriculum", "publish_date": "1999", "publishers": ["Ignatius Press"], "isbn_10": ["0898707161"], "covers": [698345, -1], "lc_classifications": ["PR1175.3 .H37 1999", "PR1175.3.H37 1999"], "publish_places": ["San Francisco"], "contributions": ["Berquist, Laura M."], "pagination": "493 p. :", "source_records": ["marc:marc_records_scriblio_net/part27.dat:104221239:832", "bwb:9780898707168", "marc:marc_loc_2016/BooksAll.2016.part27.utf8:50197729:832", "promise:bwb_daily_pallets_2023-04-21:T3-AZY-915"], "dewey_decimal_class": ["821.008"], "languages": [{"key": "/languages/eng"}], "lccn": ["98073638"], "subjects": ["Children's poetry, English.", "Children's poetry, American.", "Poetry -- Problems, exercises, etc."], "publish_country": "cau", "by_statement": "edited by Laura M. Berquist ; illustrated by Christopher J. Pelicano.", "oclc_numbers": ["41609032"], "type": {"key": "/type/edition"}, "local_id": ["urn:bwbsku:T3-AZY-915"], "ocaid": "isbn_9780898707168", "key": "/books/OL395070M", "number_of_pages": 493, "works": [{"key": "/works/OL19479056W"}], "latest_revision": 11, "revision": 11, "created": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "last_modified": {"type": "/type/datetime", "value": "2024-09-03T19:12:46.986154"}}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"publishers": ["Bethlehem Books"], "number_of_pages": 237, "description": {"type": "/type/text", "value": "In ca. 1200 B.C., Uriah the Hittite leaves his conquered homeland and, following his father's instruction, seeks refuge with an old family friend, eventually finding himself in a great battle between the Canaanite forces of Sisera and the Hebrew forces of Barak."}, "isbn_10": ["1883937388"], "subject_place": ["Palestine"], "pagination": "xvii, 237 p. ;", "covers": [930599], "lc_classifications": ["PZ7.W672 Hi 1999"], "key": "/books/OL395004M", "authors": [{"key": "/authors/OL244960A"}], "ocaid": "hittitewarrior00will", "publish_places": ["Warsaw, ND"], "subject_time": ["To 70 A.D."], "genres": ["Fiction.", "Juvenile fiction."], "source_records": ["ia:hittitewarrior00will", "bwb:9781883937386", "marc:marc_loc_2016/BooksAll.2016.part27.utf8:50130386:1337", "promise:bwb_daily_pallets_2022-11-10"], "title": "Hittite warrior", "dewey_decimal_class": ["[Fic]"], "notes": {"type": "/type/text", "value": "\"Ages 10-up\"--P. 4 of cover."}, "identifiers": {"goodreads": ["613497"], "librarything": ["33463"]}, "languages": [{"key": "/languages/eng"}], "lccn": ["98073485"], "subjects": ["Sisera (Biblical figure) -- Fiction.", "Barak (Biblical figure) -- Fiction.", "Bible O.T. -- History of Biblical events -- Juvenile Fiction.", "Bible O.T. -- History of Biblical events -- Fiction.", "Jews -- Palestine -- History -- To 70 A.D. -- Fiction.", "Canaanites -- Fiction.", "Palestine -- History -- To 70 A.D. -- Juvenile fiction.", "Palestine -- History -- To 70 A.D. -- Fiction."], "publish_date": "1999", "publish_country": "ndu", "by_statement": "Joanne Williamson.", "works": [{"key": "/works/OL2028038W"}], "type": {"key": "/type/edition"}, "local_id": ["urn:bwbsku:O8-BVF-683"], "latest_revision": 9, "revision": 9, "created": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "last_modified": {"type": "/type/datetime", "value": "2022-12-04T06:49:23.101355"}}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"publishers": ["Bethlehem Books", "Ignatius Press"], "number_of_pages": 246, "description": {"type": "/type/text", "value": "When his position at court is usurped by his half brother, a young Egyptian nobleman seeks to regain his standing through military victories against the enemy."}, "isbn_10": ["1883937671"], "subject_place": ["Egypt"], "pagination": "246 p. ;", "covers": [930617], "lc_classifications": ["PZ7.N82 Sh 2001"], "key": "/books/OL3958151M", "authors": [{"key": "/authors/OL27951A"}], "publish_places": ["Bathgate, N.D", "San Francisco"], "subject_time": ["To 332 B.C."], "languages": [{"key": "/languages/eng"}], "genres": ["Juvenile fiction.", "Fiction."], "title": "Shadow hawk", "dewey_decimal_class": ["[Fic]"], "identifiers": {"librarything": ["34114"], "goodreads": ["83108"]}, "edition_name": "1st Bethlehem Books ed.", "lccn": ["2001092576"], "subjects": ["Egypt -- History -- To 332 B.C. -- Juvenile fiction.", "Egypt -- History -- To 332 B.C. -- Fiction."], "publish_date": "2001", "publish_country": "ndu", "series": ["Living history library"], "by_statement": "Andre Norton.", "oclc_numbers": ["52047589"], "works": [{"key": "/works/OL473297W"}], "type": {"key": "/type/edition"}, "source_records": ["marc:marc_loc_2016/BooksAll.2016.part28.utf8:238848352:909", "ia:shadowhawk0000nort"], "ocaid": "shadowhawk0000nort", "latest_revision": 11, "revision": 11, "created": {"type": "/type/datetime", "value": "2008-04-01T03:28:50.625462"}, "last_modified": {"type": "/type/datetime", "value": "2025-06-19T17:18:13.216687"}}
|
||||||
@@ -1 +0,0 @@
|
|||||||
{"ISBN:9781883937386": {"url": "http://openlibrary.org/books/OL395004M/Hittite_warrior", "key": "/books/OL395004M", "title": "Hittite warrior", "authors": [{"url": "http://openlibrary.org/authors/OL244960A/Joanne_S._Williamson", "name": "Joanne S. Williamson"}], "number_of_pages": 237, "pagination": "xvii, 237 p. ;", "by_statement": "Joanne Williamson.", "identifiers": {"goodreads": ["613497"], "librarything": ["33463"], "isbn_10": ["1883937388"], "lccn": ["98073485"], "openlibrary": ["OL395004M"]}, "classifications": {"lc_classifications": ["PZ7.W672 Hi 1999"], "dewey_decimal_class": ["[Fic]"]}, "publishers": [{"name": "Bethlehem Books"}], "publish_places": [{"name": "Warsaw, ND"}], "publish_date": "1999", "subjects": [{"name": "Bible", "url": "https://openlibrary.org/subjects/bible"}, {"name": "Canaanites", "url": "https://openlibrary.org/subjects/canaanites"}, {"name": "Fiction", "url": "https://openlibrary.org/subjects/fiction"}, {"name": "History", "url": "https://openlibrary.org/subjects/history"}, {"name": "History of Biblical events", "url": "https://openlibrary.org/subjects/history_of_biblical_events"}, {"name": "Jews", "url": "https://openlibrary.org/subjects/jews"}, {"name": "Juvenile fiction", "url": "https://openlibrary.org/subjects/juvenile_fiction"}, {"name": "Hittites", "url": "https://openlibrary.org/subjects/hittites"}, {"name": "Middle east, history", "url": "https://openlibrary.org/subjects/middle_east,_history"}, {"name": "Juvenile Fiction", "url": "https://openlibrary.org/subjects/juvenile_fiction"}], "subject_places": [{"name": "Palestine", "url": "https://openlibrary.org/subjects/place:palestine"}], "subject_people": [{"name": "Barak (Biblical figure)", "url": "https://openlibrary.org/subjects/person:barak_(biblical_figure)"}, {"name": "Deborah (Biblical judge)", "url": "https://openlibrary.org/subjects/person:deborah_(biblical_judge)"}, {"name": "Sisera (Biblical figure)", "url": "https://openlibrary.org/subjects/person:sisera_(biblical_figure)"}], "subject_times": [{"name": "To 70 A.D.", "url": "https://openlibrary.org/subjects/time:to_70_a.d."}], "notes": "\"Ages 10-up\"--P. 4 of cover.", "ebooks": [{"preview_url": "https://archive.org/details/hittitewarrior00will", "availability": "borrow", "formats": {}, "borrow_url": "https://openlibrary.org/books/OL395004M/Hittite_warrior/borrow", "checkedout": true}], "cover": {"small": "https://covers.openlibrary.org/b/id/930599-S.jpg", "medium": "https://covers.openlibrary.org/b/id/930599-M.jpg", "large": "https://covers.openlibrary.org/b/id/930599-L.jpg"}}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
{}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
{"ISBN:9781883937676": {"url": "http://openlibrary.org/books/OL3958151M/Shadow_hawk", "key": "/books/OL3958151M", "title": "Shadow hawk", "authors": [{"url": "http://openlibrary.org/authors/OL27951A/Andre_Norton", "name": "Andre Norton"}], "number_of_pages": 246, "pagination": "246 p. ;", "by_statement": "Andre Norton.", "identifiers": {"librarything": ["34114"], "goodreads": ["83108"], "isbn_10": ["1883937671"], "lccn": ["2001092576"], "oclc": ["52047589"], "openlibrary": ["OL3958151M"]}, "classifications": {"lc_classifications": ["PZ7.N82 Sh 2001"], "dewey_decimal_class": ["[Fic]"]}, "publishers": [{"name": "Bethlehem Books"}, {"name": "Ignatius Press"}], "publish_places": [{"name": "Bathgate, N.D"}, {"name": "San Francisco"}], "publish_date": "2001", "subjects": [{"name": "History", "url": "https://openlibrary.org/subjects/history"}, {"name": "Juvenile fiction", "url": "https://openlibrary.org/subjects/juvenile_fiction"}, {"name": "Egypt in fiction", "url": "https://openlibrary.org/subjects/egypt_in_fiction"}, {"name": "Fiction", "url": "https://openlibrary.org/subjects/fiction"}, {"name": "Children's stories", "url": "https://openlibrary.org/subjects/children's_stories"}, {"name": "Egypt -- History -- To 332 B.C. -- Juvenile fiction.", "url": "https://openlibrary.org/subjects/egypt_--_history_--_to_332_b.c._--_juvenile_fiction."}, {"name": "Fiction, science fiction, general", "url": "https://openlibrary.org/subjects/fiction,_science_fiction,_general"}], "subject_places": [{"name": "Egypt", "url": "https://openlibrary.org/subjects/place:egypt"}], "subject_times": [{"name": "To 332 B.C.", "url": "https://openlibrary.org/subjects/time:to_332_b.c."}], "ebooks": [{"preview_url": "https://archive.org/details/shadowhawk0000nort", "availability": "borrow", "formats": {}, "borrow_url": "https://openlibrary.org/books/OL3958151M/Shadow_hawk/borrow", "checkedout": false}], "cover": {"small": "https://covers.openlibrary.org/b/id/930617-S.jpg", "medium": "https://covers.openlibrary.org/b/id/930617-M.jpg", "large": "https://covers.openlibrary.org/b/id/930617-L.jpg"}}}
|
|
||||||
@@ -1,26 +0,0 @@
|
|||||||
{
|
|
||||||
"ISBN:9780201558029": {
|
|
||||||
"publishers": [{"name": "Addison-Wesley Professional"}],
|
|
||||||
"title": "Concrete Mathematics",
|
|
||||||
"subtitle": "A Foundation for Computer Science",
|
|
||||||
"identifiers": {
|
|
||||||
"isbn_10": ["0201558025"],
|
|
||||||
"isbn_13": ["9780201558029"]
|
|
||||||
},
|
|
||||||
"authors": [
|
|
||||||
{"name": "Ronald L. Graham", "url": "https://openlibrary.org/authors/OL123456A"},
|
|
||||||
{"name": "Donald E. Knuth"}
|
|
||||||
],
|
|
||||||
"number_of_pages": 672,
|
|
||||||
"publish_date": "1994",
|
|
||||||
"cover": {
|
|
||||||
"small": "https://covers.openlibrary.org/b/id/675832-S.jpg",
|
|
||||||
"medium": "https://covers.openlibrary.org/b/id/675832-M.jpg",
|
|
||||||
"large": "https://covers.openlibrary.org/b/id/675832-L.jpg"
|
|
||||||
},
|
|
||||||
"key": "/books/OL1234567M",
|
|
||||||
"url": "https://openlibrary.org/books/OL1234567M",
|
|
||||||
"notes": "unused field to prove unknown keys are ignored",
|
|
||||||
"excerpts": [{"text": "some excerpt"}]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"created": {"type": "/type/datetime", "value": "2019-04-03T10:45:51.646607"}, "subjects": ["Problems, exercises", "Children's poetry, English", "Poetry", "Children's poetry, American", "Poetry (poetic works by one author)"], "latest_revision": 2, "key": "/works/OL19479056W", "title": "The harp and laurel wreath", "authors": [{"type": {"key": "/type/author_role"}, "author": {"key": "/authors/OL335577A"}}], "type": {"key": "/type/work"}, "last_modified": {"type": "/type/datetime", "value": "2020-08-23T12:34:39.612826"}, "covers": [698345], "revision": 2}
|
||||||
@@ -896,3 +896,66 @@ code: compare against the gate logs' "BUILD SUCCESSFUL in" times and ask the use
|
|||||||
|
|
||||||
`tasks/wave-chain.sh` now reads its commit subject from `tasks/<task>.subject`
|
`tasks/wave-chain.sh` now reads its commit subject from `tasks/<task>.subject`
|
||||||
(it was hard-coded for waves 9/10).
|
(it was hard-coded for waves 9/10).
|
||||||
|
|
||||||
|
## Wave 12 — Open Library endpoint move + Google Books cover fix: DONE 2026-09-20
|
||||||
|
Done **by the orchestrator directly, not a worker** — the user left the choice open
|
||||||
|
("do whatever you think will yield the best results"). The change is confined to
|
||||||
|
`data/metadata`, and the live API shapes had already been measured during diagnosis,
|
||||||
|
so a worker would have spent a session rediscovering them. Full write-up in
|
||||||
|
`docs/METADATA-SOURCES.md` § "The Open Library endpoint move".
|
||||||
|
|
||||||
|
**Two user-reported symptoms, both third parties answering misleadingly:**
|
||||||
|
1. `/api/books?bibkeys=...` now 404s for EVERY ISBN, including books OL still holds.
|
||||||
|
OL's docs call it the "Legacy Books API" that "may be phased out"; it is gone from
|
||||||
|
their API index. Treat as retired, not down. The app had been effectively
|
||||||
|
single-sourced on Google Books since it broke.
|
||||||
|
2. Google Books answers `zoom=2` with a grey "image not available" PNG at **HTTP 200**
|
||||||
|
for volumes with no full preview — 11 of 18 sampled. SPEC told us to force zoom=2.
|
||||||
|
|
||||||
|
**Changed:** `OpenLibraryClient` now uses `/isbn/{isbn}.json` (+ `/works/` fallback
|
||||||
|
for authors, + `/authors/` resolution via the new `AuthorNameCache`);
|
||||||
|
`normalizeCoverUrl` pins `zoom=1`, adds `w=400`, strips `edge=curl`. SPEC.md's
|
||||||
|
"Book metadata lookup" section rewritten with both rules and WHY, so nobody restores
|
||||||
|
either. Old-endpoint fixtures deleted; 9 new fixtures captured from the live API.
|
||||||
|
|
||||||
|
**Three things that would have been silent bugs** (all caught pre-ship, all pinned by
|
||||||
|
tests): an edition record can carry NO authors (9780898707168 — the user's own shelf —
|
||||||
|
has them only on the work, so a naive move drops the author); `covers` uses `-1` as a
|
||||||
|
no-cover sentinel; `description` is a bare string on some records and `{type,value}`
|
||||||
|
on others. Also: **on this endpoint 404 IS authoritative NotFound**, where on the old
|
||||||
|
one every non-2xx was a failure.
|
||||||
|
|
||||||
|
| Check | Result |
|
||||||
|
|---|---|
|
||||||
|
| `./tasks/gw assembleDebug` | exit 0 |
|
||||||
|
| `./tasks/gw testDebugUnitTest --rerun-tasks` | exit 0 — **378 tests**, 2 skipped, 0 failures (was 355) |
|
||||||
|
| `./tasks/gw verifyPaparazziDebug` | exit 0 — no pixels moved |
|
||||||
|
| `LIVE_METADATA=1` live test | **RAN, not skipped** — 9/9 Found with cover art, keyless GB, so OL alone answered |
|
||||||
|
| `grep "always 'false'"` | 0 hits |
|
||||||
|
|
||||||
|
### HAZARD #13 — two latent test races, surfaced by adding tests
|
||||||
|
Adding ~20 tests shifted suite timing and made two pre-existing races start flapping.
|
||||||
|
Neither was caused by the metadata change; both flake in whichever test happens to be
|
||||||
|
running when a window expires, NOT in the one at fault. Do not chase the named test.
|
||||||
|
|
||||||
|
1. **FIXED. `LibraryViewModelTest` leaked view models.** `LibraryViewModel` has eleven
|
||||||
|
`stateIn(viewModelScope, WhileSubscribed(5_000), ...)` flows, so its upstreams keep
|
||||||
|
running five seconds after the last collector. Nothing ever cleared the VM, so they
|
||||||
|
were still touching `Dispatchers.Main` while the NEXT test's tearDown called
|
||||||
|
`resetMain()` -> "Dispatchers.Main is used concurrently with setting it". The test
|
||||||
|
now tracks every VM it builds and cancels `viewModelScope` in tearDown (and guards
|
||||||
|
`db.close()` with `::db.isInitialized`, since a failed setUp otherwise masks the
|
||||||
|
real failure with an UninitializedPropertyAccessException).
|
||||||
|
2. **NOT FIXED — reported to the user, out of scope.** `AddBookViewModel.performSave`
|
||||||
|
has a check-then-act in-flight guard: `if (_formState.value.isSaving) return null`
|
||||||
|
and only then `update { isSaving = true }`. Two coroutines can both pass the check,
|
||||||
|
which is what `a second save while one is in flight does not create a second book`
|
||||||
|
catches when timing allows. The KDoc right above it claims a double-tap "still
|
||||||
|
can't create two books" — that claim is false today. `AddBookViewModel` takes no
|
||||||
|
metadata dependency at all, so this is provably unrelated to wave 12. One atomic
|
||||||
|
`getAndUpdate` fixes it.
|
||||||
|
|
||||||
|
**Lesson:** a green suite on this project is worth one re-run before you trust it, and
|
||||||
|
`--rerun-tasks` is mandatory — a plain `testDebugUnitTest` after a stash happily
|
||||||
|
reports BUILD SUCCESSFUL `FROM-CACHE` without executing a single test. That nearly
|
||||||
|
produced a false "pre-existing, not mine" conclusion here.
|
||||||
@@ -350,3 +350,122 @@ The 17% of books with no cover art anywhere is unchanged, and so is Open
|
|||||||
Library's ~13% transport failure rate. What changes is that a failure of one
|
Library's ~13% transport failure rate. What changes is that a failure of one
|
||||||
source is now much more likely to be covered by the other instead of surfacing
|
source is now much more likely to be covered by the other instead of surfacing
|
||||||
as `Unavailable`.
|
as `Unavailable`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The Open Library endpoint move — 2026-09-20
|
||||||
|
|
||||||
|
The user came back from a round of shelf testing with two symptoms. Both had the
|
||||||
|
same character: a third party answering 200-with-something-useless, or 404-without-
|
||||||
|
meaning-it, and the app believing it.
|
||||||
|
|
||||||
|
### 1. `/api/books` is answering 404 to everything
|
||||||
|
|
||||||
|
Reported symptom: intermittent "ISBN 9781328613042 — one or more sources couldn't be
|
||||||
|
reached … open library: http 404".
|
||||||
|
|
||||||
|
Measured from this box, and independently reproduced from the user's phone network:
|
||||||
|
|
||||||
|
| Request | Result |
|
||||||
|
|---|---|
|
||||||
|
| `/api/books?bibkeys=ISBN:9781328613042` (*The Fall of Gondolin*) | **404, 0 bytes** |
|
||||||
|
| `/api/books?bibkeys=ISBN:9780140328721` (*Fantastic Mr Fox*) | **404, 0 bytes** |
|
||||||
|
| `/api/books?bibkeys=OLID:OL1017798M` | **404, 0 bytes** |
|
||||||
|
| same, `jscmd=viewapi` / no `jscmd` | **404** |
|
||||||
|
| same, browser UA / descriptive UA / `okhttp` UA / HTTP/1.1 | **404** |
|
||||||
|
| `/isbn/9781328613042.json` | 200 — full edition record |
|
||||||
|
| `/api/volumes/brief/isbn/9781328613042.json` | 200 — edition `OL26961988M` |
|
||||||
|
| `/search.json?q=isbn:…` | 200 |
|
||||||
|
| `covers.openlibrary.org/b/isbn/…` | 200 |
|
||||||
|
|
||||||
|
The 404 carries `content-type: application/json` and OL's own `x-ol-stats` header, so
|
||||||
|
it is OL's application answering, not a CDN error page. **Open Library still holds the
|
||||||
|
book** — only that one door is shut.
|
||||||
|
|
||||||
|
**This was first written up as an outage. That was probably wrong.** OL's docs
|
||||||
|
(`/dev/docs/api/books`) call `/api/books` the "Legacy Books API" and say "Please
|
||||||
|
consider using the Book Search API above; this is a legacy endpoint and may be phased
|
||||||
|
out in the future", and it does not appear in their API index at all. A blanket,
|
||||||
|
header-independent 404 while every neighbouring endpoint serves normally fits a
|
||||||
|
retirement. There is no 410 and no `Sunset` header, so intent can't be proven from
|
||||||
|
outside — but "wait for it to come back" was never a plan either way.
|
||||||
|
|
||||||
|
**Consequence while it lasted: the app was single-sourced on Google Books.** Every
|
||||||
|
lookup that reached the user as a failure was a book Google Books doesn't have.
|
||||||
|
9781328613042 is exactly that — Google Books returns `totalItems: 0` for it, with the
|
||||||
|
key. So the error message named only Open Library because `combine()` names only
|
||||||
|
sources that *failed*, and Google Books had answered authoritatively.
|
||||||
|
|
||||||
|
### Which endpoint replaces it
|
||||||
|
|
||||||
|
| | `/api/volumes/brief` (legacy) | **`/isbn/{isbn}.json`** (chosen) | `search.json` (OL's own suggestion) |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Requests per lookup | 1 | 1 + 1 per author | 1 |
|
||||||
|
| Title | edition | **edition** | *work* title |
|
||||||
|
| Author names | inline | **refs — needs a fetch** | inline |
|
||||||
|
| Publisher | edition | **edition** | all editions mashed together |
|
||||||
|
| Year | 2018 (this edition) | **2018** | 1985 (first ed. of the work) |
|
||||||
|
| Description | — | **yes, when present** | — |
|
||||||
|
| Cover | `cover` object | **`covers: [id]`** | `cover_i` |
|
||||||
|
| Status | **legacy** | current | current |
|
||||||
|
|
||||||
|
`search.json` is work-level: for *The Fall of Gondolin* it reports 1985 and eight
|
||||||
|
publishers, which is wrong for a catalogue of specific editions someone owns.
|
||||||
|
`/api/volumes/brief` is the least work — its `data` block is nearly the old
|
||||||
|
`jscmd=data` shape — and that is the trap: swapping a retired legacy endpoint for
|
||||||
|
another legacy endpoint buys one migration and no safety.
|
||||||
|
|
||||||
|
### What the Editions API costs, and two things that would have been silent bugs
|
||||||
|
|
||||||
|
- **`authors` are references, not names.** Hence `AuthorNameCache` (process-lifetime,
|
||||||
|
unbounded by design: hundreds of short strings at most). Books by one author get
|
||||||
|
scanned in runs off one shelf, so the cache hits constantly.
|
||||||
|
- **An edition can carry no authors at all.** 9780898707168 — *The Harp and Laurel
|
||||||
|
Wreath*, on the user's own shelf — has `authors: None`; they exist only on the work.
|
||||||
|
The legacy endpoint resolved that and returned "Laura M. Berquist". Without a
|
||||||
|
work-level fallback the migration would have silently dropped the author for books
|
||||||
|
like it. Caught before shipping, and pinned by a test.
|
||||||
|
- **`covers` uses `-1` as a "no cover" sentinel** (9780898707168 -> `[698345, -1]`,
|
||||||
|
9780140328721 -> `[15152634, 8739161, -1]`). Taking `.first()` would eventually
|
||||||
|
build `b/id/-1-L.jpg`. Take the first positive id.
|
||||||
|
- **`description` is a bare string on some records and `{type, value}` on others**,
|
||||||
|
and absent on most: of 15 real editions sampled, 1 string, 5 objects, 9 absent.
|
||||||
|
- **404 now means NotFound.** The legacy endpoint reported a miss as `200 {}`, so
|
||||||
|
every non-2xx there was rightly a failure. This endpoint 404s instead, and that is
|
||||||
|
authoritative — reporting it as Unavailable would offer a retry for a book no
|
||||||
|
amount of retrying will find. Every other non-2xx is still Unavailable.
|
||||||
|
|
||||||
|
### 2. Google Books serves a placeholder image at `zoom=2`
|
||||||
|
|
||||||
|
Reported symptom: two different cover placeholders — the app's own two-tone/gold one,
|
||||||
|
and "an ugly gray-text-on-white saying image not available".
|
||||||
|
|
||||||
|
The second is Google's, and the app was asking for it. SPEC said to force `zoom=2` on
|
||||||
|
GB `imageLinks`; Google's `thumbnail` is `zoom=1`. For a volume Google has no full
|
||||||
|
preview of — the metadata-only `…AAAACAAJ` records, i.e. most small-press and older
|
||||||
|
material — **`zoom=2` is not a valid rendition and Google substitutes a placeholder
|
||||||
|
with HTTP 200** instead of 404ing. Coil loads it as a success, so `BookCover`'s
|
||||||
|
placeholder never fires, and the cover pipeline uploads it to PocketBase as the cover.
|
||||||
|
|
||||||
|
Measured over 18 volumes that have `imageLinks`:
|
||||||
|
|
||||||
|
| URL form | real cover | "image not available" |
|
||||||
|
|---|---|---|
|
||||||
|
| `zoom=2` (what the app sent) | 7 | **11** |
|
||||||
|
| `zoom=1&w=400`, `edge=curl` stripped | **18** | 0 |
|
||||||
|
|
||||||
|
The 11 were byte-identical: 15,567 bytes, md5 `c96309220b9cbd205c36d879d09a3647`.
|
||||||
|
`zoom=0`, `3` and `6` return the same artwork at 575x750 and 1280x1670 — so there is
|
||||||
|
no bigger-zoom escape, and hash-detection would mean chasing renditions forever.
|
||||||
|
`w=` is what buys resolution: Google honours it up to the source scan's native width.
|
||||||
|
|
||||||
|
**Confirmed in the live library, not just on screen.** *Abraham Lincoln's World*
|
||||||
|
(`9plhmyj5s68f58e`) has a `zoom=2` `cover_source_url`, and the file stored on
|
||||||
|
PocketBase for it is that exact placeholder — 15,567 bytes, same md5, served as
|
||||||
|
`.jpg`. The user is handling the already-poisoned covers themselves.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
`LiveMetadataLookupTest` (opt-in, `LIVE_METADATA=1`) against the rewritten client:
|
||||||
|
9 lookups, all **Found with cover art**, 1.7s–4.9s — and with the Google Books key
|
||||||
|
absent, so that is Open Library alone answering through the new endpoint.
|
||||||
+36
-3
@@ -102,14 +102,44 @@ Trigger: app start, manual pull-to-refresh, WorkManager periodic (~6h, network-c
|
|||||||
Never let sync failure surface as a crash or a blocking dialog — a quiet status line only.
|
Never let sync failure surface as a crash or a blocking dialog — a quiet status line only.
|
||||||
|
|
||||||
## Book metadata lookup
|
## Book metadata lookup
|
||||||
Primary Open Library: https://openlibrary.org/api/books?bibkeys=ISBN:{isbn}&format=json&jscmd=data
|
Primary Open Library EDITIONS API: https://openlibrary.org/isbn/{isbn}.json (302 -> the edition record)
|
||||||
Fallback Google Books: https://www.googleapis.com/books/v1/volumes?q=isbn:{isbn} (no key)
|
Fallback Google Books: https://www.googleapis.com/books/v1/volumes?q=isbn:{isbn} (keyed)
|
||||||
Cover: https://covers.openlibrary.org/b/isbn/{isbn}-L.jpg else GB imageLinks (force https, zoom=2)
|
|
||||||
Merge: prefer whichever has a title; fill blanks from the other.
|
Merge: prefer whichever has a title; fill blanks from the other.
|
||||||
|
|
||||||
|
DO NOT go back to `/api/books?bibkeys=...&jscmd=data`. On 2026-09-20 it was found
|
||||||
|
answering 404 to EVERY bibkey form for EVERY ISBN — including books Open Library
|
||||||
|
demonstrably still holds — while /isbn/, /search.json, /api/volumes/brief and
|
||||||
|
covers.openlibrary.org all served normally. OL's own docs call it the "Legacy Books
|
||||||
|
API" and say it "may be phased out in the future", and it is absent from their API
|
||||||
|
index, so treat it as retired. /api/volumes/brief (the "Legacy Partner API") is a
|
||||||
|
near drop-in for the old response shape and is ALSO legacy — that is why it was not
|
||||||
|
chosen. See docs/METADATA-SOURCES.md.
|
||||||
|
|
||||||
|
The Editions API costs extra requests, by design:
|
||||||
|
- `authors` are REFERENCES (`{"key": "/authors/OL26320A"}`), not names; each needs
|
||||||
|
a GET of /authors/{id}.json. `AuthorNameCache` holds resolved names for the
|
||||||
|
process lifetime — books by one author are scanned in runs off one shelf.
|
||||||
|
- an edition record may carry NO authors at all (real: 9780898707168), in which
|
||||||
|
case they are on the work; fall back to /works/{id}.json. Losing this fallback
|
||||||
|
silently drops the author for such books.
|
||||||
|
- author/work requests are BEST-EFFORT: the edition request already succeeded, so
|
||||||
|
a failure there degrades the record, it must never turn Found into Unavailable.
|
||||||
|
|
||||||
|
Cover: OL `covers: [id]` -> https://covers.openlibrary.org/b/id/{id}-L.jpg
|
||||||
|
else GB imageLinks (force https, **zoom=1**, add `w=400`, strip `edge=curl`)
|
||||||
Cover URL comes from a source that REPORTS one. Never synthesize the by-ISBN cover
|
Cover URL comes from a source that REPORTS one. Never synthesize the by-ISBN cover
|
||||||
URL as if it were evidence: for an edition with no art that endpoint returns 200 +
|
URL as if it were evidence: for an edition with no art that endpoint returns 200 +
|
||||||
a 43-byte 1x1 transparent GIF, which loads "successfully" and paints nothing. As a
|
a 43-byte 1x1 transparent GIF, which loads "successfully" and paints nothing. As a
|
||||||
last resort it may be used only with `?default=false`, which makes a miss a 404.
|
last resort it may be used only with `?default=false`, which makes a miss a 404.
|
||||||
|
OL's `covers` array uses -1 as a "no cover here" sentinel; take the first POSITIVE id.
|
||||||
|
|
||||||
|
DO NOT force `zoom=2` on a Google Books cover (this spec said to, and was wrong).
|
||||||
|
For a volume Google holds no full preview of, zoom=2 is not a valid rendition and
|
||||||
|
Google answers 200 with a grey "image not available" PNG rather than a 404 — which
|
||||||
|
loads successfully, so BookCover's placeholder never fires and the cover pipeline
|
||||||
|
uploads Google's placeholder to PocketBase as the book's cover. Measured 2026-09-20:
|
||||||
|
11 of 18 real volumes did this at zoom=2; 0 of 18 at zoom=1 with a width. zoom=0/3/6
|
||||||
|
are placeholders too, so there is no larger-zoom escape.
|
||||||
|
|
||||||
Lookup outcome is THREE-WAY, never a bare null. A source that could not be reached
|
Lookup outcome is THREE-WAY, never a bare null. A source that could not be reached
|
||||||
must never be reported to the user as a book that does not exist:
|
must never be reported to the user as a book that does not exist:
|
||||||
@@ -117,6 +147,9 @@ must never be reported to the user as a book that does not exist:
|
|||||||
NotFound - EVERY source answered authoritatively and none had it
|
NotFound - EVERY source answered authoritatively and none had it
|
||||||
Unavailable - no source could be reached (non-2xx, timeout, transport error)
|
Unavailable - no source could be reached (non-2xx, timeout, transport error)
|
||||||
and none of the reachable ones had it
|
and none of the reachable ones had it
|
||||||
|
On the Editions API a 404 IS authoritative NotFound — unlike the legacy endpoint,
|
||||||
|
which reported a miss as 200 with an empty object and so made every non-2xx a
|
||||||
|
failure. Every OTHER non-2xx remains Unavailable.
|
||||||
UI: Found -> the save sheet. NotFound -> manual entry pre-filled with the scanned
|
UI: Found -> the save sheet. NotFound -> manual entry pre-filled with the scanned
|
||||||
ISBN. Unavailable -> a retry affordance, with manual entry as the escape hatch;
|
ISBN. Unavailable -> a retry affordance, with manual entry as the escape hatch;
|
||||||
it must NOT claim the book is unknown.
|
it must NOT claim the book is unknown.
|
||||||
|
|||||||
Reference in new issue
Block a user