Retrieving Document
Images Under
Domain-Specific
Constraints

The Aristotelian AI Problem

On the three-way tension between discourse, truth, and measure — and why historical documents fundamentally resist large-scale pretraining.

Persuasion vs. the short, answerable question

The tension this thesis confronts is twenty-four centuries old. In Plato's dialogue, the sophist Protagoras answers in makrologia — long, beautiful speeches that persuade without ever being pinned down. Socrates keeps interrupting to demand brachylogia: short answers, one question at a time, each one checkable. Toggle the two modes and watch what happens to persuasion versus knowledge actually established.

Rhetoric maximises persuasion while establishing almost nothing; dialectic trades eloquence for things you can actually check. The modern chatbot is Protagoras with a server farm.

From the agora to the search bar

Each era leaned on a different corner of the same triangle. Click along the timeline: the sophists sold discourse; dialectic and, later, the scientific record chased truth; the index, the archive, and the search engine industrialised measure; the LLM looped back to discourse at planetary scale. The thesis asks for all three at once.

Discourse, Truth, Measure

Two and a half millennia after Socrates asked the sophist Protagoras for brachylogia — short, specific answers instead of persuasive rhetoric — we have not escaped the same three-way pull. Drag the marker: every information technology sits somewhere inside this triangle. The chatbot leans into discourse; the classic search engine offers measure without relation; the thesis asks whether a system can hold all three at once.

Why the chatbot feels more certain

A search engine leaves you certain that you did not read the thousands of pages it indexed — your doubt stays visible. A conversational model hides its sources behind a single comforting answer; keep asking and your perceived uncertainty falls asymptotically toward zero. Press "Ask again" on each side and watch the doubt behave very differently.

The "omission bias" of a sophistic AI: it is comforting precisely because it never shows you the long tail of what it left out.

A historical truth is only as precise as the sources retrieved

As John Keating put it, truth is like a blanket that always leaves your feet cold — you tug it up to your chin and your feet stick out; pull it down and you shiver at the neck. The blanket of evidence is finite: cover one part of an event and another slips back into the cold. Grab the quilt and drag it over the sleeping figure of "the past" — try to cover all of it at once.

Two depictions of the same expansion

The same westward expansion, painted twice. John Gast's American Progress (1872) bathes it in the light of Manifest Destiny; Robert Lindneux's Trail of Tears (1942) shows a people displaced. With only one source as context, the map of history is distorted. Toggle the second source and watch the reconstructed "map of events" correct itself — drawing the map of history is choosing which truths to sample.

SOURCE A · 1872

American Progress

Native Americans obscured by an advancing light.

● included

SOURCE B · 1942

Trail of Tears

R. Lindneux, 1942 · Woolaroc Museum

A multitude marching in melancholy.

○ click to retrieve

What makes a document "historical"? — drag the slider

The anatomy of a document

A document is a record of human activity fixed on a durable medium. The Manual of Archival Science separates it into three layers: the material support that gives it physical body, the record that captures information onto that support, and the information itself — the shapes, words, and people confined within. Hover each layer, then explode the stack to see them apart.

In this thesis, both photographs and textual documents count as documents — as long as the context of application is archival.

Three constraints that block standard pretraining — click each

Four Research Questions

Thesis Structure

Document, Image &
Information Retrieval

Retrieval is the task of identifying relevant information in a corpus, conditioned on a signal. Before the contributions, the shared vocabulary: queries, ranking, and the long tail that breaks conversational shortcuts.

A signal pulls relevance out of a collection

Every retrieval system maps a query-signal onto a ranking of documents. Type a query, or pick one, and watch the corpus reorganise: relevant items rise, the threshold τ draws a line, and the long tail of almost-relevant material trails off below it — the very tail a chatbot quietly hides.

The two ways to be wrong

Move the threshold above and the two classic measures pull against each other. A loose threshold retrieves everything — high recall, low precision. A strict one is pure but blind to the tail. The thesis argues a historical engine must, above all, acknowledge the absence of false negatives — knowing what it failed to find.

Controllability & Symbolic Retrieval

What if instead of asking "how similar?" we asked "does this satisfy these logical conditions?" A formal model-checking framework applied to visual search.

Checking a claim against an image, line by line

The query (man, riding, horse) compiles to a small program — a visual grammar. Instead of asking "how similar?", the routine executes: it locates objects, tests spatial predicates, and where the world contradicts the specification it records doubt rather than forcing a match. Step through the routine, or press play.

Doubt is a feature: a partial match is ranked, not discarded — and the system can state exactly which predicate failed.

Live Retrieval Space — click to place a query, scroll wheel changes threshold τ

Model Checking — match the query graph against a candidate image

GraphRAG on Noisy Historical Records

The second contribution uses word spotting to make Cypher graph queries robust against HTR errors. When a model reads Savadell instead of Sabadell, exact matching fails completely — word spotting expands literals to clusters of near-spellings.

From a sentence to a verified answer over a noisy graph

A request becomes a Cypher query over a genealogical graph built from handwritten parish records. The model decides which literals are too fragile for exact matching — names and places mangled by HTR — and wraps those in a word-spotting expansion, so the query runs over the expanded graph without losing its grounding. Step through it.

How a word becomes a position-aware fingerprint

Word spotting compares words by a descriptor, not by their literal characters. The PHOC records which letters occur and, recursively, where — the whole word, then halves, then thirds. Two OCR variants of the same word land close together in this space. Type a word and watch its pyramid build.

COCO Benchmark — Recall Results

On the Hard split — compositionality, counting, text recognition — the model-checking approach shows its clearest advantage over pure embedding methods.

Model Editing & Distributed Learning

When data cannot be centralised — because of privacy, governance, or cultural heritage law — can federated model averaging still create a generalist foundation?

Federation Simulation — click an archive to train it, then click "Aggregate"

Task Arithmetic — drag models to compose weights

Accuracy on Unseen Scripts — Centralised vs Distributed

Decoding what no single archive could

The 18th-century Borg cipher defeats a centrally pretrained model entirely — 0% character accuracy, because no training corpus ever contained its invented glyphs. Add federated experts: each archive contributes a sliver of weight, and the merged model begins to read symbols none of them individually knew. Drag to recruit archives and watch the ciphertext resolve.

Temporally Decomposable
Image Representations

Separating what an object is from when it existed — and using that orthogonal decomposition to compose entirely novel retrieval queries.

Three Composed Retrieval Modes — click to explore

Separate what from when — then transplant time

A single embedding tangles an object's category with its era. The training pipeline pulls them apart: a proxy loss builds year-agnostic category clusters, subtracting the category mean µᶜ centres every cluster into one shared temporal subspace, and the leftover residual is the era itself. The swapping trick then translates that residual onto another object — letting you ask "this jacket, as it would look in 1950," a query no label could phrase. Step through the decomposition.

Because the two subspaces are orthogonal by construction, a 1940 car and a 1990 car stay semantically linked — proven via mutual information, not merely hoped for.

Embedding Space — compose a query interactively

Date Precision@10 — TDIR vs CLIP

CLIP's temporal signal is present but not compositionally accessible. TDIR is over 8× more precise in date retrieval because the orthogonal decomposition is learned explicitly.

Computer-Vision Aided Heritage Databases

Two case studies: legislative responses to 19th-century epidemics, and the invisible female labour force revealed through historical photography.

Photo-BALL Database (1870–2020) — click a decade

Developing the people the tallies left in totals

Aggregate occupational statistics describe people in coarse sums; a photograph shows individuals. Photo-BALL detects and counts the subjects of a large dated image collection, so the visual record can surface a presence that summary tables flatten. Drag to "develop" this illustrative factory floor: as the print emerges, detections tag each worker, and the women a headline total would fold away glow gold.

Photo-BALL ran 7.2M object detections across 1M+ dated images — the scale at which dispersed photographs become a quantifiable, queryable record. (Figures above are illustrative of the method.)

Subjects Detected Over Time — an illustrative reading of the visual record

Epidemic Legislation Analysis

Word spotting on two centuries of Spanish official journals reveals how epidemic cycles — yellow fever, the 1820 plague in Mallorca, the great cholera waves of 1833–1885, and the 1918 influenza — drove the construction of the modern liberal state. Legislative language around sanidad, cuarentena, cordón sanitario spikes precisely at epidemic inflection points.

Which words trend with an epidemic?

Word spotting traced ~1.2 billion word instances across two centuries of Spanish official journals. To find vocabulary genuinely tied to cholera, each candidate word's yearly frequency is compared against the anchor "cólera" by histogram intersection — a score from 0 (never co-occurs) to 1 (perfectly in step). Pick a candidate and watch the two curves overlap.

India → Europe → Spain, fifteen years in the gazette

By measuring the log Z-score — the spike in mentions of a place above its own historical baseline — the gazette reveals how Spain learned of cholera long before it arrived. The disease is reported in India in 1818, becomes a European preoccupation by 1830, and reaches the peninsula in 1833. Step through the three stages and read what the administration knew.

Each epidemic left the state more legislative

Categorising two centuries of language as description (narrating the outbreak) versus measure (prescribing action) reveals the modern liberal state being built one crisis at a time. Scrub through the years: during an outbreak description spikes, but the baseline share of prescriptive "measure" language ratchets up and never fully recedes.