A call for decentralized Document Analysis — where keeping data apart is not a tax on accuracy, but a route to better generalization.
Privacy constraints keep archives, governments, and small businesses from pooling document data. Federated learning is usually treated as a necessary evil — a price paid for privacy. This paper argues the opposite: train models separately on each silo, average their weights, and the merged model often generalizes better — especially out-of-distribution and in low-resource regimes.
Centralized vs distributed — drag the slider
This is the first study to evaluate distributed learning in Document Analysis along three axes at once: the tasks (word spotting, handwriting/OCR, table recognition), the architectures (RNNs, transformers, graph networks), and the fine-tuning strategies (distillation, layer-wise, end-to-end).
Modern training assumes you can pile all the data in one place. For documents — sensitive administrative records, cultural-heritage collections meant to stay near their origin, confidential SME paperwork — that assumption simply fails. Distributed learning trains where the data lives and shares only the resulting weights.
"Decentralization is not merely a constraint, but a valuable opportunity to improve model robustness and adaptability."
A model is just a vector of weights θ ∈ ℝm. Train it on the whole dataset and you get the centralized θD. Train one model per silo and average them — that's the distributed θ̄D. The data never leaves its silo; only the weight vectors are shared.
The paper's footnote makes a quiet but central claim: FedAvg, model soups, task arithmetic and MAML are the same move — train separately from a shared init θ⁰, then combine in weight space. Here is that move at the level of individual parameters. Each silo is a strip of weights; the merge is the column-wise mean. Switch how you frame the combination — the resulting vector is bit-for-bit identical.
A task vector τi = θdᵢ − θ⁰ is what one silo learned on top of the shared start. Add the mean of the task vectors back onto θ⁰ and the θ⁰ terms cancel — you land on the plain average. That algebraic cancellation is why the whole family collapses to Eq. 3, and why a result proven for one (e.g. model soups) transfers to the others.
This is weight space. The shared seed θ⁰ sits on the left; each silo trains to its own optimum θdᵢ. Switch how you read the merge: as models (the centroid of the optima — model soup / FedAvg) or as task vectors (sum the arrows τᵢ = θdᵢ − θ⁰ from the seed — task arithmetic). The orange ★ does not move: the seed cancels, so both are literally the same vector.
A common objection: surely n separate trainings cost more? No. Centralized sees N points for T epochs = N·T steps. Splitting N = n₁+n₂+… into silos gives n₁·T + n₂·T + … = (n₁+n₂+…)·T = N·T. Product is distributive — the budgets are identical. Drag to re-split the data.
The same merged weights are tested through three fine-tuning regimes spanning the breadth of Document Analysis. Pick a strategy:
Datasets span in-pretraining Latin, out-of-domain Latin (artistic fonts, glares, printed distractors), and out-of-vocabulary scripts (Arabic, Chinese, Japanese, Korean, Hindi, Bangla) plus historical ciphers (Copiale, Borg, Vatican). Protocol: train each silo for the same steps as centralized, merge (Eq. 3), then fine-tune 25 epochs on a held-out target and evaluate on its test split.
In-domain, results are mixed (a small +0.5% on average). But out-of-domain Latin jumps +36.7%, and on unseen alphabets the relative mAP gain is +104% — distributed pretraining shines exactly where generalization is hardest.
Personalizing to an unseen script: distributed pretraining beats centralized by ×1.62 on average. The Borg cipher is the extreme — centralized fails outright (0.000), distributed reaches 0.505.
A leave-one-out evaluation. With no target adaptation (zero-shot) the distributed merge leads; as you grant more adaptation (partial → full end-to-end), the advantage shifts to centralized. Pick a regime:
The paper's title is a targeted call. Distributed learning is not a universal winner — it dominates in specific regimes. Describe your situation and see which side the evidence favors.
Largest gains on the smallest data (M96, 50 pages) and on independently-learned new alphabets — the gap narrows once data is abundant.
Strongest zero-shot and with light heads (GRU/LSTM); degrades under full end-to-end fine-tuning of heavy models.
Scarce annotations, distribution shift, and privacy/access constraints align precisely with where distributed learning helps.
The same compute, the same data seen — only weights are shared, and often the merged model is better.
Decentralization in Document Analysis is not a universal prescription, but a targeted call — to practitioners under low-resource, low-adaptability, or deployment-constrained conditions, for whom distributed learning is competitive and often superior.