Don't classify the year. Don't regress it. Retrieve it — learn an embedding where the closer two photographs sit, the closer their dates, and read the answer off the neighbours.
A classifier forces every photo into a decade bin and treats a one-year miss the same as a fifty-year miss. A regressor predicts a single number and is brittle far from its training mass. This paper takes a third route: train the network to rank a gallery by date-closeness to a query, so a wrong-but-near answer is still a good answer.
"In the worst case, when the exact date is not estimated, the user can obtain a relative ordering — one photograph is older than another — which is useful in archival annotation."
Years are ordinal. mAP only knows "relevant / not relevant", so to it a photo one year off and one twenty years off are identical — both just "wrong". nDCG instead uses a graded relevance r(n) and a logarithmic discount, rewarding rankings that put the closest dates on top. Reorder the gallery below and watch the two metrics react differently.
Swap two near years and mAP often doesn't budge — it already counted both as "relevant". nDCG moves, because it cares how the good answers are ordered and how the bad ones are spread through the list. That sensitivity to graded, ordinal ground truth is the paper's whole motivation.
Rank is a sum of hard comparisons 𝟙{sᵢ − sⱼ < 0} — counting how many items beat i. That indicator has no gradient. The trick (Qin et al.): replace it with a sigmoid G(x; τ) = 1 / (1 + e^(−x/τ)). Small τ hugs the true step (accurate but a near-flat gradient); large τ is smooth and trainable but blurrier. Drag τ to feel the trade-off (this is Fig. 1).
nDCG needs a relevance score for every gallery item. The choice shapes the embedding: a hard clip says "anything within a decade is equally good"; an inverse or exponential law keeps pressing for the exact year. Pick a relevance function and see its profile against the year gap |y_q − y_n|.
No labels-as-targets here. A batch goes through a ResNet-101 into embeddings; an all-versus-all cosine matrix gives every image its ranking against the others; the relevance matrix (from the years) says what the ranking should be; smooth-nDCG measures the gap and the gradient flows back into the CNN. Step through one iteration.
This is the payoff. Each point is a photograph's projection; colour runs along the year gradient. Untrained ImageNet features cluster by visual look — dates scrambled. After smooth-nDCG, the cloud reorganises into a smooth temporal manifold: neighbours share a date. Morph between the two, then drag the query to see who its nearest neighbours are.
The paper also reports a cluster-similarity heatmap, but the organisation reads far more clearly as a PCA scatter of cluster centres (Fig. 7–8): in the trained space the colour flows continuously, so a query's 10 nearest neighbours nearly all share its decade — which is exactly why k-NN on this space estimates a plausible year.
With the space sorted, prediction is just: embed the query, take its k nearest in a labelled support set, average their years. Too small a k is noisy; too large drags in unrelated dates. The sweet spot sits near k = 10. The slider below also drives the query's neighbourhood in the scatter above.
| Method | MAE ↓ | mAP | nDCG ↑ |
|---|---|---|---|
| Müller et al. — GoogLeNet (regression) | 7.5 | — | — |
| Müller et al. — GoogLeNet (classification) | 7.3 | — | — |
| Müller et al. — ResNet50 (cleaned test) | 7.12 | — | — |
| Visual-similarity baseline (ImageNet) | 12.4 | — | 0.69 |
| Smooth-nDCG · 256-image support | 8.44 | 0.12 | 0.72 |
| Smooth-nDCG · 1M support | 7.48 | — | 0.75 |
| Smooth-nDCG · 1M support (weighted k-NN) | 7.52 | — | 0.75 |
The retrieval model lands within a few months of dedicated regressors/classifiers on MAE — despite never being trained to predict a year — and tops every method on nDCG, the metric that actually reflects ordering quality. The visual-similarity baseline shows the gap: similar-looking ≠ similar-dated.
Unlike smooth-AP, smooth-nDCG scores the whole sorted list — how good the good ones are, and how the bad ones are spread.
Even a missed year yields an "older/newer" ordering against the collection — what archivists actually need.
Change the relevance matrix and the same loss transfers to other ordered or tree-structured retrieval tasks.
Unbalanced classes (trains, vehicles) can cluster by object rather than date — the embedding inherits dataset skew.
Brown et al. smoothed the path to average precision; smooth-nDCG extends it to problems where different labels should not be punished equally — ranking images by a continuous, ordered criterion rather than a categorical one.