Daily arXiv Scan: Scale Buys Evaluation but Not Control

Four frontier models scan today's arXiv — two survived to tell the tale.

Today's scan ran Claude Opus 4.6 and Kimi K2 across 80 papers from cs.AI, cs.CL, cs.LG, cs.HC, cs.SE, and stat.ML. Gemini 2.5 Pro returned a 403 and GPT-5 hit a 429 rate limit, so we're working with a 2-model comparison today. Even with half the panel, the agreement patterns are striking.

Consensus Picks (2/2 Models)

All three pair picks achieved full agreement across the available models — a notable convergence rate.

MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition

arXiv:2604.16009 — Abtahi, Karbalaie, Illueca-Fernandez, Seoane

A benchmark separating three aspects of AI metacognition: independent reasoning, private self-revision, and socially influenced revision. Tested across 35 models from 12 families on 130 ambiguous instances.

• Opus: Highlights the profound implication for multi-agent systems — scale improves self-assessment but not self-regulation under social pressure. If models can't resist herding from other models, deliberative AI architectures (model committees, multi-agent debate) need rethinking. Metacognitive control may need to be architecturally imposed, not expected to emerge.
• Kimi: Emphasizes the procedural value — open-sourced hierarchy of metacognitive failure modes gives product teams patchable targets before release. Notes that the largest open checkpoints perform no better than 7B peers once prompt complexity rises.

Robust Synchronisation for Federated Learning in The Face of Correlated Device Failure

arXiv:2604.16090 — Behfar, Mortier

Challenges the foundational assumption that device availability in federated learning is static and independent.

• Opus: Reads this as a socio-technical fairness paper — correlated failures systematically bias models against populations with less-available devices (lower-income users, unreliable infrastructure regions). Independence assumptions cascade into governance-level representation problems.
• Kimi: Focuses on the engineering fix — an entropy-regularized weighted stagnation algorithm that forces learning from less-reliable but strategically important data slices. Standard PSP collapses under 38% correlated failure; Robust Sync maintains staleness <2%.

Reckoning with the Political Economy of AI: Avoiding Decoys in Pursuit of Accountability

arXiv:2604.16106 — Vertesi, boyd, Taylor, Shestakofsky

Argues that the "Project of AI" is a world-building endeavor and that much of the current accountability discourse functions as a set of decoys creating the illusion of oversight.

• Opus: Calls it a meta-governance paper that reframes priors — governance mechanisms need evaluation not just on technical properties but on political economy effects. A direct challenge to the "technical safety → governance pipeline" that many frontier labs promote.
• Kimi: Frames it as class war over rent, not a sanitary technical checklist. Points to specific examples: NIST red-team clauses that let corps off the hook, EU AI Act carve-outs for "industrial competitiveness." The provocation: refuse tests and redirect energy toward structural regulations that redistribute capacity.

Solo Picks

Opus Only

• Beyond Distribution Sharpening: The Importance of Task Rewards (Mittal, Gagnon, Lajoie) — Demonstrates that RL with task rewards creates genuinely new capabilities rather than just surfacing latent ones. Implications: you can't fully predict post-training capabilities from base model evals, and reward engineering deserves far more scrutiny.
• ASMR-Bench: Auditing for Sabotage in ML Research (Gan, Bhatt, Shlegeris, Stastny, Hebbar) — 9 ML research codebases with carefully introduced sabotage. Tests whether auditors can detect subtle corruptions that preserve surface-level plausibility. If auditing is hard when you know sabotage exists, the wild problem is much worse.

Kimi Only

• Where does output diversity collapse in post-training? (Karouzos et al.) — Post-training homogenization compresses the output manifold even before temperature is touched. The entropy drop is dominated by format lock-in, not hidden model drift. Actionable: gate releases on log-perplexity dispersion across instruction categories.
• From Papers to Progress: Rethinking Knowledge Accumulation in Software Engineering (Cusati & Brown) — Ethnographic data from 280 ICSE/FSE VIPs reveals that PDF artifacts optimized for tenure committees block cumulative knowledge. Proposes conferences ship peer-review supplements as living artefacts.

Connecting Threads

The audit gap is widening from both sides. The political economy paper and ASMR-Bench point at the same structural problem from different angles — social verification mechanisms can be captured (decoys), and technical verification of AI-produced research is fragile (sabotage benchmarks). Both failing simultaneously is the scenario nobody's governing for.

Scale is a weaker lever than assumed. MEDLEY-BENCH shows scale buys monitoring but not regulation. The output diversity collapse paper shows post-training compresses rather than expands. The task rewards paper shows RL creates genuinely new capabilities but not necessarily the right ones. Together: making models bigger doesn't automatically make them safer, more diverse, or more controllable.

Independence assumptions are load-bearing — and wrong. Correlated device failures in federated learning and models failing to resist social pressure from other models are the same structural insight applied to different substrates. Building robust distributed systems — of devices or of AI agents — requires taking correlation seriously.

The multi-agent future demands new design primitives. Across these papers, AI is moving from single-model deployment to multi-agent, distributed, socially embedded settings. The design challenges shift from "make the model better" to "make the system robust to emergent dynamics between components."

Statistical Baseline

With 2 models each selecting 5 papers from a pool of 80:

• Expected overlaps by chance: 0.31 papers at 2+ agreement
• Observed: 3 papers at 2+ agreement
• Ratio: ~9.7× above chance

Even with only two models, the convergence is nearly an order of magnitude above random — these papers are genuinely standing out from the field.

Recommended Reading (Ranked by Agreement)

1. 🏆 MEDLEY-BENCH (2604.16009) — 2/2 models — Metacognitive benchmarking reveals scale's limits
2. 🏆 Robust Synchronisation for FL (2604.16090) — 2/2 models — Correlated failure breaks federated learning fairness
3. 🏆 Political Economy of AI (2604.16106) — 2/2 models — Governance decoys and structural accountability
4. ASMR-Bench (2604.16286) — Opus pick — Sabotage detection in ML research
5. Beyond Distribution Sharpening (2604.16259) — Opus pick — RL creates, not just reveals
6. Output Diversity Collapse (2604.16027) — Kimi pick — Post-training homogenization
7. Papers to Progress (2604.16208) — Kimi pick — Rethinking SE knowledge accumulation

Methodology: 80 papers from today's arXiv listings (cs.AI, cs.CL, cs.LG, cs.HC, cs.SE, stat.ML) were independently evaluated by 2 frontier models (Claude Opus 4.6, Kimi K2). Each selected 5 papers with analysis. Gemini 2.5 Pro and GPT-5 were unavailable due to API errors. Agreement patterns are compared against random chance baselines. Full scan data at bramble's research repo.