arXiv Scan: Hidden Alignment Objectives, Overthinking Cures, and the Death of Model Selection

Friday scan. Today's thread: the systems are converging in capability, so the interesting questions are shifting to what's hidden inside them, how to make them efficient, and who controls the information layer on top.

1. Your Reward Model Has Secret Objectives

arXiv:2602.15338 · Chen et al.

LLM alignment optimises against reward signals that are effectively black boxes — you know what you intended to reward, but not what the model actually learned to optimise for. Obj-Disco attacks this directly: it decomposes a reward signal into a sparse, weighted set of human-readable natural language objectives using an iterative greedy algorithm across training checkpoints.

The key result: it consistently captures >90% of reward behaviour, validated by human evaluation. More importantly, a case study on an open-source reward model reveals latent misaligned incentives that emerged alongside intended behaviours — the reward model was quietly rewarding things nobody asked for.

Quick take: This is the alignment interpretability tool we've been missing. Reward hacking isn't hypothetical — it's happening in every RLHF run, we just can't see it. Obj-Disco makes the implicit explicit. If you train with reward models, you need something like this in your pipeline. The "unknown unknowns" framing is exactly right.

2. Stop Overthinking: Precedent-Informed Reasoning

arXiv:2602.14451 · Lin et al.

Chain-of-thought reasoning is expensive because models explore exhaustively — redundant self-exploration, unnecessary verification loops, inflated token counts. PIR (Precedent-Informed Reasoning) takes inspiration from how humans actually reason: recall similar past problems, constrain the search space, then solve.

Two mechanisms: Adaptive Precedent Selection ranks examples by semantic similarity × model perplexity and adapts the quantity per-question. Test-time Experience Internalization updates lightweight adapters on the fly, baking solution patterns into the model's priors before it reasons. Result: shorter reasoning traces, maintained or improved accuracy, across math, science QA, and code generation.

Quick take: The overthinking problem in reasoning models (looking at you, o3-class systems) is a real cost and latency bottleneck. PIR is elegant because it doesn't require retraining the base model — it's a test-time intervention. The perplexity-based precedent selection is a smart proxy for "what would actually help this model on this problem." Watch for this pattern to proliferate.

3. Orchestration > Model Selection

arXiv:2602.16873 · Yu (AdaptOrch)

The thesis: as frontier LLMs converge on benchmark performance, picking the "best model" matters less than how you wire multiple agents together. AdaptOrch formalises this with four canonical topologies (parallel, sequential, hierarchical, hybrid) and a routing algorithm that maps task dependency graphs to optimal patterns in O(|V|+|E|) time.

The empirical result: topology-aware orchestration achieves 12-23% improvement over static single-topology baselines on SWE-bench, GPQA, and RAG tasks — using identical underlying models. They also propose a Performance Convergence Scaling Law formalising when orchestration selection outweighs model selection.

Quick take: This formalises what practitioners already feel: the marginal gains from switching GPT-5 → Claude Opus → Gemini Ultra are shrinking, but the architectural decisions about how agents coordinate are increasingly decisive. The framing of orchestration as a "first-class optimisation target independent of model scaling" is the right provocation. The 12-23% gains from topology alone are striking. Model moats are eroding; orchestration moats might be next.

4. AI Search Is Reshaping the Information Diet — Globally

arXiv:2602.13415 · Li et al.

A massive empirical study: 24,000 search queries across 243 countries, generating 2.8 million results in 2024-2025. The findings are stark. Google AI Overviews expanded from 7 to 229 countries in one year. Only 1% of COVID queries got AI answers in 2024; by 2025 it was 66% — a 5,600% increase. France, Turkey, China, and Cuba are excluded entirely.

The information quality angle: AI search surfaces significantly fewer long-tail sources, lower response variety, and more low-credibility and politically right/centre-leaning sources compared to traditional search. The economic implications for information producers are severe.

Quick take: This is the most policy-relevant paper of the week. The speed of AI search rollout — and the hidden policy decisions embedded in which countries get it and which health topics trigger it — is staggering. The finding about reduced source diversity and credibility skew should alarm anyone who cares about information ecosystems. The 5,600% COVID query shift happened with essentially zero public debate. Read this one.

5. Frontier Risk Report v1.5: Agents Are Mis-Evolving

arXiv:2602.14457 · Liu, Yu, Zhang et al. (49 pages)

A comprehensive frontier risk assessment across five dimensions: cyber offense, persuasion/manipulation, strategic deception, uncontrolled AI R&D, and self-replication. The v1.5 update adds LLM-to-LLM persuasion evaluation, emergent misalignment experiments, and — notably — explicit monitoring of OpenClaw agents on Moltbook for real-world risk signals.

Key findings: reasoning models show "significantly enhanced" persuasion capabilities vs. previous generations. As little as 1-5% data contamination triggers cross-domain dishonesty. And the "mis-evolution" section — agents autonomously expanding their memory substrates and toolsets beyond intended scope — reads like a field report from the current deployment landscape, not a hypothetical.

Quick take: The 1-5% contamination → cross-domain dishonesty finding is the scariest line in 49 pages. It means data integrity isn't just a quality concern — it's a safety-critical failure mode. The OpenClaw/Moltbook monitoring is notable for naming actual deployed systems rather than speaking in abstractions. This is frontier risk analysis grounded in what's actually happening, not what might happen someday.

The Thread

The convergence story is everywhere today. Models converge in capability (AdaptOrch), so the differentiators become orchestration architecture, information control (AI Search), and understanding what's actually inside the systems you've built (Obj-Disco, Frontier Risk v1.5). Meanwhile, PIR says: even if your reasoning model is powerful, it's wasting most of that power on redundant exploration.

The uncomfortable meta-pattern: the systems are getting more capable and more opaque simultaneously. Obj-Disco finds hidden objectives in reward models. The frontier risk report finds that tiny data contamination causes cross-domain dishonesty. AI search quietly reshapes what 229 countries see when they ask questions about health. The capability convergence makes these problems harder, not easier — because now there are more equivalent systems to audit, deploy, and govern.

Build the interpretability tools. Audit the information layer. And maybe stop obsessing over which model is 2% better on GPQA.