10 breakthrough publications that defined modern machine learning, attention, and RLHF.
DeepSeek Research
Replaces the residual connection paradigm (x + f(x)) used in every major transformer for the past decade. Projects residual paths onto Birkhoff Polytope manifold — neither explode nor collapse at any depth.
Jolicoeur-Martineau et al. (Samsung SAIL)
A 7M parameter model beats DeepSeek-R1, Gemini 2.5 Pro, and o3-mini on ARC-AGI benchmarks. Proves recursive reasoning can compensate for massive parameter counts.
Various
Physics-inspired attention using wave interference — builds holographic memory with cosines, sines, and phase modulation instead of standard dot-product attention.
RSA Team
Test-time scaling via evolutionary algorithms — combines parallel and sequential scaling. Enables Qwen3-4B to compete with DeepSeek-R1 on AIME and LiveCodeBench.
Stanford & SambaNova
Makes fine-tuning potentially irrelevant — evolves the context instead of weights. Delta updates and grow-and-refine mechanisms. 59.5% on AppWorld (+10.6pp).
Stanford
Recovers 66.8% of base model diversity hidden by RLHF alignment. Single prompting technique: ask LLMs to generate multiple responses with probability estimates. 2x diversity, zero safety loss.
Various (ICLR 2026 Oral)
Prompt optimization via evolutionary search outperforms GRPO by 10-20% while using 35x fewer rollouts. Also beats MIPROv2 by 10%+.
Liang et al. (NUS, UT Austin, Oxford)
Eliminates per-task fine-tuning — maps task prompts directly to LoRA weights via hyper-convolutional decoder. 12,000x lower overhead than full fine-tuning.
Sebastian Raschka
Overview of inference-scaling approaches — recursive language models, chain-of-thought, tree search, and their trade-offs for improved reasoning.
Akshay Kalane
RAG approach that achieves 98.7% on FinanceBench without vector DBs, embeddings, chunking, or similarity search. Challenges the assumption that RAG = vector DB.