Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

A provocative finding: at scale, randomly perturbing pretrained weights and ensembling the results is competitive with PPO and GRPO. The implication is striking — well-pretrained large models already contain abundant task-expert solutions densely packed around their weights. Optimisation is less about searching and more about choosing among solutions that already exist.