Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights reframes post-training as exploiting a dense distribution of task-expert solutions around pretrained weights. The authors show that randomly sampling and ensembling perturbations achieves within 5% of PPO and GRPO on benchmark tasks, challenging core optimization assumptions. It is cheaper than the typical fine-tuning approach as it eliminates iterative gradient steps. A beautiful insight for resource-constrained teams.