Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

Neural Thickets makes a strange claim and then backs it up: near a well-pretrained model's weights, task-specific experts are so densely packed that you can find one just by adding random Gaussian noise and keeping the perturbations that work. MIT's Yulu Gan and Phillip Isola built RandOpt around this idea — no gradients, no learning rate, just noise and ensembling — and it matches or beats PPO and GRPO on math, coding, writing, and chemistry tasks at equivalent compute. If reinforcement learning post-training turns out to be more expensive than it needs to be, this is the paper that will have called it first.