Research

To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters

Sara Dragutinovic, Yedi Zhang, Rajesh Ranganath

[arXiv] · [ICLR SciForDL workshop]

Softmax ≥ Linear: Transformers may learn to classify in-context by kernel gradient descent

Sara Dragutinovic, Andrew M Saxe, Aaditya K Singh

[arXiv] · [ICML MOSS workshop]

Strategy coopetition explains the emergence and transience of in-context learning

Aaditya K Singh, Ted Moskovitz, Sara Dragutinovic, Felix Hill, Stephanie CY Chan, Andrew M Saxe

[arXiv] · [ICML Oral]