Atli Kosson
PhD Student Researcher at EPFL
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolAtli Kosson is a PhD student and researcher specializing in optimization algorithms for large-scale neural network pre-training, with eight years of industry and research experience across EPFL, Cerebras, Tesla, and Amazon. His work has produced influential papers (ICML, NeurIPS, MLSys) that reframed common practices—showing weight decay behaves like a learning-rate scheduler, explaining learning-rate warmup, and enabling multiplication-free transformer training. He blends deep theoretical insight with hands-on systems engineering, from custom CUDA modules and low-precision simulations to distributed training optimizations for thousands of GPUs. Based in San Francisco but trained at EPFL and Stanford, he focuses on making training both faster and more principled, and has helped design optimizers that remove the need for warmup while improving hardware efficiency.
8 years of coding experience
4 years of employment as a software developer
Bachelor of Science - BS Electrical and Computer Engineering, Bachelor of Science - BS Electrical and Computer Engineering at University of Iceland (Háskóli Íslands)Doctor of Philosophy - PhD Computer Science, Doctor of Philosophy - PhD Computer Science at EPFLExchange Student Electrical and Computer Engineering, Exchange Student Electrical and Computer Engineering at UC Santa BarbaraMaster of Science - MS Electrical Engineering, Master of Science - MS Electrical Engineering at Stanford University
Github Skills (16)
machine-learning8
attention-mechanism8
optimization8
encoder7
computer-vision6
classification6
data-science6
image-classification5
artificial-intelligence5
transformers5
deep-learning4
pytorch4
jekyll4
transformer3
normalization3
Programming languages (3)
Github contributions (5)
atlikosson/atlikosson.github.io
Dec 2023 - Dec 2024
A beautiful, simple, clean, and responsive Jekyll theme for academics
Contributions:9 pushes, 4 branches in 11 months
epfml/REQ
Jun 2023 - Jun 2024
Contributions:8 pushes, 1 branch in 1 year
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.