Myle Ott
New York, New York, United States
Summary
🤩
RockstarMyle Ott is an experienced ML infrastructure engineer and researcher with 13 years building large-scale language models and training systems, currently on the technical staff at Thinking Machines Lab. Previously a founding researcher at Character.AI and a Distinguished Research Scientist at DeepMind, he led production ML efforts and research that bridge cutting-edge models and robust training infrastructure. At Meta FAIR he helped architect widely used PyTorch tooling—contributions include FSDP, memory-optimizing checkpoints, and enhancements to fairseq and RoBERTa—work that underpins many large-scale LLM trainings. His open-source impact spans high-profile repos like fairscale and fairseq, where he implemented sharded generation and parameter-sharding wrappers to improve efficiency and maintainability. Trained as a PhD in NLP from Cornell, he also has award-winning research on fake review detection, showing a long-standing blend of rigorous research and practical systems engineering. Colleagues describe him as someone who moves seamlessly between low-level performance engineering and model-centric research, often improving developer ergonomics for large-model training pipelines.
13 years of coding experience
Github Skills (20)
pytorch10
distributed-training10
python10
machine-learning10
transformer-models10
deep-learning10
natural-language-processing10
sequence-to-sequence10
nlp10
fairseq10
initializr9
testing9
ml9
initializer9
multiprecision9
Programming languages (8)
Github contributions (5)
facebookresearch/fairseq
Jul 2017 - Dec 2021
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Role in this project:
Back-end Developer & ML Engineer
Back-end Developer & ML EngineerContributions:14 releases, 25 reviews, 1613 commits in 4 years 6 months
Contributions summary:Myle's contributions primarily revolved around enhancing the functionality of the fairseq toolkit, particularly concerning sequence-to-sequence tasks. They implemented support for sharded generation to improve efficiency, fixed various bugs, and added features such as handling of the "<mask>" token in RoBERTa-based models and improving training procedures. They also updated existing models to include newly released models, like the Bart model.
aipythonsequence-to-sequencepytorchartificial-intelligence
pytorch/translate
May 2018 - Mar 2021
Translate - a PyTorch Language Library
Role in this project:ML Engineer
ML EngineerContributions:29 commits, 13 PRs, 3 pushes in 2 years 10 months
Contributions summary:Myle primarily contributed to the development and maintenance of the PyTorch-based translation library. Their work includes fixing version constraints for dependencies, adding integration tests for various model architectures (RNN, Char RNN, and multilingual models), and adapting the codebase to incorporate newer features and APIs from the fairseq library. These changes involved modifying model structures and training pipelines, indicating a focus on improving the functionality and maintainability of the translation models.
pytorchartificial-intelligencemachine-learningonnx
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.