Saaketh Narayan
Research Engineer at Meta
New York, New York, United States
Summary
🤩
Rockstar🎓
Top SchoolSaaketh Narayan is a Research Engineer with nine years of experience specializing in large language model pretraining and ML systems, currently on Meta’s Llama pretraining team in New York. He has a strong track record at Databricks Mosaic Research improving training infrastructure—notably enhancing FSDP performance and sharded checkpoint compatibility in the popular mosaicml/composer library. Saaketh pairs systems-level engineering (stream management, low-precision fixes) with dataset and training tooling work, having extended streaming dataloaders and sampling features in llm-foundry to better support fine-tuning. His background spans academic robotics research in adaptive sampling and practical ML deployments at X and Microsoft, showing a blend of research rigor and production-minded engineering. Colleagues rely on him to bridge low-level distributed training challenges and high-level model workflows, and he brings an unusual mix of product, research, and startup experience to large-scale model training.
8 years of coding experience
5 years of employment as a software developer
BASIS ScottsdaleBachelor's Business Computer Science, Bachelor's Business Computer Science at University of Pennsylvania
English, Spanish, Telugu
Github Skills (12)
neural-network10
data-loading10
pytorch10
machine-learning10
deeplearning-ai10
nlp10
deep-learning10
sdp10
python10
artificial-neural-networks10
ml10
llm10
Programming languages (4)
Github contributions (5)
mosaicml/llm-foundry
Aug 2023 - Jan 2025
LLM training code for Databricks foundation models
Role in this project:
ML Engineer
ML EngineerContributions:1 release, 188 reviews, 76 PRs in 1 year 5 months
Contributions summary:Saaketh contributed to the `llm-foundry` repository, which focuses on LLM training code. Their contributions primarily involved modifying the `StreamingTextDataset` and associated dataloaders to support various functionalities. These changes included adding features such as `shuffle_block_size`, `sampling_method`, and device batch size handling, which improved the dataset's flexibility and compatibility with fine-tuning tasks. Furthermore, the user updated streaming arguments for `StreamingDataset` subclasses, suggesting an understanding of data loading and its impact on model training.
deep-learningllmneural-networksnlppytorch
mosaicml/composer
Jul 2023 - Jan 2025
Supercharge Your Model Training
Role in this project:ML Engineer
ML EngineerContributions:4 releases, 103 reviews, 89 PRs in 1 year 5 months
Contributions summary:Saaketh primarily contributed to the improvement and maintenance of the MosaicML Composer library, focusing on integrating and optimizing Full Sharded Data Parallel (FSDP) features. Their work involved patching and modifying the FSDP implementation to enhance computation overlap, improve stream management for unshard operations, and ensure correct handling of multi-unshard streams. They also addressed issues in low-precision layer normalization and made the sharded checkpoint loading backwards-compatible.
pytorchml-systemsdeep-learningneural-networksmachine-learning
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.