Rohan Varma
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top School
Top expert inArtificial Intelligence and Machine Learning Technologies
Rohan Varma is a research-focused software engineer with 11 years of experience building and scaling distributed ML systems, currently a Member of Technical Staff in San Francisco. He has led fault-tolerant training and large-scale RL infrastructure at Meta Superintelligence and most recently held research engineering leadership at Jump Trading before joining Reflection. A core PyTorch contributor and co-creator of FSDP and the torchtune post-training library, he has deep hands-on expertise in PyTorch RPC, DDP, and multi-GPU distributed training—contributions that include parameter-server tutorials and production-ready multi-GPU refactors in widely used PyTorch repos. He pairs research rigor with production engineering, shipping systems for training at 100k+ GPU scale and tuning attention/FFN components in tests and docs. Based out of the Bay Area with a CS degree from UCLA, he blends open-source impact with experience across industry ML teams, from startups to hyperscale platforms. Notably, his GitHub work demonstrates both pedagogical contributions (tutorials) and low-level distributed systems engineering that accelerate others’ adoption of PyTorch.
11 years of coding experience
8 years of employment as a software developer
University of California, Los Angeles
Stackoverflow
61reputation
17kreached
2answers
1question
Github Skills (18)
pytorch10
ropc10
distributed-training10
python10
testing10
drpc10
machine-learning10
rpc10
procedures10
llm9
attention-mechanism9
neural-network9
documentation9
mnist8
react-router6
Programming languages (9)
Github contributions (5)
pytorch/torchtune
Mar 2024 - Jul 2024
PyTorch native post-training library
Role in this project:
ML Engineer
ML EngineerContributions:183 reviews, 112 PRs, 237 pushes in 3 months
Contributions summary:Rohan's commits primarily focus on modifying and improving the documentation and tests related to the feed-forward network (FFN) within the PyTorch native post-training library. They clarified documentation and relaxed a test for the FFN. Furthermore, they added and used functions, such as `fixed_model_init` and `fixed_tensor_init`, in attention tests. These changes involve fine-tuning and testing core components of the library.
pytorch/examples
Feb 2020 - Apr 2020
A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
Role in this project:ML Engineer
ML EngineerContributions:7 reviews, 20 commits, 4 PRs in 2 months
Contributions summary:Rohan primarily contributed to developing a distributed training system using PyTorch's RPC framework. Their work focused on implementing a parameter server to manage model parameters and facilitate communication between trainers. They added MNIST data loading and loss calculation to demonstrate a basic training loop, and integrated GPU support for faster training. Furthermore, the user refactored code and added multi-GPU support within the training process.
pytorchvisiondeep-learningreinforcement-learningreinforcement
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.