Sourab Mangrulkar
Applied Scientist II at Amazon
Bengaluru, Karnataka, India
Summary
🤩
Rockstar🎓
Top SchoolSourab Mangrulkar is an Applied Scientist II with nine years of experience building and productionizing ML and deep learning solutions across Amazon, Microsoft, and Hugging Face. He specializes in scalable model training and distributed systems, contributing key FSDP, mixed-precision, and memory-optimization integrations to flagship Hugging Face libraries (transformers, accelerate, trl, peft). At Amazon he delivered BERT-based relevance and click-prediction models that drove measurable marketplace impact and authored multiple conference papers. Comfortable moving models from research to production (SageMaker, ONNX) and optimizing large-model fine-tuning (LoRA, QLoRA, 8-bit), he blends research rigor with pragmatic engineering. Based in Bengaluru, he brings deep expertise in distributed training trade-offs and a track record of performance and efficiency improvements that aren’t obvious from titles alone.
9 years of coding experience
6 years of employment as a software developer
MPC, MPC at Narayana Junior CollegeBachelor’s Degree Computer Science and Engineering, Bachelor’s Degree Computer Science and Engineering at National Institute of Technology GoaSchool Science, School Science at The Manik Public School
English, Hindi, Marathi
Stackoverflow
11reputation
138reached
0answers
1question
Github Skills (19)
transformers10
pytorch10
distributed-training10
parallelization10
python10
machine-learning10
sdp10
reinforcement-learning10
fs10
deepspeed10
lora10
memory-optimization10
fine-tuning9
deep-learning9
nlp9
Programming languages (6)
Github contributions (5)
huggingface/peft
Nov 2022 - Mar 2023
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
Role in this project:
ML Engineer
ML EngineerContributions:8 releases, 594 reviews, 216 commits in 3 months
Contributions summary:Sourab's commits primarily focused on adding, modifying and refactoring code related to parameter-efficient fine-tuning (PEFT) techniques for large language models, including techniques like LoRA, prefix-tuning and prompt tuning. Their contributions involve the creation and modification of several modules related to the fine-tuning process, which involves the use of Embedding and Linear layers. The user has also added support for 8-bit quantization in the training process and enhanced the training and merging functionalities of the LoRA implementation.
parametermachine-learningfineparameter-efficientfine-tuning
huggingface/accelerate
Apr 2022 - Jan 2023
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Role in this project:ML Engineer
ML EngineerContributions:341 reviews, 63 commits, 241 PRs in 9 months
Contributions summary:Sourab's commits focused on incorporating and enhancing PyTorch's Fully Sharded Data Parallel (FSDP) features within the Hugging Face `accelerate` library. They added new FSDP functionality, handled manual wrapping, and improved the integration, including automatic mixed precision. Additionally, the user contributed to a peak memory usage tracker, providing examples to monitor memory efficiency, particularly when using FSDP and MoE models. This work was targeted towards enabling and optimizing distributed training on various hardware configurations with a focus on reducing memory footprint.
pytorchpytorch-modelsmixed-precisiondeep-learninggpu
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.