Abhay Gupta
Research Scientist, Machine Learning at Databricks
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolAbhay Gupta is a research scientist and technical lead with a decade of experience building efficient training and inference algorithms for large language and multi-modal models, currently at Databricks on the MosaicML team. He led ML systems work at Cerebras, focusing on weight/activation sparsity, compression (pruning, quantization, speculative execution), and hardware–software co-design, and earlier applied ML to finance and sales at Microsoft. His contributions to the popular mosaicml/llm-foundry repository reflect hands-on improvements to LLM training code and architecture—adding activation functions, flexible norm handling, and cross-attention support to scale foundation models. Based in San Francisco, he pairs deep academic training in computer vision from Carnegie Mellon and IIT Hyderabad’s CS honors program with practical production experience. Colleagues know him for mentoring engineering teams and building external research partnerships that translate cutting-edge ideas into deployable systems. Off-hours he’s an avid traveler and car and music enthusiast, bringing a pragmatic, creative mindset to complex ML infrastructure challenges.
10 years of coding experience
6 years of employment as a software developer
Bachelor of Technology - BTech Computer Science & Engineering (Honors), Bachelor of Technology - BTech Computer Science & Engineering (Honors) at Indian Institute of Technology HyderabadMaster of Science - MS Computer Vision, Master of Science - MS Computer Vision at Carnegie Mellon University
English, Hindi
Stackoverflow
1reputation
0reached
0answers
0questions
Github Skills (10)
neural-network10
attention-mechanism10
pytorch10
nlp10
deep-learning10
trainings10
python10
modeling10
llm10
model-optimization9
Programming languages (5)
Github contributions (5)
mosaicml/llm-foundry
Jul 2024 - Dec 2024
LLM training code for Databricks foundation models
Role in this project:
ML Engineer
ML EngineerContributions:8 reviews, 28 PRs, 39 pushes in 4 months
Contributions summary:Abhay primarily contributes to the LLM training code within the Databricks foundation model repository. Their work includes enabling new functionalities like passing epsilon values for norm layers, and adding features such as quickgelu activation functions. They also focused on generalizing the build_inner_model function and enabling cross attention layers, reflecting a focus on model architecture and training efficiency.
deep-learningllmneural-networksnlppytorch
gupta-abhay/deep-prediction
Sep 2019 - Dec 2019
Contributions:93 commits, 1 push, 1 branch in 3 months
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.