Kirthi Sivamani
Senior Deep Learning Engineer at NVIDIA
Palo Alto, California, United States
Summary
🤩
Rockstar🎓
Top SchoolKirthi Sivamani is a Senior Deep Learning Engineer at NVIDIA with seven years of experience building high-performance ML and distributed systems, focused on accelerating Transformer models on NVIDIA GPUs. A Purdue Computer Engineering graduate with a 4.0 GPA, he has deep practical expertise in FP8 optimization, tensor parallelism, and framework-agnostic kernel work—contributing notably to the popular NVIDIA/TransformerEngine repository. Based in Palo Alto, he moved from intern to senior engineer at NVIDIA, driving bug fixes, checkpointing improvements for activation recomputation, and softmax kernel development that improve both training efficiency and inference memory use. Kirthi combines strong research foundations from Purdue (including published work) with production-grade engineering, making him equally comfortable designing algorithms and hardening them for GPU-scale deployment. Notable but less obvious: he pairs systems-level thinking with hands-on low-level precision work, optimizing numerical formats and parallelism to squeeze performance from modern GPU architectures.
7 years of coding experience
4 years of employment as a software developer
The Mother's International SchoolBachelor's degree, Computer Engineering, GPA 4.0/4.0, Bachelor's degree, Computer Engineering, GPA 4.0/4.0 at Purdue University
English, Hindi
Github Skills (13)
cuda10
pytorch10
machine-learning10
deeplearning-ai10
p810
deep-learning10
gpu10
python9
transformer-models9
optimization8
optimisation8
optmization8
performance-optimization8
Programming languages (6)
Github contributions (5)
NVIDIA/TransformerEngine
Oct 2022 - Jan 2023
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.
Role in this project:
ML Engineer
ML EngineerContributions:1 release, 736 reviews, 25 commits in 3 months
Contributions summary:Kirthi primarily contributes to the `nvidia/transformerengine` repository, which focuses on accelerating Transformer models on NVIDIA GPUs. Their work centers around optimizing the library for FP8 precision, a key feature highlighted in the repository description. The commits demonstrate their focus on bug fixes related to FP8 autocasting, tensor parallelization improvements, and enhancing the checkpointing functionality for activation recomputation in the context of FP8. They also worked on the framework agnostic softmax kernels.
better-performancememorypythontrainingbit
ksivaman/efficientCamScan
Apr 2019 - Apr 2019
Contributions:24 commits, 31 pushes, 1 branch in 3 days
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.