Murali Andoorveedu
Senior Software Security Engineer at NVIDIA
San Jose, California, United States
Summary
🤩
Rockstar🎓
Top SchoolMurali Andoorveedu is a Senior Software Security Engineer based in San Jose with eight years of experience building and optimizing machine learning systems and inference infrastructure. He has deep hands‑on experience across LLM serving, accelerator stacks, and GPU performance tuning—contributing to high-profile open-source projects like CuPy and vLLM where he improved integer matrix multiplication performance and helped enable pipeline-parallel serving. At AWS he worked on the Neuron stack for Trainium, and his recent roles span ML infrastructure, security, and deployment at startups and NVIDIA. Murali blends low-level performance engineering (CUDA, Triton, ONNX) with production MLOps for multi-node deployments and LoRA integrations. He is an active problem-solver who pairs rigorous testing and documentation with practical optimizations, and he frequently seeks tech talks and cross-team collaboration to learn and share. Notably, his background includes firmware/RTL scripting and research in bio-inspired multi-agent control, reflecting a broad systems-to-algorithms perspective.
8 years of coding experience
4 years of employment as a software developer
Bachelor of Applied Science - BASc Engineering Science - ECE Major, Bachelor of Applied Science - BASc Engineering Science - ECE Major at University of Toronto
Github Skills (24)
qx10
pytorch10
python10
machine-learning10
distributed-systems10
inference10
pipelining10
numpy10
llm10
mlops10
gpu10
performance-optimization10
pipe10
cupy10
pipeline10
Programming languages (3)
Github contributions (5)
vllm-project/vllm
Jan 2024 - Mar 2025
A high-throughput and memory-efficient inference and serving engine for LLMs
Role in this project:
MLOps Engineer
MLOps EngineerContributions:139 reviews, 26 PRs, 1 push in 1 year 2 months
Contributions summary:Murali primarily contributed to the infrastructure and tooling around distributed inference and serving of large language models. Their work involved supporting pipeline parallel (PP) serving, including modifications to the executor and model runner codebases. They also added support for Qwen models with PP and fixed issues related to LoRA integration and multi-node deployment, indicating expertise in optimizing and deploying LLM serving infrastructure. Furthermore, the user improved existing features such as chunked prefill and speculative decoding, suggesting experience with performance optimization techniques.
amdcudadeepseekgpthpu
cupy/cupy
May 2022 - Aug 2022
NumPy & SciPy for GPU
Role in this project:Back-end Developer & Performance Engineer
Back-end Developer & Performance EngineerContributions:1 review, 7 commits, 2 PRs in 3 months
Contributions summary:Murali focused on optimizing the performance of integer matrix multiplication within the CuPy library. They modified block size parameters in the `_routines_linalg.pyx` file to improve performance and ensure correctness. Furthermore, the user integrated a SciPy special function, `cosm1`, into CuPy and included necessary tests and documentation. This included adjustments to the test setup to enhance accuracy, showcasing a commitment to rigorous testing.
cudapythoncusolvergpunumpy
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.