Richard Huo

Senior System Software Engineer at NVIDIA

Seattle, Washington, United States
email-iconphone-icongithub-logolinkedin-logotwitter-logostackoverflow-logofacebook-logo
Join Prog.AI to see contacts
email-iconphone-icongithub-logolinkedin-logotwitter-logostackoverflow-logofacebook-logo
Join Prog.AI to see contacts

Summary

🤩
Rockstar
🎓
Top School
De Huo is a system software engineer based in Seattle who builds and optimizes large-scale GPU and cloud inference platforms. With a background spanning OCI compute and AI services at Oracle to Triton inference work at NVIDIA, he focuses on distributed model serving, GPU resource management, Kubernetes, and production-grade model training/serving pipelines. Proficient in Python, Rust, Go, C++, Java, and shell scripting, he brings a systems-first approach to cloud-native ML infrastructure and low-level performance engineering. Early work on hypervisor fleet CI/CD, image building and large fleet patching gives him practical expertise in reliability and automation for at-scale compute environments. Although relatively early in his career, he has rapidly progressed into senior roles driving LLM and CV serving initiatives across cloud and edge deployments. His cross-cutting experience—spanning virtual machine infrastructure to GPU-backed inference—makes him effective at bridging infra, runtime, and ML serving concerns.
code1 year of coding experience
job5 years of employment as a software developer
bookUniversity of California Santa Cruz
bookBachelor of Engineering Electrical Engineering and Automation, Bachelor of Engineering Electrical Engineering and Automation at Hohai University
languagesEnglish, Chinese
github-logo-circle

Github Skills (31)

pytorch10
kubernetes10
runtimes10
data-center10
preprocess10
tensorrt10
machine-learning10
inference10
large-language-models10
llm-inference10
nvidia10
deep-learning10
gpu10
cuda10
rust10

Programming languages (3)

C++RustPython

Github contributions (5)

github-logo-circle
richardhuo-nv/vllm

Sep 2025 - Jun 2026

A high-throughput and memory-efficient inference and serving engine for LLMs
Contributions:2 pushes, 1 branch in 9 months
richardhuo-nv/TensorRT-LLM

Aug 2025 - Aug 2025

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in performant way.
Contributions:4 pushes in 2 days
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.
Request Free Trial