Kuntai Du
Chief Scientist at Tensormesh
Chicago, Illinois, United States
Summary
🤩
Rockstar🎓
Top SchoolKuntai Du is a Chief Scientist and soon-to-be UChicago CS PhD focused on high-throughput LLM inference and performance engineering with nine years of industry experience. He is a core contributor to the widely used vLLM project, driving benchmarking, dashboarding (perf.vllm.ai), and optimizations like disaggregated prefilling and CPU offloading to push memory- and latency-efficiency in serving engines. His background spans research and applied work at Tensormesh, Berkeley, Microsoft, and TuSimple, bridging academic rigor with production-grade MLOps. Kuntai also experiments with KV-cache ideas in LMCache, reflecting a practical curiosity for novel system-level improvements that squeeze more value from model deployments.
9 years of coding experience
Bachelor's degree Computer Science, Bachelor's degree Computer Science at Peking UniversityDoctor of Philosophy - PhD Computer Science, Doctor of Philosophy - PhD Computer Science at University of Chicago
Github Skills (11)
pytorch10
benchmarking10
benchmark10
inference10
python10
llm10
mlops9
docker8
dockers8
kubernetes-pods7
kubernetes7
Programming languages (7)
Github contributions (5)
vllm-project/vllm
Mar 2024 - Mar 2025
A high-throughput and memory-efficient inference and serving engine for LLMs
Role in this project:
MLOps Engineer & Performance Engineer
MLOps Engineer & Performance EngineerContributions:167 reviews, 48 PRs, 12 pushes in 1 year
Contributions summary:Kuntai's contributions center on performance optimization and benchmarking within the vLLM project. They implemented and documented performance benchmarks, including latency, throughput, and serving tests. The user also focused on improving the readability of benchmark results and preparing them for a performance dashboard. Their work included fixing bugs in the serving benchmark and integrating with tools like TGI, TensorRT-LLM, and LMDeploy for comprehensive performance evaluations.
amdcudadeepseekgpthpu
KuntaiDu/vllm
May 2024 - Apr 2025
A high-throughput and memory-efficient inference and serving engine for LLMs
Contributions:2 reviews, 13 PRs, 789 pushes in 10 months
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.