Hanzhi Zhou
Member Of Technical Staff at OpenAI
Seattle, Washington, United States
Summary
🤩
Rockstar🎓
Top SchoolHanzhi Zhou is a machine learning and systems engineer with nine years of experience building high-performance ML infrastructure for companies like OpenAI, Apple, and TikTok. He specializes in GPU/TPU performance engineering, distributed training and inference, and MLOps—contributing to prominent open-source projects such as vLLM and ByteDance’s Monolith to accelerate LLM and recommender workloads. At Apple he focused on foundation model pre- and post-training performance; at TikTok he was a core contributor optimizing multi-node GPU training and inference kernels. He combines a strong systems background (CUDA graph optimizations, custom all-reduce kernels, pinned/non-blocking memory copies) with applied ML across vision, robotics, and recommender systems. A UVa CS graduate with a 3.99 GPA, he also led student teams to build widely used campus software, reflecting both technical depth and product-minded delivery. Notably, his work often targets low-level GPU memory and kernel-level bottlenecks that yield outsized gains in large-model throughput and scalability.
9 years of coding experience
7 years of employment as a software developer
High School Diploma, High School Diploma at The High School Affiliated to Renmin University of ChinaHigh School Diploma, High School Diploma at International BaccalaureateBachelor of Science - BS, Computer Science, GPA 3.99/4.0, Bachelor of Science - BS, Computer Science, GPA 3.99/4.0 at University of Virginia
Github Skills (20)
pytorch10
c-language10
python10
memory-management10
gpu-programming10
machine-learning10
inference10
mlops10
tensorflow10
performance-optimization10
cuda10
cprogramming-language10
back-end-development9
distributed-systems9
hashtable9
Programming languages (10)
Github contributions (5)
bytedance/monolith
Dec 2022 - Jan 2023
A Lightweight Recommendation System
Role in this project:
Back-end Developer & MLOps Engineer
Back-end Developer & MLOps EngineerContributions:7 commits, 10 comments, 5 issues in 20 days
Contributions summary:Hanzhi primarily contributed to the development and optimization of a recommendation system. Their work involved integrating GPU-accelerated hash table operations, which likely improves the performance of embedding lookups, a core operation in recommendation systems. They also worked on fused operations and global norm calculations with clipping, suggesting efforts to optimize model training and potentially address gradient-related issues. Additionally, the user made changes related to GPU memory oversubscription and integrated GPU support within the project, hinting at MLOps and performance engineering tasks.
recommendationbytedancerecommendation-system
vllm-project/vllm
Oct 2023 - Mar 2025
A high-throughput and memory-efficient inference and serving engine for LLMs
Role in this project:MLOps Engineer
MLOps EngineerContributions:53 reviews, 8 PRs, 148 comments in 1 year 5 months
Contributions summary:Hanzhi primarily contributed to the performance and efficiency of the VLLM inference engine, focusing on CUDA graph optimization and memory management. Their work involved making non-blocking memory copies and utilizing pinned memory to reduce overhead, enhancing the performance of the model execution. Furthermore, the user implemented custom all-reduce kernels to accelerate distributed training or inference, along with refactoring IPC buffer initialization for distributed systems. These changes indicate a focus on improving the efficiency and scalability of LLM inference.
amdcudadeepseekgpthpu
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.