Hao Zhu
Senior Manager, Accelerated Spark Applications
Sunnyvale, California, United States
Summary
👤
Senior🎓
Top SchoolHao Zhu is a senior engineering manager specializing in accelerating Spark applications on GPUs and leading global technical escalation teams for complex data platforms. With over a decade of hands-on experience across RDBMS, MPP engines, SQL-on-Hadoop, MapR platforms and Kubernetes, he blends deep root-cause analysis with pragmatic performance tuning and stress testing. At NVIDIA he focuses on GPU-accelerated Spark work, contributing to high-profile open-source efforts like the NVIDIA Spark RAPIDS plugin and Dataproc GPU initialization tooling. He has a proven track record building and coaching distributed, cross-geo teams while interfacing with engineering and product to drive fixes and feature prioritization. Equally comfortable reading Java source for RCA or writing automation in Perl/Python/Scala, he brings an uncommon mix of customer-facing escalation leadership and low-level systems troubleshooting. Based in Sunnyvale, he leverages production DBA and Oracle high-availability experience to inform resilient, performant big-data solutions.
10 years of coding experience
10 years of employment as a software developer
Bachelor of Science (B.S.), Computer Science, Bachelor of Science (B.S.), Computer Science at Shanghai Jiao Tong University
English
Github Skills (19)
spark10
google-cloud-dataproc10
databricks10
bash10
cloud-infrastructure10
rapids10
gpu10
documentation10
scala9
dask9
cuda8
ubuntu8
big-data8
gcp8
docker6
Programming languages (7)
Github contributions (5)
NVIDIA/spark-rapids
Mar 2021 - Dec 2022
Spark RAPIDS plugin - accelerate Apache Spark with GPUs
Role in this project:
Back-end Developer & Technical Writer
Back-end Developer & Technical WriterContributions:253 reviews, 57 commits, 95 PRs in 1 year 9 months
Contributions summary:Hao Zhu primarily focused on enhancing documentation and configuration scripts for the `spark-rapids` plugin. His contributions included updating documentation for decimal support, the 22.04, 22.06, 22.08, and 22.10 releases, and also updating GCP documentation and Databricks init scripts. He also addressed issues related to S3 configuration and corrected the version information in various demo scripts.
gpu-accelerationgpuapachebig-dataspark
GoogleCloudDataproc/initialization-actions
Mar 2021 - May 2022
Run in all nodes of your cluster before the cluster starts - lets you customize your cluster
Role in this project:DevOps Engineer & Cloud Engineer
DevOps Engineer & Cloud EngineerContributions:15 reviews, 11 commits, 16 PRs in 1 year 2 months
Contributions summary:Hao's contributions primarily revolve around the setup and configuration of GPU drivers and related dependencies within the Google Cloud Dataproc environment. They made significant changes to `install_gpu_driver.sh` for installing drivers on various operating systems and updating dependencies. Furthermore, the user made changes to the configuration for using Rapids within Dataproc, which included updating the Spark RAPIDS plugin version and modifying the Dask configuration. These changes highlight their focus on improving the utilization of GPUs within a Dataproc cluster.
nodesgoogle-cloud-dataprockubernetescustomizeterraform
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.