Jian He
Summary
🤩
RockstarJian He is a software engineer with 12 years of experience building scalable, cost-efficient ML and data infrastructure across companies like Meta and Alibaba. He specializes in optimizing large-scale GPU training and heterogeneous CPU/GPU Kubernetes clusters to drive significant resource and performance gains for production ML workloads. A long-time Apache Hadoop committer and PMC member, Jian has deep roots in distributed systems and resource management, contributing substantive YARN and RM improvements used widely in big-data environments. At Meta he leads Ads ML Training production work to scale PyTorch ranking models, and at Alibaba he helped open-source projects such as KubeDL and Morphling that simplify ML on Kubernetes. His practical blend of low-level resource management, controller/webhook engineering, and MLOps reliability work makes him equally comfortable debugging kernel-scale issues and shaping platform strategy. Based in the Bay Area, he pairs hands-on coding with cross-team leadership to bring complex ML systems into efficient production.
12 years of coding experience
Github Skills (30)
kubernetes10
container10
cap10
apache-hadoop10
hadoop10
resource-management10
java10
scheduler10
cnc10
javas10
mlops10
go10
sched10
tensorflow10
state-machine10
Programming languages (11)
Github contributions (5)
apache/hadoop-common
Jun 2014 - Aug 2014
Mirror of Apache Hadoop common
Role in this project:
Back-end Developer
Back-end DeveloperContributions:52 commits in 2 months
Contributions summary:Jian contributed code to augment the RMStateStore with a state machine, which involved adding new states and transitions for application management. They also fixed a potential null pointer exception in the ProportionalCapacityPreemptionPolicy and changed it to log CSV data at a debug level. Further contributions included fixing an issue where the RM might not send application-clean-up signals to NMs after RM restart and fixing a bug in GetApplicationsRequestPBImpl.
apache-hadoophadoop
apache/hadoop
Feb 2014 - Feb 2018
Apache Hadoop
Role in this project:Back-end Developer
Back-end DeveloperContributions:497 commits in 4 years
Contributions summary:Jian contributed significantly to the development of Apache Hadoop, focusing on features related to resource management within the YARN framework. Their work involved enhancements to the Resource Manager (RM) to support container resizing, and improve the performance of the system. They were also involved in adding new endpoints for updating application timeouts and improving the consistency of the RM web UI. They also worked on refactoring core components related to ZK StateStore and improving various lock mechanisms to prevent deadlocks.
apache-hadoophadoop
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.