Junjie Wang
Software Engineer at Meta
Mountain View, California, United States
Summary
🤩
Rockstar🎓
Top SchoolJunjie Wang is a software engineer with 11 years of experience building performant back-end systems and distributed ML infrastructure, currently contributing to Meta and the PyTorch distributed project. He specializes in optimizing collective communications and enabling tensor-parallel and sequence-parallel training patterns, including sharded tensor operations and embedding performance improvements in the widely used pytorch/pytorch repo. Junjie blends hands-on performance engineering with production product delivery—previous roles span engineering and management at Afterpay and DataVisor and full-stack work at Shutterfly. Based in Mountain View, he pairs an M.Eng. in Computer Engineering with practical experience in parallel computing techniques (MPI) and a pragmatic “show me the code” ethos.
11 years of coding experience
6 years of employment as a software developer
Bachelor's degree, Electrical, Electronics and Communications Engineering, Bachelor's degree, Electrical, Electronics and Communications Engineering at Huazhong University of Science and TechnologyMaster of Engineering (M.Eng.), Computer Engineering, Master of Engineering (M.Eng.), Computer Engineering at Stevens Institute of Technology
English, Chinese
Github Skills (11)
ccl10
cuda10
pytorch10
machine-learning10
distributed-training10
deep-learning10
performance-optimization10
python10
gpu9
back-end-development9
fs9
Programming languages (4)
Github contributions (5)
pytorch/examples
May 2022 - Dec 2022
A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
Role in this project:
ML Engineer
ML EngineerContributions:8 reviews, 14 commits, 20 PRs in 6 months
Contributions summary:Junjie contributed to examples within the PyTorch library, specifically focusing on tensor parallelism (TP) and sequence parallelism (SP) for distributed training. Their work involved creating and updating examples demonstrating the use of new TP APIs and integrating TP and DDP (Distributed Data Parallel) into the example run scripts. The commits also include the addition of examples for 2D parallel training, combining TP/SP with Fully Sharded Data Parallel (FSDP).
pytorchvisiondeep-learningreinforcement-learningreinforcement
pytorch/pytorch
Oct 2021 - Jan 2023
Tensors and Dynamic neural networks in Python with strong GPU acceleration
Role in this project:Back-end Developer & Performance Engineer
Back-end Developer & Performance EngineerContributions:1488 reviews, 240 commits, 261 PRs in 1 year 3 months
Contributions summary:Junjie primarily contributed to the PyTorch distributed training library, focusing on enabling and optimizing distributed operations for tensor parallelism. They worked on enabling the `nan_to_num` operation for sharded tensors and enhancing features related to partial tensor operations and embedding operations. The user's contributions also involve optimizing collective communications for the embedding and embedding bag operations, aiming to improve model training efficiency.
pythongpu-accelerationdeep-learninggpunumpy
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.