Shu Wang
Sr. Software Engineer at NVIDIA
Austin, Texas, United States
Summary
👤
Senior🎓
Top SchoolShu Wang is a Senior Software Engineer at NVIDIA with eight years of experience bridging high-performance ML frameworks and computational electromagnetics research. A Ph.D. candidate in Electrical Engineering with a minor in Applied Mathematics, Shu has deep expertise in domain decomposition algorithms, numerical PDEs, MPI/CUDA programming, and signal integrity analysis applied to full-wave IC board simulation. At NVIDIA he drives determinism, op fusion, and NV library integration for XLA, while contributing upstream to major open-source ML projects—including performance-focused FP8 and fused-attention work in TensorFlow, JAX, and Flax. Shu’s background in adjoint sensitivity analysis and large-scale parallel solvers informs a pragmatic approach to memory and precision trade-offs in GPU-accelerated ML kernels. Based in Austin, he blends academic rigor with production-grade engineering to optimize both algorithmic fidelity and runtime performance.
8 years of coding experience
6 years of employment as a software developer
Bachelor’s Degree, Optoelectronics, 3.55, Bachelor’s Degree, Optoelectronics, 3.55 at Nankai UniversityBachelor of Engineering (B.E.), Optoelectronics, 3.55, Bachelor of Engineering (B.E.), Optoelectronics, 3.55 at 天津大学Master's degree, Electrical and Electronics Engineering, 4.0, Master's degree, Electrical and Electronics Engineering, 4.0 at The University of New MexicoMaster of Engineering (ME), Laser and Optical Engineering, 3.8, Master of Engineering (ME), Laser and Optical Engineering, 3.8 at 北京航空航天大学
Chinese, English
Github Skills (13)
cuda10
machine-learning10
p810
tensorflow10
jax10
gpu-acceleration10
python10
linear-algebra10
flax10
numerical-computing9
cudnn9
neural-network9
back-end-development8
Programming languages (5)
Github contributions (5)
google/flax
Sep 2023 - Sep 2024
Flax is a neural network library for JAX that is designed for flexibility.
Role in this project:
ML Engineer
ML EngineerContributions:7 reviews, 7 PRs, 11 comments in 1 year
Contributions summary:Shu primarily contributed to the implementation of FP8 (8-bit floating-point) support within the Flax neural network library, specifically focusing on integrating it with the `TrainState` class and the `DenseGeneral` module. Their work involved defining custom operations and applying them within existing JAX-based training workflows, enhancing the library's capabilities. This includes incorporating amex history, scale factors for quantization, and backpropagation support.
deep-learningneural-networksneural-networkflaxjax
jax-ml/jax
Jun 2024 - Mar 2025
Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
Role in this project:ML Engineer
ML EngineerContributions:6 reviews, 7 PRs, 22 comments in 9 months
Contributions summary:Shu made several contributions focused on improving the JAX library's support for fused attention and FP8 operations. Their work involved fixing issues, adding tests, and enabling FP8 support for dot product attention within the CUDA backend. These changes included modifications to the backend configuration and underlying functions, specifically tailoring them for FP8 capabilities. Furthermore, the user added support for e8m0fnu, enhancing the range of supported data types within the project.
pytorchpythonjitautomatic-differentiationgpu
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.