Scott Gray
Member Of Technical Staff at OpenAI
San Francisco, California, United States
Summary
🤩
Rockstar🎓
Top SchoolScott Gray is a Member of Technical Staff at OpenAI with 11 years of engineering experience focused on high-performance ML infrastructure and GPU-optimized kernels. Trained in both physics and computer science at UIUC, he brings a systems-minded approach to squeezing maximum performance from hardware across architectures. His open-source work includes significant contributions to blocksparse and Intel Nervana's neon framework, where he implemented optimized convolution/pooling engines, batch-norm improvements, and fused operations that materially speed sparse transformer and CNN workloads. Based in San Francisco, Scott blends deep low-level CUDA/kernel expertise with practical backend engineering to move research code into production. Colleagues rely on him for performance debugging and for adapting algorithms to real-world hardware constraints. He often tackles non-obvious bottlenecks such as asymmetric query/key dimensions and fused op paths to unlock model efficiency gains.
11 years of coding experience
Physics, Computer Science, Physics, Computer Science at University of Illinois Urbana-Champaign
Github Skills (21)
transformers10
sparse-array10
convolutional-neural-networks10
gpgpu10
batch-normalization10
python10
gpu-programming10
sparse-matrix10
machine-learning10
sparse-file10
kernel10
deep-learning10
tensorflow10
gpu10
performance-optimization10
Programming languages (5)
Github contributions (5)
openai/blocksparse
Jan 2018 - Jul 2019
Efficient GPU kernels for block-sparse matrix multiplication and convolution
Role in this project:
ML Engineer
ML EngineerContributions:21 commits, 3 PRs, 21 pushes in 1 year 6 months
Contributions summary:Scott primarily contributed to the development and optimization of GPU kernels within the blocksparse library, which focuses on efficient sparse matrix operations. Their work includes implementing support for specific hardware architectures like sm_50, improving performance through code changes, and adding features such as fused softmax_cross_entropy operations. Furthermore, they integrated support for asymetric query/key dimensions, enhancing the library's applicability to diverse machine learning models, particularly in the context of sparse transformers.
cudamatrix-multiplicationsparse-matrixgpusparse
NervanaSystems/neon
Sep 2015 - Jul 2016
Intel® Nervana™ reference deep learning framework committed to best performance on all hardware
Role in this project:Backend Developer
Backend DeveloperContributions:36 commits, 35 comments in 9 months
Contributions summary:Scott primarily contributed to the development and optimization of the Intel Nervana deep learning framework, specifically focusing on improving the performance of convolutional neural networks. Their commits include implementing new engines for convolution and pooling, enhancing batch normalization, and creating a new build system for kernels. These improvements were benchmarked and tested to ensure optimal performance on various hardware platforms.
deep-learningbest-performanceintelmachine-learningperformance
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.