Thor Johnsen
Deep Learning Software Engineer at NVIDIA
Houston, Texas, United States
Summary
🤩
Rockstar🎓
Top SchoolThor Johnsen is a Deep Learning Software Engineer with 7+ years focused on scaling and optimizing large-scale ML training, currently at NVIDIA where he develops high-performance PyTorch and TensorFlow components to push GPUs to maximal scale and fastest convergence. He brings two decades of HPC and domain expertise from Chevron and Schlumberger building parallelized seismic modeling, imaging and inversion codes across CPU/GPU clusters and accelerators. An experienced CUDA developer and optimizer, Thor has contributed to NVIDIA's widely used apex library—implementing fused layernorm and multi-tensor LAMB kernels and improving distributed allreduce behavior. He combines low-level performance engineering (including CUDA kernels and assembly-era experience) with a deep understanding of physics-driven applications, enabling practical speedups for both research and industry workflows. Based in Houston, he blends production-grade ML systems work with subject-matter expertise in geophysics, making him a go-to engineer for maximal-scale training and computationally intensive simulation stacks.
8 years of coding experience
18 years of employment as a software developer
Norwegian University of Science and Technology
Github Skills (16)
kernel10
cuda10
pytorch10
distributed-training10
deep-learning10
optimizers9
optimizer9
machine-learning9
back-end-development8
cprogramming-language7
tensorflow7
operation7
tensor7
tensorrt7
c-language7
Programming languages (3)
Github contributions (5)
NVIDIA/apex
Oct 2018 - Nov 2022
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
Role in this project:
Back-end Developer & ML Engineer
Back-end Developer & ML EngineerContributions:4 reviews, 242 commits, 45 PRs in 4 years 1 month
Contributions summary:Thor primarily contributed to the `nvidia/apex` repository, which focuses on PyTorch extensions for mixed precision and distributed training. Their commits involved implementing and optimizing CUDA kernels, specifically for fused layer normalization, and the multi-tensor LAMB optimizer. The user also added features related to allreduce in DDP and worked on general bug fixes related to the optimizer.
distributed-trainingpytorch
thorjohnsen/bert
Dec 2018 - Feb 2019
TensorFlow code and pre-trained models for BERT
Contributions:22 commits, 17 pushes, 6 branches in 2 months
bertpre-trained-modeltensorflow
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.