Sage Moore
Software Engineer at neuralmagic
Greater Boston Area United States
Summary
🤩
Rockstar🎓
Top SchoolSage Moore is a software engineer with 11 years of experience focused on high-throughput ML inference and production-ready runtime engineering. Based in Greater Boston and currently at NeuralMagic, Sage has deep hands-on experience optimizing inference engines—contributing ROCm support and kernel fixes to vllm and enhancing deepsparse with elastic benchmarking and multi-model concurrency. Their work spans MLOps and ML engineering, bridging low-level kernel compatibility with higher-level stream and context management for efficient CPU and non-CUDA deployments. Sage’s contributions to well-known open-source projects demonstrate a knack for making cutting-edge models practical at scale. They hold a Computer Science degree from UT Austin and consistently push performance and portability trade-offs in real-world systems. An under-the-radar strength is their ability to translate kernel-level changes into measurable CI and runtime improvements across heterogeneous hardware.
11 years of coding experience
Bachelor's degree, Computer Science, Bachelor's degree, Computer Science at The University of Texas at Austin
Github Skills (21)
benchmark10
performance-monitor10
python10
machine-learning10
benchmarking10
inference10
amd10
roc10
mlops10
performance-analysis10
cuda10
pytorch9
multithreading9
onnx9
nlp8
Programming languages (2)
Github contributions (5)
vllm-project/vllm
Feb 2024 - Apr 2025
A high-throughput and memory-efficient inference and serving engine for LLMs
Role in this project:
MLOps Engineer
MLOps EngineerContributions:83 reviews, 18 PRs, 113 comments in 1 year 2 months
Contributions summary:Sage primarily contributed to the optimization and integration of the VLLM project with ROCm platforms. This includes adapting the code to ensure compatibility, particularly by updating the reshape_and_cache functions. Additionally, they worked on correcting kernel-level issues and disabling specific features, such as moe_wna16_gemm and chunked prefill/paged attention, on non-CUDA platforms, and fixing the AMD CI groups to improve performance. The contributions suggest a focus on enabling and refining ROCm support for high-throughput and memory-efficient inference.
amdcudadeepseekgpthpu
neuralmagic/deepsparse
Feb 2022 - Aug 2022
Sparsity-aware deep learning inference runtime for CPUs
Role in this project:ML Engineer
ML EngineerContributions:21 reviews, 10 commits, 19 PRs in 6 months
Contributions summary:Sage primarily focused on enhancing the deepsparse inference runtime, adding and refining benchmark scenarios, and optimizing the engine's performance. They added an "elastic" scenario to the benchmarking tools, which involved modifications to the benchmark script and stream handling. Their contributions included adding and integrating new classes, such as `Context` and `MultiModelEngine`, to support concurrent model execution. Furthermore, the user modified the engine to align with updated stream management practices.
llm-inferenceruntimetensorflowsparsificationmachinelearning
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.