A fast MoE impl for PyTorch
Role in this project:
ML Engineer Contributions:12 reviews, 143 commits, 12 PRs in 1 year
Contributions summary:Jiezhong primarily focused on implementing and modifying components for a fast Mixture of Experts (MoE) implementation within a PyTorch environment. Their contributions included adding and refining memory-to-memory attention mechanisms, and incorporating different position-wise feed-forward network configurations, including hierarchical and sparse approaches. They also worked on resolving bugs, and incorporating a customized CUDA kernel for matrix multiplication within the MoE architecture.
pytorch
DeepInf: Social Influence Prediction with Deep Learning
Contributions:20 commits, 13 pushes, 7 comments in 4 months
deep-learningpytorchsocial-networkgraph-attention-networksgraph-convolutional-networks