🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT!🌏 Train a 26M-parameter GPT from scratch in just 2h!
Role in this project:
ML Engineer Contributions:1 release, 3 reviews, 11 PRs in 8 months
Contributions summary:Jingyao's commits primarily focus on implementing and updating the core machine learning model, likely a GPT variant, as indicated by the repository description. They were responsible for the initial open-source release of the MiniMind model, demonstrating the creation and configuration of model components, including attention mechanisms, feedforward networks, and the integration of rotary positional embeddings. Subsequent commits involve updates to the model, dataset processing, and evaluation scripts, suggesting ongoing development and improvement of the language model's architecture and training pipeline.
artificial-intelligencelarge-language-model
Contributions:2 releases, 7 commits, 2 pushes in 4 months