A framework for few-shot evaluation of language models.
Role in this project:
ML Engineer Contributions:7 commits, 1 PR in 5 days
Contributions summary:Eric primarily contributed to the `lm-evaluation-harness` repository by adding and modifying math tasks within the evaluation framework. Their work involved defining the structure of math problems, implementing functions to process results, and handling string parsing for answer extraction. They also made iterative updates to the code, including fixes for string parsing and removal of testing code.
language-modelevaluation-frameworktransformer
SkyRL-v0: Train Real-World Long-Horizon Agents via Reinforcement Learning
Contributions:2 PRs, 553 pushes, 176 branches in 1 year 1 month