Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.
Role in this project:
ML Engineer Contributions:3 reviews, 31 PRs, 34 comments in 7 months
Contributions summary:Zijie contributed significantly to the implementation and examples of LLM inference and fine-tuning, particularly for MiniCPM and GLM-4 models. Their work included adding GPU and CPU examples, adapting code for NPU, and updating the examples for Llama3.1 and Llama3.2-vision. They demonstrated expertise in integrating models with the IPEX-LLM library.