Repository History
5 repositories tagged with fine-tuning
Axolotl: Streamlining LLM Fine-tuning with a Powerful Open-Source Framework
Axolotl is a comprehensive, free, and open-source framework designed to simplify the post-training and fine-tuning processes for large language models (LLMs). It offers extensive model support, diverse training methods, and robust performance optimizations, making it an invaluable tool for researchers and developers. With easy configuration and cloud-ready deployment, Axolotl empowers users to efficiently customize and enhance LLMs.
Mergoo: Efficiently Merge and Train Multiple LLM Experts
Mergoo is an open-source Python library designed to simplify the merging of multiple Large Language Model (LLM) experts. It enables efficient training of these merged LLMs, allowing users to integrate knowledge from various generic or domain-specific models. The library supports several merging methods, including Mixture-of-Experts and Mixture-of-Adapters, across popular base models.
PEFT: State-of-the-Art Parameter-Efficient Fine-Tuning
PEFT (Parameter-Efficient Fine-Tuning) is a cutting-edge library from Hugging Face designed to efficiently adapt large pretrained models for various downstream applications. It dramatically reduces computational and storage costs by fine-tuning only a small subset of model parameters. This approach enables achieving performance comparable to fully fine-tuned models, making advanced AI accessible on more modest hardware.
maestro: Streamlining Fine-Tuning for Multimodal Models like PaliGemma 2 and Florence-2
maestro is a powerful tool designed to accelerate the fine-tuning process for multimodal models. It encapsulates best practices, handling configuration, data loading, reproducibility, and training loop setup efficiently. The project currently offers ready-to-use recipes for popular vision-language models, including Florence-2, PaliGemma 2, and Qwen2.5-VL.
LLaMA-Factory: Unified Efficient Fine-Tuning for 100+ LLMs & VLMs
LLaMA-Factory is an open-source project offering a unified and efficient framework for fine-tuning over 100 large language models (LLMs) and vision-language models (VLMs). Recognized at ACL 2024, it provides a comprehensive suite of tools and algorithms for various training approaches. This repository simplifies the complex process of adapting powerful models for specific tasks with ease and scalability.