Repository History
103 repositories tagged with LLM
Colibri: Run 744B GLM-5.2 MoE on Consumer Machines with Pure C
Colibri is an innovative project that enables running the massive 744B-parameter GLM-5.2 Mixture-of-Experts (MoE) model on consumer-grade machines with as little as 25GB of RAM. It achieves this remarkable feat through a pure C engine with zero dependencies, streaming model experts from disk on demand. This allows users to interact with a frontier-class LLM without requiring expensive GPU hardware.
TencentDB Agent Memory: Enhancing AI Agents with Layered Long-Term Memory
TencentDB Agent Memory provides AI agents with fully local, long-term memory through a 4-tier progressive pipeline, eliminating external API dependencies. It significantly reduces token usage and improves task success rates by employing symbolic short-term memory and layered long-term memory. This innovative approach helps agents learn workflows and retain context more effectively.
Ludwig: Low-Code Declarative Deep Learning for LLMs and AI Models
Ludwig is a powerful, low-code declarative deep learning framework designed for building custom LLMs, neural networks, and other AI models. It simplifies the process of training, fine-tuning, and deploying models, from LLM fine-tuning to tabular classification, using a simple YAML configuration without boilerplate Python code. This makes advanced AI development accessible and efficient for a wide range of applications.
Lamini: The Official Python Client for Generative AI API
Lamini is the official Python client and SDK designed to interact with the Lamini API, enabling developers to create their own Generative AI applications. It provides a straightforward interface for integrating powerful AI capabilities into Python projects. This package simplifies the process of building and deploying generative AI solutions.
xTuring: Build, Personalize, and Control Your Own LLMs
xTuring is an open-source framework designed to simplify the process of building, personalizing, and controlling Large Language Models (LLMs). It provides an easy way to fine-tune open-source LLMs on your own data, offering features from data pre-processing to efficient training and inference. This tool empowers developers to create private, personalized LLMs locally or in their private cloud environments.
torchtune: PyTorch Native Library for LLM Post-Training and Experimentation
torchtune is a PyTorch native library designed for authoring, post-training, and experimenting with Large Language Models (LLMs). It offers hackable training recipes, simple PyTorch implementations of popular LLMs, and best-in-class memory efficiency. Please note: torchtune is no longer actively maintained as of 2025.
RAGChecker: A Fine-grained Framework for Diagnosing RAG Systems
RAGChecker is an advanced automatic evaluation framework developed by Amazon Science, specifically designed to assess and diagnose Retrieval-Augmented Generation (RAG) systems. It offers a comprehensive suite of metrics and tools for in-depth analysis of RAG performance. This framework empowers developers and researchers to thoroughly evaluate and enhance their RAG systems with precision.
rerankers: Unified API for Reranking and Cross-Encoder Models
rerankers is a lightweight, low-dependency Python library that provides a unified API for various reranking and cross-encoder models. It simplifies the integration of different reranking approaches into retrieval architectures, offering a consistent interface for diverse models like cross-encoders, RankGPT, T5, and API-based rerankers. This library aims to make reranking more accessible and easier to implement for developers.
LLM Compressor: Optimize LLMs for Deployment with vLLM
LLM Compressor is a Transformers-compatible Python library designed to apply various compression algorithms to Large Language Models (LLMs). It enables optimized deployment, especially with vLLM, by offering a comprehensive set of quantization techniques for weights, activations, and KV Cache. This tool seamlessly integrates with Hugging Face models, making LLM optimization accessible and efficient.
LightLLM: A Lightweight and High-Speed LLM Inference and Serving Framework
LightLLM is a Python-based framework designed for efficient Large Language Model (LLM) inference and serving. It stands out for its lightweight architecture, impressive scalability, and high-speed performance, making it an excellent choice for deploying LLMs. The framework integrates and builds upon the strengths of various leading open-source implementations to deliver optimized results.
TensorRT-LLM: Optimizing Large Language Model Inference on NVIDIA GPUs
TensorRT-LLM is an open-source library by NVIDIA designed to optimize inference for Large Language Models (LLMs) and Visual Generation models. It offers a user-friendly Python API, state-of-the-art optimizations, and specialized kernels to ensure efficient performance on NVIDIA GPUs. This powerful tool enables developers to deploy LLMs with high throughput and low latency, from single-GPU setups to multi-node deployments.
DataDreamer: Streamlining Synthetic Data Generation and LLM Workflows
DataDreamer is an open-source Python library designed for efficient prompting, synthetic data generation, and model training workflows. It simplifies the process of creating complex LLM workflows, generating high-quality synthetic datasets, and aligning or fine-tuning models. Built to be simple, efficient, and research-grade, DataDreamer empowers users to build reproducible and shareable AI solutions.