Repository History
258 repositories tagged with AI
OpenWebAgent: An Open Toolkit for LLM- and LMM-based Web Agents
OpenWebAgent is an open toolkit designed to empower model-based web agents, streamlining human-computer interactions by automating tasks on webpages. It offers a convenient framework for developing LLM- and LMM-based web agents, providing both plugin and server source code for easy integration and customization. This project was featured as an ACL'24 Demo, showcasing its innovative approach to web automation.
ChatArena: Multi-Agent Language Game Environments for LLMs
ChatArena is a Python library designed to provide multi-agent language game environments for Large Language Models (LLMs), aiming to foster the development of communication and collaboration capabilities in AI. It offers a flexible framework for defining players, environments, and interactions based on Markov Decision Processes. Please note that as of August 11, 2025, this project has been deprecated due to a lack of widespread community use and is no longer receiving updates or support.
Agentarium: A Python Framework for AI Agent Simulations
Agentarium is an open-source Python framework designed for creating and managing simulations with AI-powered agents. It offers an intuitive platform for designing complex, interactive environments where agents can act, learn, and evolve. This powerful tool simplifies the orchestration of multiple AI agents and their interactions.
Lighteval: Your All-in-One Toolkit for LLM Evaluation
Lighteval is a comprehensive toolkit from Hugging Face for evaluating Large Language Models (LLMs) across various backends. It enables users to dive deep into model performance by saving detailed, sample-by-sample results and supports over 1000 evaluation tasks. The framework offers extensive customization options, allowing users to create custom tasks and metrics tailored to their specific needs.
PromptBench: A Unified Framework for LLM Evaluation and Robustness
PromptBench is a comprehensive Python library designed for the evaluation and understanding of Large Language Models (LLMs). It provides a unified framework for assessing model performance, exploring various prompt engineering techniques, and evaluating robustness against adversarial attacks. This tool empowers researchers to conduct in-depth analyses of LLMs across diverse datasets and models.
EvalPlus: Rigorous Evaluation for LLM-Synthesized Code
EvalPlus is a robust framework designed for the rigorous evaluation of code generated by Large Language Models (LLMs). It extends standard benchmarks like HumanEval and MBPP with significantly more tests, offering precise assessment of code correctness and efficiency. This tool is crucial for developers and researchers aiming to thoroughly validate LLM-synthesized code.
AgentEvals: Robust Evaluation Tools for LLM Agent Trajectories
AgentEvals is a powerful open-source package from LangChain designed to simplify the evaluation of agentic applications. It provides a collection of ready-made evaluators and utilities, with a particular focus on analyzing agent trajectories, the intermediate steps an agent takes to solve problems. This helps developers understand and improve the reliability and performance of their LLM agents.
no-mistakes: AI-Driven Git Proxy for Flawless Pull Requests
no-mistakes is an innovative Git proxy that streamlines the pull request workflow by ensuring code quality before it reaches your remote. It uses an AI-driven validation pipeline in a disposable worktree, automatically applying safe fixes and escalating complex issues for human review. This tool helps developers maintain clean, high-quality codebases and open perfect PRs effortlessly.
XGrammar: Fast, Flexible, and Portable Structured Generation for LLMs
XGrammar is an open-source library for efficient, flexible, and portable structured generation, developed by mlc-ai. It leverages constrained decoding to guarantee 100% structural correctness for outputs like JSON and regex. Optimized for near-zero overhead, XGrammar offers universal deployment across various platforms, hardware, and programming languages, making it a leading solution for structured output from large language models.
FreeLLMAPI: Stack 16 Free LLM Tiers for 1.7 Billion Tokens/Month
FreeLLMAPI is an OpenAI-compatible proxy that aggregates the free tiers of 16 LLM providers, offering access to approximately 1.7 billion tokens per month. It simplifies access to diverse models through a single endpoint, featuring smart routing, automatic failover, and encrypted key storage. This powerful tool is designed for personal experimentation, allowing developers to leverage multiple free LLM resources efficiently.
Jsonformer: Bulletproof Structured JSON Generation from Language Models
Jsonformer is a powerful library designed to generate syntactically correct and schema-conforming JSON from language models. It addresses the common challenge of unreliable JSON output by focusing on generating only content tokens, making the process more efficient and robust. This approach ensures bulletproof structured data generation for various applications.
Hiring Agent: An AI Agent for Resume Evaluation and Scoring
Hiring Agent is an open-source AI agent designed to evaluate and score resumes objectively. It extracts structured data from PDF resumes, enriches it with GitHub profile signals, and provides a fair, explainable evaluation with detailed scores and evidence. This tool supports both local LLMs via Ollama and cloud-based options like Google Gemini.