LazyLLM: Low-Code Development for Multi-Agent LLM Applications

Introduction

LazyLLM is a powerful, low-code development tool that simplifies the creation of multi-agent Large Language Model (LLM) applications. It enables developers to build complex AI solutions with minimal effort, supporting continuous iteration and optimization. LazyLLM provides a convenient workflow for application setup, along with numerous standard processes and tools for every stage of development. Its core philosophy emphasizes rapid prototyping, bad-case analysis with scenario-specific data, and iterative algorithm refinement and model fine-tuning to progressively enhance application performance.

Installation

Installing LazyLLM is straightforward. You can choose between a minimal installation or a full installation that includes all dependencies.

To install LazyLLM and its necessary dependencies:

pip3 install lazyllm

For a full installation, including all dependencies:

pip3 install lazyllm
lazyllm install full

For installation from source or on Windows/macOS, please refer to the official documentation.

Examples

LazyLLM can be used to build a variety of AI applications. Here are a few examples:

1. Chatbot

You can quickly set up a simple chatbot using LazyLLM.

import lazyllm
chat = lazyllm.OnlineChatModule()
lazyllm.WebModule(chat).start().wait()

To use a locally deployed model, ensure you have at least one inference framework (like LightLLM or vLLM) installed:

import lazyllm
# Model will be downloaded automatically if you have an internet connection.
chat = lazyllm.TrainableModule('internlm2-chat-7b')
lazyllm.WebModule(chat, port=23466).start().wait()

For a more advanced multimodal chatbot with intent recognition, LazyLLM allows for complex pipelines integrating various models for tasks like speech recognition, image Q&A, drawing, and music generation.

2. Retrieval Augmented Generation (RAG)

LazyLLM also facilitates the creation of RAG applications, combining retrieval and generation for enhanced responses.

Here's an example for an online deployment:

import lazyllm
from lazyllm import pipeline, parallel, bind, _0, Document, Retriever, Reranker

prompt = 'You will act as an AI question-answering assistant to complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'

documents = Document(dataset_path="file/to/yourpath", embed=lazyllm.OnlineEmbeddingModule(), manager=False)
documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)
with pipeline() as ppl:
    with parallel().sum as ppl.prl:
        prl.retriever1 = Retriever(documents, group_name="sentences", similarity="cosine", topk=3)
        prl.retriever2 = Retriever(documents, "CoarseChunk", "bm25_chinese", 0.003, topk=3)

    ppl.reranker = Reranker("ModuleReranker", model="bge-reranker-large", topk=1) | bind(query=ppl.input)
    ppl.formatter = (lambda nodes, query: dict(context_str="".join([node.get_content() for node in nodes]), query=query)) | bind(query=ppl.input)
    ppl.llm = lazyllm.OnlineChatModule(stream=False).prompt(lazyllm.ChatPrompter(prompt, extra_keys=["context_str"]))

lazyllm.WebModule(ppl, port=23466).start().wait()

More examples, including painting masters, multimodal chatbots, knowledge bases, and various intelligent agents, can be found in the official documentation.

Why Use LazyLLM

LazyLLM is designed to free algorithm researchers and developers from complex engineering implementations, allowing them to focus on algorithms and data. It simplifies the AI application building process for beginners, eliminating concerns about task scheduling, API service details, model fine-tuning frameworks, or web development. For experienced experts, LazyLLM offers high flexibility, with customizable and extensible modules to integrate custom algorithms and advanced production tools. The framework provides a consistent user experience across different service providers and models, whether online or local, and carefully integrates 2-3 of the most advantageous tools for each component, prioritizing quality and practical effectiveness over sheer quantity.