langcorn: Serve LangChain LLM Apps and Agents with FastAPI

Introduction

Langcorn is an API server that simplifies the deployment of LangChain models and pipelines. It integrates seamlessly with FastAPI, providing a high-performance, scalable, and robust solution for serving your language processing applications and agents. Designed for LLMops, Langcorn automates the serving process, allowing developers to focus on building powerful LLM-powered experiences.

Key features include easy deployment, ready-to-use authentication, well-documented RESTful API endpoints, asynchronous processing, and support for custom pipelines and processing.

Installation

Getting started with Langcorn is straightforward. You can install the package using pip:

pip install langcorn

Examples

Quick Start

To run a single LangChain chain, define it in a Python file (e.g., examples.ex1:chain) and then start the Langcorn server:

langcorn server examples.ex1:chain

Alternatively, you can run it as a Python module:

python -m langcorn server examples.ex1:chain

Serving Multiple Chains

Langcorn supports serving multiple chains simultaneously. Simply list them when starting the server:

python -m langcorn server examples.ex1:chain examples.ex2:chain

Integrating with FastAPI

For more control, you can integrate Langcorn directly into an existing FastAPI application:

from fastapi import FastAPI
from langcorn import create_service

app: FastAPI = create_service("examples.ex1:chain")

# To serve multiple chains:
# app: FastAPI = create_service("examples.ex2:chain", "examples.ex1:chain")

# Then run your FastAPI app with Uvicorn:
# uvicorn main:app --host 0.0.0.0 --port 8000

Why Use It

Langcorn addresses the challenges of deploying LangChain applications by offering a streamlined, high-performance solution. Its integration with FastAPI ensures robust API serving, while features like built-in authentication and custom API key handling provide necessary security and flexibility. Developers benefit from automatic documentation via FastAPI's /docs endpoint, asynchronous processing for faster responses, and the ability to manage LLM kwargs per request. It also provides structured handling for LLM memory, making it ideal for conversational AI applications. Langcorn simplifies the operational aspects of LLM deployment, allowing you to focus on the core logic of your AI agents and applications.

Links

• GitHub Repository: https://github.com/msoedov/langcorn
• Live Example: https://langcorn-ift9ub8zg-msoedov.vercel.app/docs#/