joinly: Make Your Meetings Accessible to AI Agents

Summary

joinly.ai is an open-source connector middleware designed to integrate AI agents into video calls. It enables agents to actively participate, interact in real-time, and perform tasks during meetings across platforms like Google Meet, Zoom, and Microsoft Teams. The project emphasizes a privacy-first approach, offering self-hosting capabilities and flexibility with various LLM, TTS, and STT providers.

Use at your own risk

OSRepos shares public repositories for knowledge and discovery only. Any installation, execution, configuration, or use of code from these repositories is the user's own responsibility. Always review the repository, source code, dependencies, licenses, and security implications before running or installing anything. OSRepos is not responsible for issues, damages, or losses resulting from third-party repositories.

Introduction

joinly.ai is a powerful open-source connector middleware that empowers AI agents to seamlessly join and actively engage in video conferences. It transforms your meetings by allowing AI agents to perform tasks, respond in real-time via voice or chat, and interact naturally with participants. Built with a focus on flexibility and privacy, joinly.ai supports various LLM providers, Text-to-Speech (TTS), and Speech-to-Text (STT) services, making it a versatile tool for enhancing productivity and automation in virtual meetings.

Key features include live interaction capabilities, a built-in conversational flow to handle interruptions and multi-speaker scenarios, and cross-platform compatibility with major video conferencing tools. It is 100% open-source, self-hostable, and privacy-first, giving users full control over their data and AI integrations.

Installation

Getting started with joinly.ai is straightforward using Docker. This quick guide will help you run a basic conversational agent client.

Prerequisites: Ensure you have Docker installed on your system.

Create an .env file: In a new directory, create an .env file with your desired LLM provider's API key. For example, using OpenAI:

# .env
JOINLY_LLM_MODEL=gpt-4o
JOINLY_LLM_PROVIDER=openai
OPENAI_API_KEY=your-openai-api-key

Pull the Docker image:

docker pull ghcr.io/joinly-ai/joinly:latest

Launch joinly in a meeting: Start your meeting on Zoom, Google Meet, or Microsoft Teams. Then, run the following command from the directory containing your .env file, replacing <MeetingURL> with your meeting link:

docker run --env-file .env ghcr.io/joinly-ai/joinly:latest --client <MeetingURL>

For more advanced configurations, including GPU support or running an external client, refer to the official documentation.

Examples

joinly.ai showcases its capabilities through practical demonstrations where AI agents perform real-world tasks within meetings:

GitHub Demo: An AI agent answers questions by accessing web information and then creates an issue in a GitHub repository, all live during a meeting.
Notion Demo: An agent connects to Notion via the MCP (Meeting Control Protocol) and edits page content in real-time during a meeting.

These examples highlight the potential for automation and intelligent assistance that joinly.ai brings to collaborative environments.

Why Use joinly?

joinly.ai offers compelling reasons for integration into your workflow:

Live Interaction: Agents can execute tasks and respond instantly, either by voice or chat, enhancing real-time collaboration.
Natural Conversations: Its built-in logic manages interruptions and multi-speaker interactions, ensuring a smooth conversational flow.
Broad Compatibility: Works across popular platforms like Google Meet, Zoom, and Microsoft Teams, providing flexibility.
Customizable AI Stack: Supports various LLM providers, including local options like Ollama, and allows you to choose your preferred TTS/STT services (Whisper, Deepgram, ElevenLabs, Kokoro).
Open-Source and Privacy-First: Being 100% open-source and self-hostable, it prioritizes user privacy and control over data.
Extensible: The MCP server provides a rich set of tools for agents, including joining/leaving meetings, speaking text, sending chat messages, getting chat history, transcripts, and even video snapshots.