{"name":"KBLaM: Knowledge Base Augmented Language Models for Enhanced LLMs","description":"KBLaM, developed by Microsoft, is the official implementation of \"Knowledge Base Augmented Language Models\" presented at ICLR 2025. This innovative method enhances Large Language Models by directly integrating external knowledge bases, offering an efficient alternative to traditional Retrieval-Augmented Generation (RAG) and in-context learning. It eliminates external retrieval modules and scales computationally linearly with knowledge base size, rather than quadratically.","github":"https://github.com/microsoft/KBLaM","url":"https://osrepos.com/repo/microsoft-kblam","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/microsoft-kblam","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/microsoft-kblam.md","json":"https://osrepos.com/repo/microsoft-kblam.json","topics":["Jupyter Notebook","AI","LLM","Machine Learning","Knowledge Base","Natural Language Processing","Deep Learning","Microsoft"],"keywords":["Jupyter Notebook","AI","LLM","Machine Learning","Knowledge Base","Natural Language Processing","Deep Learning","Microsoft"],"stars":null,"summary":"KBLaM, developed by Microsoft, is the official implementation of \"Knowledge Base Augmented Language Models\" presented at ICLR 2025. This innovative method enhances Large Language Models by directly integrating external knowledge bases, offering an efficient alternative to traditional Retrieval-Augmented Generation (RAG) and in-context learning. It eliminates external retrieval modules and scales computationally linearly with knowledge base size, rather than quadratically.","content":"## Introduction\nKBLaM, developed by Microsoft, is the official implementation of \"Knowledge Base Augmented Language Models\" (ICLR 2025). This project introduces a novel approach to augmenting Large Language Models (LLMs) with external knowledge. Unlike Retrieval-Augmented Generation (RAG), KBLaM eliminates the need for external retrieval modules, and it offers a significant advantage over in-context learning by ensuring its computational overhead scales linearly with the knowledge base size, rather than quadratically. This makes KBLaM a promising solution for integrating vast amounts of knowledge into LLMs more efficiently.\n\n## Installation\nGetting started with KBLaM is straightforward. You can install the package directly from the repository:\n\nbash\npip install -e .\n\n\nTo utilize Llama models, you will need to generate a token from Hugging Face and log in via the command line:\n\nbash\npip install huggingface_hub\nhuggingface-cli login\n\n\n## Examples\nKBLaM provides tools for synthetic dataset construction and model training. For dataset generation, you'll need an Azure OpenAI endpoint. You can construct synthetic knowledge bases and question-answer pairs using `dataset_generation/gen_synthetic_data.py` and generate KB embeddings with `dataset_generation/generate_kb_embeddings.py`. Supported embeddings include `text-embedding-ada-002` and `all-MiniLM-L6-v2`.\n\nAn example of model training is provided:\n\nbash\npython train.py --dataset_dir <Your dataset directory> --train_dataset synthetic --N 120000 --B 20 --total_steps 601  --encoder_spec OAI --use_oai_embd --key_embd_src key --use_data_aug --use_cached_embed\n\n\nIt is recommended to use the `--use_cached_embed` argument to prevent recomputation of embeddings, which can be time-consuming.\n\n## Why Use KBLaM\nKBLaM offers a compelling alternative for enhancing LLMs with knowledge. Its core strength lies in its ability to directly integrate knowledge without external retrieval, leading to improved efficiency and scalability. The method trains adapters over the knowledge part, leaving the base LLM unmodified for text input. This means if no knowledge base is provided, the model behaves exactly like the base model. KBLaM is intended for research purposes, focusing on accuracy of retrieval, refusal rate, and alignment of answers with the knowledge base. It currently supports popular models like `meta-llama/Meta-Llama-3-8B-Instruct`, `meta-llama/Llama-3.2-1B-Instruct`, and `microsoft/Phi-3-mini-4k-instruct`.\n\n## Links\n*   **GitHub Repository**: [https://github.com/microsoft/KBLaM](https://github.com/microsoft/KBLaM){:target=\"_blank\"}\n*   **arXiv Paper**: [KBLaM: Knowledge Base Augmented Language Models](https://arxiv.org/abs/2410.10450){:target=\"_blank\"}\n*   **Hugging Face - Llama-3-8B-Instruct**: [https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct){:target=\"_blank\"}\n*   **Hugging Face - Phi-3-mini-4k-instruct**: [https://huggingface.co/microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct){:target=\"_blank\"}","metrics":{"detailViews":6,"githubClicks":12},"dates":{"published":null,"modified":"2026-02-28T21:45:50.000Z"}}