rag-zero-to-hero-guide: Your Comprehensive Path to Mastering RAG

Introduction

The rag-zero-to-hero-guide repository by KalyanKS-NLP is an invaluable resource for anyone looking to master Retrieval-Augmented Generation (RAG). This comprehensive guide takes you from the foundational concepts of RAG to advanced implementation techniques, making it suitable for both beginners and experienced AI engineers. It meticulously covers the "What, Why, and How" of RAG, offering practical examples and a curated toolkit of essential libraries and frameworks.

Installation

This repository primarily serves as a learning guide and a collection of resources, rather than a traditional software package requiring installation. To access the content, including detailed markdown explanations and Jupyter Notebooks, simply clone the repository to your local machine:

git clone https://github.com/KalyanKS-NLP/rag-zero-to-hero-guide.git
cd rag-zero-to-hero-guide

Individual notebooks within the guide may require specific Python libraries. It is recommended to set up a virtual environment and install dependencies as needed for each example. For instance, notebooks using LangChain or RAGAS will require those libraries to be installed in your environment.

Examples

The rag-zero-to-hero-guide offers a wealth of practical examples and structured courses:

• RAG Basics Course: Dive into the fundamentals with notebooks like "RAG from Scratch," demonstrating RAG implementation without frameworks, and "RAG with LangChain," showcasing integration with a popular LLM framework. It also includes examples for "Website RAG," "YouTube Video RAG," and "Agentic RAG with CrewAI."
• RAG Evaluation Course: Learn how to assess your RAG systems effectively. This section provides insights into evaluation metrics for both retrievers and generators, with practical implementations using libraries such as RAGAS and DeepEval. It also covers techniques for detecting hallucination in RAG outputs.
• RAG Toolkit: Explore an extensive collection of tools categorized by function, including:
  • Frameworks: LangChain, LlamaIndex, Haystack, fastRAG, Llmware.
  • Data Extraction: Libraries for web scraping (Crawl4AI, ScrapeGraphAI) and document parsing (Docling, Llama Parse, PyMuPDF4LLM).
  • Vector Databases: SQLite-Vec, FAISS, PGVector, Chroma, Qdrant, Pinecone, Weaviate, Milvus.
  • Advanced RAG: Sections dedicated to Agentic RAG (CrewAI, AutoGen, LangGraph) and Graph RAG (GraphRAG, Nano GraphRAG).
• RAG Survey Papers: A curated list of academic papers providing in-depth research and insights into the evolving landscape of RAG.

Why Use

This repository is an essential resource for anyone involved in Generative AI and LLM development due to several key advantages:

• Structured Learning: It provides a clear, step-by-step roadmap to understand RAG, from basic principles to complex implementations and evaluation.
• Practical Application: With numerous Jupyter Notebook examples, you can immediately apply theoretical knowledge to real-world scenarios.
• Comprehensive Toolkit: The curated list of frameworks, libraries, and tools saves significant time in discovering and selecting the right components for your RAG projects.
• Up-to-Date Research: Access to a collection of RAG survey papers ensures you stay informed about the latest advancements and research trends.
• Community and Expertise: Maintained by Kalyan KS, a recognized expert in NLP, the guide reflects deep industry knowledge and best practices.

Links

• GitHub Repository: https://github.com/KalyanKS-NLP/rag-zero-to-hero-guide
• Kalyan KS on LinkedIn: https://www.linkedin.com/in/kalyanksnlp/
• Kalyan KS on X (Twitter): https://x.com/kalyan_kpl
• Kalyan KS on YouTube: https://www.youtube.com/@kalyanksnlp