Repository History
3 repositories tagged with GenAI
OmniParse: Ingest, Parse, and Optimize Data for GenAI Frameworks
OmniParse is a powerful platform designed to ingest, parse, and optimize any unstructured data, from documents to multimedia, into structured, actionable formats. It enhances compatibility with GenAI frameworks, preparing data for applications like RAG and fine-tuning. This tool simplifies the complex process of data preparation for AI, making it accessible and efficient.
GenAIScript: Automatable GenAI Scripting with TypeScript/JavaScript
GenAIScript is an open-source project from Microsoft that enables programmatic assembly of prompts for Large Language Models (LLMs) using JavaScript and TypeScript. It allows developers to orchestrate LLMs, tools, and data directly in code, streamlining the development of GenAI applications. This framework offers seamless Visual Studio Code integration and a flexible command-line interface for efficient GenAI scripting.
Data Prep Kit: Accelerating Data Preparation for GenAI and LLM Applications
Data Prep Kit is an open-source project designed to accelerate unstructured data preparation for GenAI and LLM applications. It provides a comprehensive set of modules and transforms to cleanse, transform, and enrich data for pre-training, fine-tuning, instruct-tuning LLMs, or building Retrieval Augmented Generation (RAG) applications. The kit is highly scalable, supporting processing from a laptop to data center scale using Python, Ray, and Spark runtimes.