{"name":"OLMoE.swift: Private, Offline AI Experience on iOS and macOS","description":"OLMoE.swift provides a unique, privacy-focused AI experience by running large language models directly on your device. This Swift-based application ensures your queries and data remain private, operating entirely offline without an internet connection. It offers a robust solution for local AI inference on iOS and macOS, with integration options for Hugging Face.","github":"https://github.com/allenai/OLMoE.swift","url":"https://osrepos.com/repo/allenai-olmoe.swift","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/allenai-olmoe.swift","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/allenai-olmoe.swift.md","json":"https://osrepos.com/repo/allenai-olmoe.swift.json","topics":["Swift","AI","LLM","On-Device AI","Privacy","Offline","iOS","macOS"],"keywords":["Swift","AI","LLM","On-Device AI","Privacy","Offline","iOS","macOS"],"stars":null,"summary":"OLMoE.swift provides a unique, privacy-focused AI experience by running large language models directly on your device. This Swift-based application ensures your queries and data remain private, operating entirely offline without an internet connection. It offers a robust solution for local AI inference on iOS and macOS, with integration options for Hugging Face.","content":"## Introduction\nOLMoE.swift is an open-source project from AllenAI that brings a fully private, offline, and on-device AI experience to your Apple devices. Built with Swift, this repository provides the necessary tools and code to run large language models (LLMs) locally, ensuring your data never leaves your device. It's designed for users who prioritize privacy and wish to interact with AI models without an internet connection.\n\n## Installation\nTo get started with OLMoE.swift, you first need to clone the repository:\n\nsh\ngit clone https://github.com/allenai/OLMoE.swift.git\n\n\n**Building the iOS app for a simulator:**\n1. Open the project in Xcode.\n2. Ensure the target device is set to an appropriate device (e.g., iPhone 15 Pro or higher).\n3. Run the project.\n\nFor detailed instructions on running on macOS or a physical device, please refer to the official [OLMoE.swift README](https://github.com/allenai/OLMoE.swift#readme).\n\n## Examples\nOLMoE.swift also supports integration with Hugging Face for running models. You can install `transformers` and `torch` and then use the following Python code:\n\npython\nfrom transformers import OlmoeForCausalLM, AutoTokenizer\nimport torch\n\nDEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Load different ckpts via passing e.g. `revision=step10000-tokens41B`\nmodel = OlmoeForCausalLM.from_pretrained(\"allenai/OLMoE-1B-7B-0125\").to(DEVICE)\ntokenizer = AutoTokenizer.from_pretrained(\"allenai/OLMoE-1B-7B-0125\")\ninputs = tokenizer(\"Bitcoin is\", return_tensors=\"pt\")\ninputs = {k: v.to(DEVICE) for k, v in inputs.items()}\nout = model.generate(**inputs, max_length=64)\nprint(tokenizer.decode(out[0]))\n\nThis example demonstrates how to load an OLMoE model and tokenizer, then generate text using a simple prompt.\n\n## Why Use OLMoE.swift?\nOLMoE.swift stands out for several compelling reasons:\n*   **Uncompromised Privacy:** All queries and data are processed locally on your device, ensuring complete privacy. Nothing is stored or sent to external servers.\n*   **Offline Functionality:** The application works perfectly without an internet connection, even in Flight Mode, making it ideal for use anywhere, anytime.\n*   **Open Source:** Being open source, the project fosters transparency, allowing developers to inspect, modify, and contribute to its development.\n*   **On-Device AI:** Experience the power of large language models directly on your iOS or macOS device, leveraging local hardware for efficient inference.\n\n## Links\nExplore OLMoE.swift further using these official links:\n*   [GitHub Repository](https://github.com/allenai/OLMoE.swift)\n*   [AllenAI Playground](https://playground.allenai.org)\n*   [Discord Community](https://discord.gg/sZq3jTNVNG)\n*   [License Information](https://github.com/allenai/OLMo/blob/main/LICENSE)","metrics":{"detailViews":10,"githubClicks":25},"dates":{"published":null,"modified":"2025-10-20T07:00:51.000Z"}}