Open-source alternatives to Ollama

Local LLM inference server for Apple Silicon. — LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

↔ alternative to Ollama

Local LLM inference engine that auto-tunes kernels for speed. — Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.

↔ alternative to Ollama

Pure Rust local LLM inference engine with OpenAI API — ⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

↔ alternative to Ollama

Local AI server for running LLMs on your hardware. — Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

↔ alternative to Ollama

Local offline LLM chat app for Mac and Windows — This codebase is for a React and Electron-based app that executes the FreedomGPT LLM locally (offline and private) on Mac and Windows using a chat-based interface

↔ alternative to Ollama

Run local LLMs on Apple Silicon with MTP — The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

↔ alternative to Ollama

Native local LLM inference server for Apple Silicon. — Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

↔ alternative to Ollama

Run open-weight LLMs locally and privately. — Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

↔ alternative to Ollama

Retro UI for running local LLMs. — 📎 Clippy, now with some AI

↔ alternative to Ollama