mi·do·naミ・ド・ナ

Home / Library / Help / Set up a runner / koboldcpp

Guide

Set up koboldcpp

koboldcpp runs a language model on your own machine and serves it over a local API that mi·do·na can talk to. Nothing leaves your computer. It ships as a single executable with a simple launcher and no install step - a long-time favorite in the roleplay community. This is the short version; koboldcpp's own wiki covers the rest.

Get a model running

  1. Download koboldcpp

    Grab the build for your OS from the koboldcpp releases. Windows gets koboldcpp.exe; macOS and Linux have their own binaries, or you can run from source. There's nothing to install - it's a single file you run directly.

  2. Get a GGUF model

    Download a model in .gguf format from Hugging Face. Start in the 7-8B range for a model that fits most machines; larger is smarter but heavier.

  3. Launch and load

    Run the executable to open the launcher, choose your .gguf file and a preset, and hit Launch. Prefer the terminal? Point it straight at the model:

    By default it listens on port 5001.

    Terminal
    ./koboldcpp model.gguf

  4. Endpoints & CORS

    koboldcpp serves the KoboldAI Lite UI at http://localhost:5001 and an OpenAI-compatible API at http://localhost:5001/v1. It sends permissive CORS headers by default, so mi·do·na can reach it from the browser with no extra setup.

Next: connect mi·do·na

With a model loaded, koboldcpp is ready. Open the connection panel in mi·do·na, point it at http://localhost:5001/v1, and load models. See Connect a model.

Served over HTTPS and can't reach localhost? The browser can block an HTTPS page from talking to http://localhost - most often in Safari. See

Reaching a local model over HTTPS

.

This is a quick-start. For anything deeper, see the official koboldcpp wiki at github.com/LostRuins/koboldcpp/wiki.