Local models and privacy routing

Run Miosbot on a model on your own computer, and keep customer and secret data off cloud models.

Status: rolling out. Local models and privacy routing are built and switch on in an update after launch. Until then, Miosbot uses the cloud models you choose.

Use a model on your own computer

Point Miosbot at any OpenAI-compatible endpoint, such as Ollama, LM Studio, vLLM or llama.cpp. Miosbot checks the endpoint in the background and lists its models in the model picker. A slow or missing endpoint never stops Miosbot from starting.

Privacy routing

With privacy routing on, Miosbot tags each request as public, customer or secret:

Customer and secret requests go to a model on your computer.
If no local model is available, Miosbot refuses with a plain reason. It never quietly falls back to a cloud model.
Results from tools that contain customer data are kept away from cloud models too.
When Miosbot isn’t sure, it treats the request as private.

Mix cloud and local

You can set a chain of models per task, mixing cloud and local. Miosbot moves to the next model in the chain only when a provider is down.