BotPsy-0.8B: Run Tether’s Tiny Tool-Calling Model Locally (Benchmarks, GGUF, Ollama, llama.cpp)

Tether’s AI research team, the people behind the QVAC local-AI platform, has released BotPsy-0.8B: a tiny open model built for one job, calling tools. It is small enough to run on a phone or an old laptop, yet it beats models two and a half times its size on most agent benchmarks.

It came out on 11 November 2026 under the Apache 2.0 licence, with ready-made GGUF files from 0.53 GB. You can have it answering tool calls in llama.cpp, Ollama or LM Studio in a few minutes.

Below you get the full benchmark table, which file to download, copy-paste commands for every runtime, a Python tool-calling example, limits, troubleshooting and how it compares with other small models.

What BotPsy-0.8B Is

BotPsy is a specialist, not a chatbot that knows everything. It is trained to read a list of tools (functions described in JSON), decide which one to call, and fill in the right arguments.

Tether built it for a private, on-device personal assistant. Think “add the dentist on Friday at 3 pm”, “search the web for train times” or “turn on Wi-Fi”, with the facts coming from tools rather than the model’s memory.

The team’s reasoning is simple. Cloud models cost money, change without notice and see your data. A small model tuned for one task can handle many routine agent jobs on your own hardware.

Quick answer: use BotPsy as the “router” brain of a local agent that calls tools, not as a general knowledge chatbot.

Specs at a Glance

Benchmarks: Every Number Tether Published

All scores below are vendor-reported, in thinking mode, using one shared test harness. The multipliers show how much bigger each rival is.

What the numbers say

• Tool use: best on five of six tool benchmarks. Only Claw-Eval goes to the 2B model.
• Multi-turn: BFCL v4 multi-turn jumps from 7.5 to 46.1, and τ²-bench telecom hits 82.5.
• Knowledge: GPQA and MMLU-Pro stay far behind the 2B model. Size still limits what it “knows”.
• Efficiency: about 3x fewer reasoning tokens than its base on GPQA (3,690 vs 11,563), and looping on IFBench falls from 22.9% to 1.4%.
• Safety: unsafe answers in automated red-teaming fall from 36% to 21%, and from 46% to 31% under prompt injection.

How Tether Trained It

1. Supervised fine-tuning: full fine-tuning on tool-use chats, including multi-step agent runs and cases where no tool should be called.
2. Multi-teacher distillation: three Qwen3.5-4B teachers, each RL-trained for tool use, reasoning or instruction following, coached the student token by token.
3. RL round 1 (DAPO): rewards for correct call format and partial credit for matching tool names, argument names and values.
4. RL round 2: math, code, structured output and multi-step tool use in a simulated workplace.

Training data mixes public sets such as xLAM, ToolACE, Toucan-1.5M, When2Call and Nemotron collections with an unreleased synthetic set. The training code and data are not published.

Which File to Download (and Hardware)

Tether did not publish memory or speed figures. My estimate from the file sizes: Q8_0 with an 8K context needs roughly 1.5 to 2 GB of RAM, and any CPU from the last few years will run it at usable speed. A GPU is a bonus, not a requirement.

Tip: tool calling gets worse with heavy quantisation on tiny models. Start with Q8_0 and only drop to Q4_K_M if you are short on memory.

How to Run BotPsy Locally, Step by Step

Option 1: Ollama (quickest)

1. Install Ollama from ollama.com (see our Ollama guide if you are new).
2. Pull and chat: ollama run hf.co/qvac/BotPsy-0.8B-GGUFs:Q4_K_M. Swap Q4_K_M for Q8_0 for the recommended file.
3. Use the API: send tools to http://localhost:11434/v1/chat/completions with the OpenAI format.

Option 2: llama.cpp server (best tool-call parsing)

1. Install llama.cpp: on Windows run winget install llama.cpp; on macOS or Linux use the official installer from the llama.cpp project, or build from source.
2. Download the file: hf download qvac/BotPsy-0.8B-GGUFs botpsy-0.8b-q8_0.gguf --local-dir .
3. Start the server with tool calling and thinking on (one line):

llama-server -m botpsy-0.8b-q8_0.gguf --jinja -c 32768 --temp 0.3 --top-p 0.95 --top-k 20 --presence-penalty 1.5 --chat-template-kwargs '{"enable_thinking": true}' --port 8080

Open http://localhost:8080 for the built-in web UI, or call /v1/chat/completions from any OpenAI client. Tether’s own benchmarks used llama.cpp CUDA build b12169, so use that build or newer.

Option 3: LM Studio

Search for qvac/BotPsy-0.8B-GGUFs in the Discover tab and pick Q8_0. To use it with MCP tools, follow our LM Studio MCP and agent mode guide.

Option 4: vLLM (servers and batching)

vllm serve qvac/BotPsy-0.8B --language-model-only --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3

Not sure which runtime fits? Our vLLM vs llama.cpp vs Ollama comparison explains the trade-offs.

Other runtimes

• Docker Model Runner: docker model run hf.co/qvac/BotPsy-0.8B-GGUFs:Q4_K_M
• Lemonade: lemonade pull qvac/BotPsy-0.8B-GGUFs:Q4_K_M
• QVAC SDK (JavaScript): npm i @qvac/sdk plus the Bare runtime for your platform; set tools: true in the model config.
• Transformers: needs a release with Qwen3.5 support (tested on 5.19.0). Install flash-linear-attention and causal-conv1d for speed.

Tool Calling From Python

This works against llama-server (port 8080) or Ollama (change the URL to port 11434). Install the client with pip install openai.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="local")

tools = [{
    "type": "function",
    "function": {
        "name": "create_event",
        "description": "Add an event to the user's calendar",
        "parameters": {
            "type": "object",
            "properties": {
                "title": {"type": "string"},
                "day": {"type": "string", "description": "Day of the week"},
                "time": {"type": "string", "description": "Start time, 24h HH:MM"}
            },
            "required": ["title", "day", "time"]
        }
    }
}]

resp = client.chat.completions.create(
    model="botpsy",
    messages=[{"role": "user", "content": "Add the dentist on Friday at 3 pm."}],
    tools=tools,
    temperature=0.3,
    max_tokens=4096,
)

msg = resp.choices[0].message
for call in msg.tool_calls or []:
    print(call.function.name, call.function.arguments)

A correct run prints create_event with arguments like title “Dentist”, day “Friday” and time “15:00”. Your code then runs the real function and sends the result back as a tool message.

For the same pattern with cloud models, see our Gemini function calling tutorial and structured outputs guide.

BotPsy vs Other Small Models

*Average of six tool-use benchmarks, as reported by Tether on the BotPsy model card.

If you need a model that answers questions from its own knowledge, a 4B to 8B model is still the better choice. If you need a fast, cheap brain that picks the right tool, BotPsy is the strongest sub-1B option I have seen numbers for.

Limits, Licence and Safety

• Difficulty: Tether says it is best for low-to-medium tasks. Complex multi-step web research scores only 15.
• Knowledge: it is designed to call tools for facts. Do not trust its memory for dates, prices or names.
• Banking-style tasks: on τ³-banking, no tested model scored above 5%, BotPsy included.
• Languages: English first. Multi-IF scores are 24.5 in Hindi and 57.6 in Russian.
• Long instructions: accuracy drops as instructions pile up in one prompt.
• Prompt injection: still 31% unsafe under injection tests, so never let it act on untrusted web text without checks.

Warning: Tether says BotPsy is not meant for safety-critical decisions or anything involving money, personal information or health. Put a human confirmation step before any action that matters.

Licence: the weights are Apache 2.0, which allows commercial use, modification and redistribution with attribution. The model card describes the intended use as research, education and tool-using assistants, and lists third-party data credits in ATTRIBUTIONS.md.

Troubleshooting

Tool calls come back as plain text with <tool_call> tags

Your server is not parsing them. Start llama-server with --jinja, or in vLLM add --enable-auto-tool-choice --tool-call-parser qwen3_xml.

“unknown model architecture” or the GGUF will not load

Your llama.cpp, Ollama or LM Studio build is too old for the Qwen3.5 hybrid architecture. Update to the latest release; Tether tested with llama.cpp b12169.

The answer is cut off or empty

With thinking on, the model writes a reasoning block first. Set max_tokens (or predict) to at least 2,048.

vLLM errors about vision or processor config

Qwen3.5 ships with multimodal parts. Add --language-model-only so vLLM loads the text model only.

It picks the wrong tool or invents arguments

• Use Q8_0 instead of Q4_K_M.
• Keep the recommended sampling (temperature 0.3, presence penalty 1.5).
• Write clear tool descriptions and mark required fields.
• Offer fewer tools per request; tiny models struggle with long tool lists.

Transformers says the model type is not recognised

Upgrade Transformers to a version with Qwen3.5 support; the card was tested on 5.19.0.

My Take

BotPsy is a good example of where small models are heading: not “a little ChatGPT”, but a specialist that does one job very well. Beating a 2B model on tool use while being 0.53 GB on disk is impressive, even allowing for the fact that these are Tether’s own numbers.

The smart way to use it is as the first layer of a local agent. Let BotPsy decide which tool to call and fill in arguments, then hand anything hard to a bigger local model or a cloud API.

I would wait for independent BFCL results before betting a product on it. For hobby agents, home automation and phone assistants, it is well worth a download today.

FAQ

Who made BotPsy-0.8B?

Tether AI Research, the team behind Tether’s QVAC local-AI platform. It is fine-tuned from Alibaba’s Qwen3.5-0.8B.

Is BotPsy free for commercial use?

The weights use the Apache 2.0 licence, which permits commercial use with attribution. Check ATTRIBUTIONS.md in the repo for third-party data credits.

Can BotPsy run on a phone?

Yes, that is its purpose. The Q4_K_M file is 0.53 GB, and Tether built it for on-device use; llama.cpp and the QVAC SDK both support ARM chips.

Does BotPsy need a GPU?

No. At 0.8B parameters it runs on a normal CPU. A GPU only makes it faster.

Does it work with Ollama tool calling?

Yes, you can run it with ollama run hf.co/qvac/BotPsy-0.8B-GGUFs:Q4_K_M and pass tools through the API. If calls show up as raw text, use llama-server with --jinja.

How do I turn thinking mode on or off?

It is off by default. Turn it on with enable_thinking=True in the chat template, or --chat-template-kwargs '{"enable_thinking": true}' in llama-server.

Is BotPsy good for general chat or questions?

Not really. Its knowledge scores (GPQA 38.1, MMLU-Pro 48.9) are well below 2B models, so use it with tools like web search.

What does the Tether connection mean for privacy?

Nothing leaves your machine when you run the weights locally. The model has no built-in network calls; only the tools you connect can reach the internet.

Sources

• Hugging Face: qvac/BotPsy-0.8B model card (benchmarks, training, limits)
• Hugging Face: qvac/BotPsy-0.8B-GGUFs (files, sizes, run commands)
• Hugging Face blog: “BotPsy” by QVAC / Tether AI Research (11 November 2026)
• QVAC SDK documentation (qvac.tether.io)
• Qwen: Qwen3.5-0.8B model card (base architecture)
• AI Weekly: daily AI news listing for BotPsy-0.8B (November 2026)