Models

The Models tab is where you download AI brains that run inside askTheodor itself, with no external program and no per-use cost. These power the Built-in provider — the one that always works, even offline and before you’ve connected anything. This tab checks whether each model will run well on your machine, downloads it with a progress bar, and tracks how much disk space your models use.

How this differs from Providers

On the Providers tab you connect external services (cloud APIs, or local runners like Ollama). Here, the models live in the app. Think of Providers as “phone numbers you dial” and Models as “what’s installed in the building.”

Your machine at a glance

Two cards sit at the top so you know what you’re working with before you download anything.

System card 🖥️

• What it shows: your operating system, processor, total and free memory, free disk space, and which hardware acceleration is available (Metal on Apple silicon, CUDA on NVIDIA, AVX2, or plain CPU). Two small badges show whether the in-app engine and embedder are compiled into this build.
• When you’d look: before downloading, to understand which models the app marks as a good fit.

Storage card 💾

• What it shows: how much disk the models you’ve already installed take up, how many there are, how much free space remains on that drive, and the folder they’re stored in.
• When you’d look: to decide whether you have room for another model, or to find which ones to delete.

Installed models

A list of every model already on this machine. Each row shows its size, quantisation, and family, and whether it’s the DEFAULT (the one the Built-in provider uses unless a worker picks another).

Per row:

• Set default — make this the model the Built-in provider reaches for. Change it when you’ve installed a better-fitting model and want it used automatically.
• Delete — remove the model and free its disk space. Use it to reclaim room or drop a model that didn’t run well. (It asks first.)
• + Import GGUF (top-right of the section) — register a .gguf model file you already have on disk as a custom built-in model, instead of downloading from the catalogue.

Available models (the catalogue)

A curated list, sorted by what runs best on your machine — already-installed first, then highest fit-score, then alphabetical. Each entry shows:

• A system-fit badge — Excellent, Good, Marginal, or Unsupported — with a one-line reason (“uses ~X GB, you have Y free”). This is the single most useful number on the tab: it tells you, before you wait for a download, whether the model will actually run.
• The size, quantisation, and parameter count (in billions).
• Family, whether it supports tools (✓ / partial / ✗), whether it supports vision (images), and whether it’s code-specialised.

Per entry:

• Download — fetch the model, with a live percentage bar and integrity verification. It’s disabled for Unsupported models (they’d exceed your machine’s resources).
• Cancel — stop a download in progress.
• Delete — remove it again once installed.

Not sure which to pick?

Start with whatever sits at the top of the list with an Excellent or Good badge — that’s the app’s recommendation for your hardware. A small model like Qwen 2.5 1.5B is a friendly starting point on most machines.

Why a model might say “Unsupported”

Big models need a lot of memory. If a model is marked Unsupported, downloading it would either fail or run unbearably slowly on this machine. Pick a smaller one, or use a cloud provider for heavy work instead.

Telemetry (optional, local-only)

A collapsible section at the bottom.

• What it does: when switched on, it records the speed (tokens per second) and total volume of the Built-in provider’s work, and shows chats, prompt/completion tokens, and average tokens/second.
• Default: off — it’s opt-in.
• When to change it: turn it on briefly to compare how fast different model sizes run on your machine, then leave it off if you don’t need it. Reset clears the counters. The numbers never leave your computer.

Planned: local voice models

This same in-app download mechanism will soon offer Whisper (speech-to-text) and Piper (text-to-speech) models, so high-quality voice can run fully on-device. See Voice for the details.

In practice

Trust the system-fit score over the parameter count. A model rated Marginal on your machine will technically run and will feel unusable — slow enough that you stop reaching for it. Start with something rated Excellent, confirm the workflow is worth having, and only then try a larger one.

Check tools support before assigning a local model to a working worker. A model marked Partial can hold a decent conversation but will call tools unreliably — which shows up as a worker that seems to ignore its instructions or “forgets” to search. For any worker that needs tools, that badge matters more than raw quality.

Where local models genuinely win: privacy (nothing leaves the machine), cost (no per-token billing), offline operation, and high-volume routine work where “good enough, free, and repeatable” beats “excellent and metered.” Where they lose: long context, hard reasoning, and anything you’d publish without editing.

Disk and memory are the real constraints. Downloaded models are large, and a model that fits your disk but not comfortably in memory will thrash. Prune models you tried and abandoned — they’re among the biggest files the app will ever put on your machine.

🎓 Learn it hands-on: Pick a brain per worker

Terms in this page

• Built-in provider — the in-app model option that runs models from this tab, with no external program and no per-use cost.
• GGUF — a file format for packaged local AI models. The catalogue downloads GGUF files, and you can import your own.
• Model catalogue — the curated list of downloadable models on this tab.
• System-fit score — the app’s judgement (Excellent / Good / Marginal / Unsupported) of how well a model will run on your specific machine.
• Parameters — a rough measure of a model’s size and capability, counted in billions. More parameters usually means smarter but heavier.
• Quantisation — a compression technique that shrinks a model so it needs less memory and disk, with a small quality trade-off.
• Family — the lineage a model belongs to (e.g. Qwen, Llama). Models in a family share traits.
• Tools support — whether the model can reliably call the app’s tools (search, file access, etc.). “Partial” means it can sometimes but not dependably.
• Vision — whether the model can understand images you give it, not just text.
• Hardware acceleration (Metal / CUDA / AVX2) — ways your computer’s chips speed up AI work. Metal is Apple silicon; CUDA is NVIDIA graphics cards; AVX2 is a CPU feature; otherwise it falls back to plain CPU.
• Engine / embedder — the in-app components that run the model and turn text into searchable numbers, respectively. The badges show whether your build includes them.
• Token — the unit AI models read and write in (~¾ of a word).
• Tokens per second — how fast a model generates text on your machine. Higher is snappier.
• Telemetry — optional, local-only usage statistics that stay on your computer.
• Whisper / Piper — open speech models (planned for this tab): Whisper turns speech into text, Piper turns text into spoken audio.
• Default model — the model the Built-in provider uses automatically unless a worker is set to use a specific one.