What's New

This page summarizes the most recent additions to LlamaForge. Each entry links to the full reference for that capability. For the longer-term direction, see the project's ROADMAP.md.

v0.19.0: See inside a model, talk to it, and leave it running

Everything from the v0.18 line in one release, plus start at login on Windows. The short version:

• Model X-ray. Every layer of a GGUF as a bar: what sits on the GPU and what spills to RAM at your -ngl, and for a MoE an Experts on CPU slider. Fit my VRAM finds the placement, Load it with shows the flags (-ngl 41 --n-cpu-moe 22), Apply to this model saves them. It works on a model before you download it too (it reads the header over HTTP), and compares quants side by side. See Models.
• Can I run it? pages for 42 GPUs, laptops and Radeon / Arc included, each with the llama.cpp line to start with. Can I run it?
• Talk to it. Voice chat with any loaded model, hands-free. Chat from your phone on the same Wi-Fi, and talk from it too over the Tailscale HTTPS link the Phone dialog offers (phones only allow the mic on HTTPS).
• Compare honestly. Blind Arena (answers side by side, vote, then see the names), checked tasks with right answers, and Try a llama.cpp PR to see whether a pull request or an update is faster on your card before you switch.
• OpenAI's Batch API on your own GPU.
• Leave it running. Several models at once that keep VRAM free on their own, idle models that sleep and wake, a router that comes back when it dies, a GPU box with no screen, and start at login: now on Windows too (Setup > Startup), next to Linux and macOS.
• Bring what you have. Your Ollama models without a re-download, AMD and Intel GPUs detected, embedding and reranker models set up for RAG, paste-ready settings for Open WebUI / Continue and others.
• Housekeeping. Free disk space (unused models, byte-identical duplicates), re-upload watch for quants that changed on Hugging Face, template doctor, recommended samplers per model.

Each piece has its own entry below.

v0.18.42: Can I run it? on 42 cards, laptops included

Can I run it? now covers 42 graphics cards, up from 19: laptop GeForce cards from the RTX 3060 Laptop (6 GB) to the RTX 5090 Laptop (24 GB), the RTX 2080 Ti, 3050, 3080 Ti, 4060 Ti 8GB, 4070 Ti, 4080 Super and 5060, more Radeons, and Intel Arc A770 and B580. The index groups them into Desktop GeForce, Laptop GeForce, and Radeon and Intel Arc. Every page has the same verdict and llama.cpp line, worked out from the real GGUF headers.

v0.18.41: The llama.cpp line on every "Can I run it?" page

Each Can I run it? page now ends its verdict with the command that runs the pick, such as llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M -ngl 41 --n-cpu-moe 22 -c 8192 --no-mmproj on a 12 GB card, and the table gives the flags for every quant. Same memory math as the verdict, worked out from the real GGUF headers by the page generator.

v0.18.40: Keep the X-ray's flags

The X-ray of a model you have now does what the one in Discover does, and saves the result: a line says whether the weights fit your VRAM, a MoE gets the Experts on CPU slider, and Load it with shows the flags. Fit my VRAM moves the sliders to where the most weight fits, and Apply to this model writes them into its settings after asking, comments and other settings untouched. See Models.

v0.18.39: The exact flags for a MoE that doesn't fit

X-ray before download now places a MoE the way you'd run it: every layer on the GPU and the experts of the first few blocks in system RAM (--n-cpu-moe), so the CPU only runs a few active experts, never attention. An Experts on CPU slider moves that line, and Load it with shows the flags for the view, like -ngl 41 --n-cpu-moe 7, with a Copy button. The share card prints them too. See Discover.

v0.18.38: X-ray a model before you download it

Discover's file rows have an X-ray button. It reads only the file's GGUF header from Hugging Face, a few MB, and shows every layer and where it would land on your GPUs, starting at the most GPU layers whose weights fit your VRAM. A line says whether it fits, or by how much it's over. Compare with another quant and Share image work here too. See Discover.

v0.18.37: Share a quant compare as one image

With Compare with on, the X-ray's Share image now draws both quants on one 1200x630 card: the two layer stacks side by side on one scale, what the other quant costs (weights, GPU, system RAM) as the headline, and no paths. Handy for the "which quant fits my card?" threads. See Models.

v0.18.36: Which quant fits? Compare them in the X-ray

The X-ray now has Compare with: pick another quant of the same Hugging Face repo and see both layer stacks side by side, on one scale and under your model's own settings. LlamaForge reads only that file's GGUF header over HTTP Range, a few MB, so you see where a Q8_0 or an IQ4_XS would sit (GPU or system RAM, block by block) before downloading 20 GB. The GPU layers slider moves both. See Models.

v0.18.35: Share your model's X-ray

The X-ray has a Share image button. It turns what's on screen, slider included, into a 1200x630 card: one column per layer, coloured by what the layer holds, with the part in system RAM hatched, plus the quant mix and the GPU / RAM split. Post "my 26B at 10 GPU layers" next to "all layers" and the difference is plain to see. The card holds no paths. See Models & Tuning.

v0.18.34: Model X-ray

Every model's info card has an X-ray button. It takes the GGUF apart from its header, one bar per layer, split into attention, FFN, MoE experts and the rest, and hatches what stays in system RAM under your settings. On Gemma 4 26B it showed why 5 GB of a 15.4 GB model sat in RAM: n-cpu-moe=10 keeps the first ten blocks' experts there (about 440 MB each), plus the 748 MB embedding. Drag the GPU layers slider and watch the blocks move; nothing is saved. See Models & Tuning.

v0.18.33: Talk from your phone

The phone remote got a Talk button. Ask out loud from the sofa, and your PC hears you, answers, and the phone plays the answer in Qwen3-TTS's voice. It's the Voice tab's Talk, on the phone, and it stops recording by itself when you pause. Phones only allow the mic on HTTPS pages, so the Phone dialog now offers a Tailscale HTTPS link when this PC is on a tailnet: one tailscale serve --bg 8092 and there's no certificate to make. See Talk from your phone.

v0.18.32: Talk, hands-free

The Voice tab holds a conversation now. Press Talk, ask, and a model on your GPU hears you, answers in a sentence or three, and Qwen3-TTS reads the answer out. Tick Hands-free and the mic reopens after each answer and stops when you pause, so you just keep talking. One model both hears and answers, so nothing swaps: on an RTX 5080, Gemma 4 E4B heard each question in about 0.2 s and answered in under half a second. The one-click model is now Gemma 4 E4B: Voxtral Mini turned out to answer spoken questions instead of writing them down. See Talk.

v0.18.31: Talk to it

The Voice tab now listens. Speech to text has a Record button: talk, press Stop, and the words come back from a model on your own GPU (Voxtral Mini turned 6.7 seconds of speech into text in under half a second on an RTX 5080). Speak it back hands them to Qwen3-TTS, so the whole voice loop runs on your machine. Nothing that can hear yet? One click gets Voxtral Mini 3B. Scripts get OpenAI's /v1/audio/transcriptions: the openai SDK works unchanged. See Voice.

v0.18.30: OpenAI's Batch API, on your GPU

Got a thousand documents to summarise? Stats → Batches takes a .jsonl of chat requests in OpenAI's batch format and works through them into a results file: now, or tonight once nobody has used the router for 20 minutes. Scripts written for OpenAI's /v1/files and /v1/batches work unchanged: point the openai SDK at the panel. A batch survives a restart, pauses while a speed comparison or Auto-Forge needs the GPUs, and a bad request fails one line, not the file. See Batches.

v0.18.29: Asked right after the update

When an update finishes, the Engine card now asks: you're on b7123 now, is it as fast as b7080 was on your models? The build you were on is already picked in the compare speed row, so it's one model and one click. The old build stays installed, so Use takes you back if the answer is no. See Build & Update.

v0.18.28: Is the update faster?

Compare speed now works on official builds too, not only PR builds. Got a new nightly but still running the old one? The Engine card's compare speed row runs llama-bench on your model with the build you're on and with the one you picked, and shows the change before you press Use. A regression on your GPU shows up as a minus sign in a table, not as a slow evening. See Build & Update.

v0.18.27: Is the PR faster?

A PR build now has a Compare speed button. Pick a model and LlamaForge runs llama-bench on it with your build and then with the PR, same model file and GPU settings, and shows both speeds side by side with the change in percent. A change inside the runs' own spread says within noise, so you don't mistake jitter for a speedup. Copy as markdown puts the table on your clipboard for the PR's thread. See Build & Update.

v0.18.26: Try a llama.cpp PR

Paste a llama.cpp pull request number into Build / Update and LlamaForge builds that PR beside your own build, with your GPU's flags. Then pick one model to run on it: that model loads on the PR in its own process, while the router and everything else stay on your build. The commit is fixed when you look the PR up, and Remove cleans it all away. See Build & Update.

v0.18.25: Checked tasks overnight

Checked tasks can now run themselves. Pick a time under Every night at and press Schedule: once nobody has used a model for 20 minutes, LlamaForge runs your ticked models through the 15 tasks before 06:00, and the table is up to date when you sit down. See Blind Arena.

v0.18.24: Share your checked tasks

Share image in Checked tasks turns the table into a 1200x630 card: one square per task, green for right, plus each model's score and speed. If your fastest model isn't the one that gets the most right, the card says so: "Fastest: ministral-3-3b, 228 tok/s, 8 of 15 right." See Blind Arena.

v0.18.23: Checked tasks

The Arena tab now has a second way to compare models. Checked tasks asks your models 15 short tasks with one right answer each (extraction, math, JSON, following instructions, reasoning), marks them with a script, and puts them in a table with each model's speed. Every model gets one line against the one in your main slot, like "gemma-4-12b beats qwen3-8b on 3 of 5 categories, 40% slower", and you can see exactly what each one got wrong. It runs in the background, one model at a time; start it before bed. See Blind Arena.

v0.18.22: Faster arena rounds

After you vote, Same prompt, new pair asks the same prompt of two other models, and Rematch these two asks it again of the same two, shuffled again. Your last ten prompts sit in a Recent prompts list above the prompt box, kept in this browser only. Building a board that means something takes a few dozen rounds; now each one is a click. See Blind Arena.

v0.18.21: Arena board as an image

Share image under the Arena board draws it as a 1200x630 card: your top six models with Elo and wins-losses-ties, the vote count and your GPU names. Download it or copy it to the clipboard, ready for a post. It holds no prompts, paths or hosts. See Blind Arena.

v0.18.20: Blind Arena

A new Arena tab: two of your models answer the same prompt, shown only as Left and Right in random order. You vote, then the names and speeds appear, and an Elo board on your PC ranks your models on your own prompts and GPUs. See Blind Arena.

• The arena loads models as needed and unloads only the ones it loaded, never one you had running.
• Answers are shown as plain text, with any reasoning folded away so the vote is about the answer.
• Copy as markdown turns the board into a table to post.

v0.18.19: Chat from your phone

Click Phone in the top bar, turn it on and scan the QR code: your phone gets a chat page for the models on your PC, streaming from your own GPUs with live tokens per second and GPU memory. Nothing to install on the phone, no account, no cloud relay. On a Tailscale network it works from anywhere. See Phone Remote.

• It's off until you turn it on. The panel stays loopback-only; the phone gets a separate listener on port 8092 that can list models and chat, and nothing else.
• The link's token rides in the QR code's #fragment, which never crosses the network. New link unpairs every phone at once.
• The QR code is drawn by LlamaForge itself, so the panel still ships no third-party code.

v0.18.18: Live demo in your browser

Try LlamaForge without installing it. The demo is the real panel, served from the docs site, with every screen answered from a recording of a real RTX 5080 + 5060 Ti box: its models, GPU cards, Auto-Forge leaderboard (Qwen3.8-27B at 118 tokens/s on code), stats, Discover, Embers and Help. Click anything; changes are answered with a note that nothing runs there.

• The recording is made by tools/demo_record.py from a running panel. It keeps a fixed set of models, replaces paths, host names, LAN addresses and keys with neutral ones, and refuses to write if anything personal is left.
• The models were idle when it was recorded, so no model shows as loaded.

v0.18.17: LlamaForge agent skill

Your coding agent, on your own GPU. npx skills add dadwritestech/LlamaForge (or the Claude Code plugin: /plugin marketplace add dadwritestech/LlamaForge) installs a skill that teaches Claude Code, Codex and other agents to drive LlamaForge: see what fits your GPUs, download and load a model, speed-tune it, and hand it grunt work. It asks you before anything slow or disruptive. See Agent Skill.

The MCP server gained Auto-Forge: forge_start, forge_status, forge_apply and forge_undo. An agent can start a run, follow it and show you the result; on the test machine forge_status reported a Qwen3.8-27B setup at 118 tokens/s on code. It applies the winner only when you say so.

v0.18.16: Re-upload watch

GGUF makers often replace a file in a release's first weeks to fix a template or a broken quant, and until now nothing told you. Once a day LlamaForge asks Hugging Face whether each model's file still matches yours. A changed one gets a re-uploaded badge, the date and commit message, and a Re-download button. See Models.

• One small request per model, no downloads: 11 models checked in 0.7 s here.
• On this machine it caught a Gemma 4 31B whose repo had replaced the file with "Added Gemma official chat template update".
• The old file is replaced only when the new one has fully downloaded.

v0.18.15: Recommended samplers

A model's editor has a Samplers row. Recommended reads the temperature, top-p, top-k and other sampler values the model's makers publish in generation_config.json, from the GGUF repo or the base model on its card, and shows them next to yours. Use these writes only the values that differ. See Models.

• The source is linked, so you can check where a number came from.
• Here, Qwen3.8-27B's base repo recommends temp 1, top-k 20, top-p 0.95; the GGUF repo has no generation_config.json, which is why the base model is the fallback.
• Prose recommendations in a model card aren't parsed: too fuzzy to apply with one click.

v0.18.14: Template doctor

A model's editor has a Template row. Check compares the chat template inside your GGUF with the repo you downloaded it from and the base model's, and says when yours is stale. A stale template is the usual cause of broken tool calls and thinking that won't switch off. One click uses the newer template through llama.cpp's --chat-template-file; the GGUF isn't touched. See Models.

• It reads only the repo file's header, never the weights. A check took about 1.5 s per model here.
• On this machine it found a Gemma 4 31B whose repo had since been re-uploaded with a template that handles null tool arguments.
• A difference from the base model is only a hint: GGUF makers often change templates on purpose.

v0.18.13: Free disk space

Setup has a Free Disk Space card. One click lists the models nobody has used in 90 days (from Stats), largest first, and any GGUF that's a byte-identical copy of another, like the same quant in two apps' folders. Each row has a Delete button that asks first, then unloads the model, deletes the file (every part of a split one) and removes it from your config. See Setup.

• It won't delete a file another model points at, a link into another app's cache, or anything but a .gguf. Shared mmproj files stay.
• If Stats is younger than 90 days, the card says so instead of calling everything unused.

v0.18.12: Sleep idle models

Setup has a Sleep idle models card. Pick a number of minutes and any llama.cpp model that sits idle that long frees its VRAM and RAM, stays in the list as sleeping (a hollow light), and wakes on the next request. On a test here a 3B model went from about 3.2 GB of VRAM to under 0.5 GB asleep, and woke for a chat request in 0.65 s. See Setup.

• It's llama.cpp's own --sleep-idle-seconds, set for every model; it applies from each model's next load and needs a build that has the flag (the card says when yours doesn't).
• A sleeping model still gets Chat, Unload and the Claude Code button, and the Stats poller leaves it asleep.

v0.18.11: Share your rig

Stats has a Share my rig button: a 1200x630 image (and markdown) of your GPUs, RAM and OS, your five most-used models with their speed and token counts, and how many tokens this machine has generated. Download it, copy it, post it. See Stats.

• Model names lose any folder part, and nothing on the card can carry a path, key or host.
• The speed is now llama.cpp's own timing. The old Tok/s column divided tokens by 4-second poll windows, so parallel requests and late polls pushed it far too high. The new figure is the router's per-slot generation speed averaged over every token, and reads "-" until a model has generated under this version. That's also what the Stats tab shows now.

v0.18.10: The router comes back on its own

If llama.cpp's router crashes, every app pointed at it gets "connection refused" until someone notices. LlamaForge now probes it every 5 seconds and, after 15 seconds without an answer, starts it again with the same settings. Models aren't preloaded; the next request loads its model as usual.

• It only watches a router it has seen running, and it stays out of the way of restarts you or another setting are doing.
• When the router keeps dying right after it starts, it backs off (4, 8, 16, 32 s) and stops after 5 quick deaths. The Models tab then shows why, read from the router log (out of memory, a bad flag, a missing file), instead of crash-looping in the background.
• Models shows a short line when it restarted the router, with the count and the time.
• Off switch: router_watchdog: false in config.json (see Configuration).

v0.18.9: Link straight to a quant

• #discover?repo=owner/name&file=x.gguf. Opening the panel with that on the end of its URL (http://127.0.0.1:8090/#discover?repo=...&file=...) goes to Discover, opens that Hugging Face repo with its fit ratings, and highlights the quant with its Download button focused. Leave out file to just open the repo; a pasted huggingface.co/owner/name URL works as the repo too.
• It only shows, it never downloads. The download still takes your click, so a web page that links to the panel can't start a multi-gigabyte download. Anything that isn't a plain owner/name repo or a .gguf path is ignored.

v0.18.8: Embedding and reranker models, set up for RAG

• Recognized by what they are, not their name. A model scan reads the GGUF header: a rank pooling type, or an encoder with a classifier head (how llama.cpp itself tells them apart), means a reranker; a mean, cls or last pooling type, or an encoder architecture such as BERT or nomic-bert, means an embedding model. Before, only a filename containing "embed" counted, and rerankers weren't detected at all.
• Started the right way. Rerankers get reranking = true in models.ini, so the router starts them with --reranking and they answer /v1/rerank; embedding models keep embeddings = true. A rescan never clears a flag you set by hand. Models found in Ollama's store get the same check.
• Tagged and explained. The model list shows an embed or rerank tag. Client Config on one of them gives the /v1/embeddings or /v1/rerank request instead of a chat one, plus Open WebUI's Documents settings and, for embeddings, Continue's embed role. Checked end to end on llama.cpp b12032 with nomic-embed-text v1.5 and jina-reranker-v1-tiny-en.

v0.18.7: AMD and Intel GPUs show up

• VRAM for cards nvidia-smi can't see. When nvidia-smi has nothing to report, LlamaForge asks your llama.cpp build for its device list (llama-server --list-devices). A Vulkan, ROCm or SYCL build lists each GPU with its total and free memory, so AMD and Intel cards now get the GPU tiles, the VRAM preview and fit ratings. If a card appears under two backends it's counted once. On Linux, an AMD card shows up from sysfs even before a GPU build is installed.
• Integrated GPUs don't inflate the total. An iGPU reports shared system RAM as its memory (a Ryzen iGPU here claimed 16 GB), so it's left out when a real card is present. If it's the only GPU, as on a laptop, it's kept.
• What's missing. These cards show memory but not utilization or temperature; the tile says which backend the numbers came from. Elastic VRAM and Auto-Forge stay NVIDIA-only. This was checked against real Vulkan and CUDA builds on an NVIDIA machine, not on real AMD or Intel hardware yet, so reports are welcome.

v0.18.6: Start at login, if you want it

• llamaforge service on. On Linux it writes a systemd user unit, on macOS a launch agent. Either one starts the router and dashboard when you log in, without opening a browser, for an always-on box behind Open WebUI or Continue. llamaforge service off removes it, and llamaforge uninstall turns it off too. It's off unless you turn it on, and the dashboard still listens on 127.0.0.1 only. See Start at login.

v0.18.5: A GPU box with no screen

• run.sh over SSH prints the tunnel. When you start LlamaForge in an SSH session (or on Linux with no display, or with LLAMAFORGE_HEADLESS=1), run.sh no longer tries to open a browser. It prints the ssh -N -L 8090:127.0.0.1:8090 ... line to run on your own computer, using the address and port of your current SSH session. The dashboard still listens on 127.0.0.1 only. See Headless machines.

v0.18.4: Paste-ready settings for your apps

• Client Config fills in your apps. Next to curl and the OpenAI environment variables, a model's Client Config now has an Apps section with that model's settings for Open WebUI (where to click, and the host.docker.internal address when Open WebUI runs in Docker and the router is on 127.0.0.1), a Continue config.yaml, Cline's OpenAI Compatible fields and an aider command. Each has a Copy button.

v0.18.3: Thinking effort, per model

• Set how hard a model thinks. Models whose chat template takes a reasoning effort (Qwen3.8, gpt-oss and others) get an effort dropdown in the Thinking row, next to the budget. It lists only the levels that template knows, read from the GGUF (Qwen3.8 offers low, medium and xhigh), and writes llama.cpp's --reasoning-effort for that model, so every app that talks to the router gets it without sending anything. Needs a llama.cpp build from v0.8.0 on. See Models.

v0.18.2: Multi-model that keeps VRAM free on its own

• Client loading no longer means guessing. With Multi-model on and let client requests load models ticked, any app can ask the router for a model the planner never checked. On llama.cpp v0.8.0 and newer, LlamaForge now hands the router your VRAM kept free per GPU as its --models-memory-margin, so the router unloads its least recently used model until the new one fits instead of running out of memory. Older builds keep working as before.

v0.18.1: Ollama users skip the download

• First run knows about Ollama. If you already use Ollama, the setup wizard's model step now leads with Use the N models you already have in Ollama. One click adds them in place (nothing is downloaded or copied) and picks the first chat model, so you go straight to tuning and loading. The starter downloads are still there below it. See Setup.

v0.18.0: Bring your Ollama models

• Your Ollama models, on llama.cpp. Ollama stores models as GGUF files without a .gguf name, so a scan never saw them. Setup > Scan now reads Ollama's manifests (~/.ollama/models, OLLAMA_MODELS, or the Linux service's folder) and lists every model it downloaded, marked from Ollama, with the vision projector attached. Adding one points models.ini at Ollama's own file: nothing is copied, and Ollama keeps working. A model Ollama converted for its own engine may not load in upstream llama.cpp; the load error says so. See Setup.

v0.17.3: A star, once, when it's earned

• One ask for a GitHub star. The one-line installer clones the repo, so most people who use LlamaForge never see its GitHub page. LlamaForge now asks once, in a small card at the bottom right, right after it has done something for you: an Auto-Forge run that made a model at least 1.1x faster, or the 5th model you load. Pick Not now and it never asks again. It makes no network request; the button is a plain link. The sidebar also has a small star link next to the version.

v0.17.2: Share your Auto-Forge result

• Share. When Auto-Forge finds a setup faster than yours, a Share button makes it a 1200x630 image card (before and after tokens a second, the speedup, your GPUs, the context, whether answers stay the same) and a markdown table with the flags it chose. Neither holds a file path, key or host. See Auto-Forge. API: /api/forge/share.
• Auto-Forge survives more out-of-memory cases. An out-of-memory line that llama.cpp retries past no longer fails the candidate, the window steps down when the asked context doesn't fit, and when your current setup is faster than every candidate (say it uses a drafter Forge can't rebuild) the result says so and offers Apply anyway instead of pushing Apply.
• Fix. Lite mode shows GPU layers again on llama.cpp builds whose first spelling of the flag is --gpu-layers.

v0.17.1: Claude Code launch fixes

• Claude Code on a folder works from the Launch button on Windows. The terminal window no longer closes the moment it opens, and Claude Code no longer gets 401 invalid x-api-key when LlamaForge itself was started from a Claude session (it used that session's login instead of the panel key).
• No more 500 on the first message. Claude Code 2.1 sends system messages in the middle of a conversation; Qwen's chat template rejects those ("System message must be at the beginning"). The Anthropic endpoint now moves them where every template accepts them.

v0.17.0: Auto-Forge, Claude Code on a folder

• Auto-Forge. Finds the fastest way to run a model on your GPUs by measuring it, not guessing. In a private sandbox it tries GPU placements, KV cache types and speculative drafters (DFlash, MTP) at the context you ask for, benchmarks code, prose and prompt speed, and checks token for token that a drafter gives the same answers as the model without it. Pick what you use the model for (coding, chat, long documents, agents), quick or thorough, and optionally allow lossy KV or try without the vision projector. Apply writes only what it changed into models.ini; Undo puts your values back. On a Qwen3.8-27B with an RTX 5080 and an RTX 5060 Ti it went from 26.8 to 126.0 tokens/s on code and from 26.5 to 71.0 on prose. The old setup sat mostly on the slower card at 70k context, and Forge was asked for 32k; the 5080 alone without a drafter did 52.1 (one machine, one model). NVIDIA only, and new. See Auto-Forge. API: /api/forge/status, /api/forge/start, /api/forge/cancel, /api/forge/apply, /api/forge/undo.
• Claude Code on a folder, one click. A Claude Code button on a loaded model checks that it's ready (CLI found, Anthropic endpoint on, context size, jinja, a small model for background requests), lets you pick a folder, and opens a terminal there running claude against the local model. The endpoint and key are set on that one process, so your other Claude Code sessions and ~/.claude/settings.json are untouched, and an inherited cloud API key never reaches the local endpoint. See API. API: /api/claude/check, /api/claude/pick-folder, /api/claude/launch.
• Fixes. A metrics scrape on Stats and a slot read by Elastic VRAM never load a model any more. A router that can't start because its port is taken now says so, with a fix. Controls you could only click are keyboard buttons now, and inputs that only had a placeholder have labels. Python and JavaScript lint run in CI. The docs are also served as llms.txt.

v0.16.4: Elastic VRAM, live RAM

• Elastic VRAM. When another app needs your GPU, LlamaForge moves a loaded model's layers to the CPU instead of unloading it (MoE experts first), keeps the conversation through llama.cpp's slot save and restore (after one router restart; the router gets a save folder only while Elastic is on), and moves the layers back once the memory is free. A resized model is much slower, and the dashboard row says so; click its chip to grow it back. NVIDIA only, off by default, and new: not yet tested on many machines. See Setup > Elastic VRAM. API: /api/elastic, /api/elastic/grow.
• Live system RAM on Stats. A System RAM meter sits next to the GPU memory in every look, also on machines without a GPU. "Used" leaves out cache the OS can hand back, like Task Manager's "In use". See Stats.
• A warning before an unknown architecture fails. If a model's architecture is newer than the llama.cpp checkout that would load it, the model editor says so (and to update the build) instead of the load failing with unknown model architecture. Embedding models such as EmbeddingGemma now have a section in models.ini.
• Fixes. /v1/audio/speech answers an unsupported response_format (say mp3) with a 400 instead of WAV under the wrong name. Ember quotes copied from PDFs match across line-break and soft hyphens. Reloading models.ini works on router builds that serve it on a different route (thanks @Vergillll), and allowed values: lists that wrap over several lines in --help are read whole (thanks @Vergillll).

v0.16.3: Game mode

• Game mode. Start a game and LlamaForge unloads your models; quit it and they load again. Games are recognised by their Steam, Epic, GOG, Xbox, Riot, EA or Ubisoft folder, by names you add, or (Windows) by exclusive full screen. Pick models to keep loaded through it. Off by default. See Setup.
• Discover shows VRAM in use next to the fit rating, so you can see what's already taken.

v0.16.2: load progress, Thinking and Vision switches

• Real load progress. A loading model shows loading N% and a bar through its stages (weights, draft model, vision projector) instead of a stopwatch. The numbers are llama.cpp's own load callback, which the router streams on GET /models/sse; LlamaForge listens only while something is loading. A router build without that stream keeps the elapsed-seconds display. See Models & Tuning.
• Thinking: Auto / On / Off, plus a budget. A one-click row in the model editor over llama.cpp's --reasoning and --reasoning-budget. The model's chat template decides the hint: a template with a thinking switch (enable_thinking and the like) really turns thinking off; a template that only shows <think> or [THINK] tags may keep thinking anyway. Models with neither don't get the row.
• Vision on/off. Turning a model's vision off removes its mmproj line and remembers the path, so the projector's VRAM is free and the file stays on disk. A rescan or re-download doesn't turn it back on; On restores the line.

v0.16.1

• Model Scout runs overnight again. An ember with no model of its own now uses the auto-load model from Setup when nothing is loaded, instead of skipping its jobs. See Embers.

v0.16: Embers, several models at once, an MCP server, Voice

• Voice: text to speech on your GPU. A new Voice tab speaks through llama.cpp's own llama-tts with Qwen3-TTS (10 languages) or the tiny English Pocket TTS (faster than real time, even on a CPU), optionally in a voice you record or upload (only clone voices you may use). The panel also serves an OpenAI-style POST /v1/audio/speech, so OpenAI clients get local speech. No VRAM is held between requests. See Voice.
• Several models at once. Multi-model mode places your main model on the fastest GPU that fits it and keeps worker models off that GPU. Fit is checked against footprints measured on this machine, and each verdict says whether it's measured, predicted or a rough estimate. Workers are evicted least-recently-used, and only after you agree to it. A model with no context size set is refused, with the reason. The model rows get Load as worker, Make main and Unload X and load, and Setup has a Multi-model card. API: /api/slots, /api/slots/plan, /api/slots/main, /api/slots/apply.
• Per-model builds. Pin a model to any installed llama.cpp build. An ik_llama build runs as its own llama-server on 127.0.0.1:8100 and up, with arguments translated between the builds. Crash exit codes are named instead of shown as raw numbers.
• "Runs on" for GGUFs. The file card says whether a GGUF runs on any build, mainline only or ik_llama only. The diagnosis explains invalid ggml type N instead of leaving you with the number.
• Embers. Small local agents that keep watch on a topic, keep a wiki and write you a brief on a schedule. A wiki item only counts if it quotes its source verbatim. Model Scout needs no setup, and Forge builds a new ember by interviewing you (tested on 8 models from 3B to 27B). See Embers.
• MCP server. Claude Code, Codex or any MCP client can check status, load and unload models, check fit, search and download from Hugging Face, and hand a task to a local model. See MCP Server.
• pi from Setup. The pi coding agent card installs, updates and removes pi under <root>/agents/pi (with --ignore-scripts), and can install Node.js through winget, Chocolatey or Homebrew. See Connect an Agent.
• Stowage, the new default look. Each GPU on Models and Stats is drawn as a cargo hold ruled in 1 GiB cells, with every loaded model stowed in it at its measured size. Open a model and it shows where it would go: if it won't fit, the part that doesn't fit hangs past the end of the hold and anything that would be unloaded is marked. A ledger adds up used, booked and free to the tenth. Light, dark and colorblind-safe. Hearth and Classic are one click away. Motion is subtle, never runs on the views that refresh every few seconds, and is off when your system asks for reduced motion. See Theming & Accessibility.
• No font CDN. The panel's fonts ship with it, so it renders the same offline and no third party learns you opened it.
• Stats count every loaded model, including models running as their own process, with live tokens per second for each. See Stats.
• Vision projectors pair again. A scan only attached an mmproj file to models on a fixed list of architectures, so Gemma 4, Qwen 3.5 and other current vision models lost theirs (Discussion #16). A projector now pairs when its width matches the model's, the same check llama.cpp makes when it loads one. Several quants of one model can share a projector, and with several projectors F16 wins. Files named <model>-mmproj-*.gguf count as projectors, in a scan and in Discover.
• Logs stay small. With verbose = true a router log could grow to gigabytes, and the panel read all of it every time the log view refreshed, using tens of GB of RAM (#26). The panel now reads only the end of a log, and once a router, vLLM or model log passes 50 MB it moves to .1 (7 old copies kept), both when the process starts and every minute while it runs. On Windows, a router started by run.ps1 is rotated from its next restart from the panel.
• Docs search reads the whole page, not only titles and headings, and says so when nothing matches.
• Fixes. An out-of-memory load on a model with split-mode = tensor now says that tensor split turns llama.cpp's fit off and suggests split-mode = layer. A pi_run that gave no answer and had every tool call fail is reported as a failure. The ik_llama update check reads ik_llama's own default branch (main). A busy router port points you at config.json.

v0.15: llama.cpp sizes memory, honest diagnosis, safer by default

• llama.cpp's --fit decides context, GPU layers and split. LlamaForge no longer pins ctx-size, n-gpu-layers or tensor-split. Auto-tune clears them, and the old [*] ctx-size = 150000 pin is removed once on upgrade. See First Run and models.ini Format.
• Load failures quote the attempt that failed. The diagnosis reads only the last load of that model from the router log and quotes llama.cpp's own error line. If a value you pinned turned fit off, the out-of-memory hint names it. See Troubleshooting.
• Split GGUFs count all their shards in sizes and fit estimates. Speed and fit badges are labelled as rough estimates.
• A download ends in Load & Chat, and a first run with no models suggests starters sized to your VRAM.
• Security: the router always runs with an API key, and CORS is localhost-only unless you opt into LAN. Recipes import only allowlisted tuning knobs. stop and uninstall touch only processes and files LlamaForge owns. Vulnerabilities can be reported privately (see SECURITY.md).
• The running version is shown next to the logo and requested in bug reports.

Community recipe gallery

Click browse recipes on the Profiles strip to see tested setups people have shared: the model, the hardware it ran on, what the key knobs do, and an on this machine badge when you already have the file. Import runs the normal recipe import, so a missing model downloads from Hugging Face and the same knob filter applies. The list comes live from the repo's recipes/ folder, so a merged recipe appears without a release; offline, LlamaForge shows the copy that shipped with your version. To add yours, export it with ↗ and open a pull request (see recipes/README.md).

See Models & Tuning and HTTP API.

Shareable recipes

Any llama.cpp profile can be shared as a recipe: click ↗ on its chip and copy readable JSON with the model file, the Hugging Face repo it came from, its knobs, and the llama.cpp build it was made on. Paste a recipe into + import recipe to get the same preset and profile. If you don't have the model, Download from <repo> & import fetches it. Only tuning knobs are imported; paths, hosts, keys and logging flags are always dropped.

See Models & Tuning and HTTP API.

One-click launch profiles

Save a model, a preset, and (optionally) a specific installed llama.cpp build as a named profile from the model's editor, then launch the whole combination from a ▶ chip above the model list. If the pinned build isn't the active one, LlamaForge switches to it, waits for the router, applies the preset, and loads the model. That's useful when a new llama.cpp release regresses one model and you want to keep it on the build that worked. Builds a profile pins are kept out of the automatic pruning after an update.

See Models & Tuning and HTTP API.

Focused scanning and a lighter Models tab

Setup can now persist one or more model folders instead of walking every drive, while an explicit blank list still selects the platform defaults. A llama.cpp model can be Unregistered from models.ini without deleting its GGUF file. The Models view also avoids polling collapsed logs, shares GPU telemetry through a short cache instead of spawning nvidia-smi per browser poll, and keeps the CRT scanline look without the expensive full-viewport blend mode.

Fresh bootstrap runs now accept an existing llama.cpp checkout and derive its build/server paths, removing the hand-edit step reported by WSL2 users.

See Setup, Models & Tuning, and Install.

Fail-closed network and explicit credentials

Network Access now distinguishes the loopback-only dashboard from the router it controls. LlamaForge accepts only local (127.0.0.1) or LAN (0.0.0.0) router scope, requires a usable key for every newly configured LAN router, and refuses LlamaForge-owned starts/restarts when that policy is unsafe. Existing manual or legacy settings are shown for repair rather than silently rewritten. Routine dashboard state is redacted; Client Config, Agent Config, and generated keys are deliberate no-store actions. This is tested product behavior in an early preview, not a formal security standard or audit certification.

See Security, Setup, HTTP API, and Connect an Agent.

ik_llama as a second llama-family engine

The Build tab now builds and drives ik_llama.cpp alongside stock llama.cpp, switched with one control (POST /api/engine/switch). The switch is gated on a capability probe: a binary whose llama-server lacks router mode (--models-preset) is refused with an explanation rather than taking the router down, and ik_llama keeps its own models.ini registry (a -ikllama sibling of the main one). Per-model tuning comes along automatically because the knob schema is parsed from whichever binary is active.

See Build & Update and models.ini Format.

Bind a preset as a model's default

Named presets can now be bound to a model, not just applied once. Binding writes the preset's knobs into the model's section and — the point of it — editing a bound preset re-syncs every model using it. A ◉/○ dot on each preset chip toggles the binding; the bound chip is highlighted. Hand edits still win, and unbinding leaves the knobs in place.

See Models & Tuning.

Auto-wired MTP draft models

A scan now attaches an mtp-* speculative draft sidecar to its parent model the way mmproj already is, filling spec-draft-model. It enables spec-type=draft-mtp only when the sidecar actually declares NextN layers — the signal llama.cpp itself gates on — so a model that can't use MTP is never broken. The wiring is additive: it never overwrites a spec-type you set by hand (e.g. an ngram mode).

See models.ini Format.

Offload-aware VRAM-fit rating

The FITS / TIGHT / CPU OFFLOAD badge in Discover now defers to the same physics estimate as the Will-it-run panel instead of a raw size-vs-VRAM guess. A large MoE that runs fast with its experts on CPU reads as TIGHT rather than being mislabeled CPU OFFLOAD just for being bigger than VRAM. The size heuristic remains the fallback when no prediction is available.

See Discover.

A more forgiving first run

Several fresh-install rough edges are gone. A missing config.json is seeded from config.example.json instead of crashing; models.ini is created if absent; a relative models_ini resolves against the repo root so the router never comes up with zero models; a busy router port is reported by name instead of failing silently; a freshly installed tool is detected without a restart; and a build whose llama-server succeeded but whose UI-asset step failed reports "built, with warnings" instead of BUILD FAILED.

See First Run, Setup, and Troubleshooting.

Lite and Advanced modes, with a guided first run

A first-run wizard now walks a new install through engine detection, hardware review, model selection, and a recommended tune — then applies it and loads the model. The dashboard runs in one of two modes: Lite presents a reduced, task-oriented set of controls; Advanced exposes every server flag. A hardware auto-tune proposes per-model settings (GPU-layer offload, KV-cache type, context ceiling, and intent presets for balanced / speed / context / coding) sized to the detected VRAM.

See First Run and Models & Tuning.

Anthropic-compatible endpoint

The panel exposes an Anthropic-compatible POST /v1/messages endpoint that translates to the local OpenAI-style router, with full streaming and tool-use support. Combined with the existing OpenAI-compatible surface, LlamaForge can serve clients written for either API against your local models.

See HTTP API.

One-click agent setup

A Connect an Agent panel generates — and optionally writes — the configuration for Claude Code, Codex, and pi.dev, pointing each at your local endpoint. Claude Code is scoped to 127.0.0.1; generated files are backed up before any change is written.

See Connect an Agent.

Context Wiki

A working directory of Markdown context documents, composed into named profiles and selected per model. An active profile is delivered two ways: injected into requests through the Anthropic shim and the OpenAI proxy, or exported into an agent's native context file (CLAUDE.md / AGENTS.md) inside a managed marker region. Because the injected prefix is stable, the router's prompt-cache reuses it across requests.

See Context Wiki.

Light and dark themes, plus a colorblind-safe mode

The dashboard now offers a Light theme alongside the original dark one, and an independent Colorblind-safe mode that applies a universal Okabe–Ito status palette and adds non-color cues (glyphs and labels) so status never depends on hue alone. The two axes are orthogonal — all four combinations are valid — and each choice persists per device.

See Theming & Accessibility.

In-app documentation

The documentation you are reading is available inside the dashboard under the Help view and is also published as a static site. Both surfaces render from a single Markdown source, so they never drift.

A collapsible sidebar layout

Navigation moved from a top tab bar to a left sidebar that collapses between a compact icon rail and a labeled, expanded state. Settings are pinned at the bottom; the layout adapts to narrow windows with an overlay drawer. All existing navigation, theming, and keyboard shortcuts are unchanged.

See Keyboard Shortcuts.

Refine benchmark in the Models panel

A Refine button now sits inline in each model's editor (beside the Presets bar). Pick an intent (balanced / speed / context / coding), click Run, and it auto-generates knob recommendations, benchmarks candidates with real completion requests (~200 tokens each), and applies the fastest config. A results table shows tok/s per candidate and which was chosen. The same autotune engine used in the first-run wizard is now available anytime from the main Models tab.

See Models & Tuning.

VRAM "Will-it-run" panel

Before downloading, the Discover tab now shows a Will-it-run panel that predicts whether a model quant will fit your GPU and at what approximate speed. It factors in MoE active-vs-total parameters, your GPU's memory bandwidth (with manual overrides in Setup), and the quant's size — then rates it as FITS, TIGHT, or CPU OFFLOAD with an estimated tok/s. The same estimate appears as a badge when you expand a model in Discover.

See Discover.