HTTP API
The LlamaForge dashboard backend (backend/server.py) listens on panel_port (default 8090) and serves the web UI, the /api/* management API, and two agent-facing, provider-compatible chat endpoints. The /api/* routes below are the GET_ROUTES and POST_ROUTES tables in backend/routes.py; backend/server.py holds the HTTP handler and serves /v1/messages, /v1/chat/completions, /web/* and /docs/img/* itself.
/api/* request/response bodies are JSON. Every JSON response has Cache-Control: no-store.
If vLLM support isn't available on the host, every route is first checked by _vllm_gate(); requests to /api/vllm/* paths return an error response instead of reaching the normal handler.
Agent-facing endpoints
These are the endpoints external coding agents (Claude Code, Codex, etc.) talk to — see backend/agentsetup.py for the config generators that point agents at them.
| Method | Path | Purpose |
|---|---|---|
| POST | /v1/messages | Anthropic Messages API-compatible endpoint. Requires anthropic_shim_enabled: true in config.json (the default) and uses conditional _shim_auth_ok: auth is skipped for local router scope (and current behavior also skips when no key is configured); when enforced, it accepts either x-api-key or Authorization: Bearer <key>. Supports "stream": true (SSE) via _anthropic_stream, translates to the OpenAI-shaped request, and forwards to the router. |
| POST | /v1/messages/count_tokens | Anthropic-compatible token-count estimate for a would-be /v1/messages request. Same enable and conditional auth behavior as /v1/messages. |
| POST | /v1/chat/completions | OpenAI Chat Completions-compatible endpoint with the same conditional _shim_auth_ok behavior. Injects the active wiki context profile as a system message (_inject_openai_system) before forwarding to the router. Supports "stream": true. |
| POST | /v1/audio/speech | OpenAI speech-compatible text to speech through llama.cpp's llama-tts (see Voice). Body: input, optional model, voice, language, response_format (wav, the default, or pcm; anything else such as mp3 is a 400 rather than WAV under the wrong name). Same conditional _shim_auth_ok behavior. Returns audio/wav or audio/pcm; errors in the OpenAI shape. |
| POST | /v1/audio/transcriptions | OpenAI transcription-compatible speech to text through a router model whose mmproj has an audio encoder (see Voice). multipart/form-data: file (WAV, MP3 or FLAC, 25 MB), optional model, language, prompt, response_format (json or text), temperature. Same conditional _shim_auth_ok behavior. Errors in the OpenAI shape; 502 when the model refuses the audio. |
| POST | /api/load | Load a model into the router. Body: {"model": "<id>"}. Proxies to the router's /models/load. With multi-model on, the body also takes role (main or worker) and evict, and the load goes through the placement planner; a refusal comes back with a reason. |
| POST | /api/unload | Unload a model from the router. Body: {"model": "<id>"}. Proxies to the router's /models/unload. |
| POST | /api/unload_all | Unload every currently loaded/loading model (except the router's default entry). |
Model and preset management
| Method | Path | Purpose | |
|---|---|---|---|
| GET | /api/state | Dashboard state: models (llama.cpp + vLLM merged), GPU telemetry, platform, public config projection, and onboarding status. config is an exact allowlist (theme, cvd, skin, auto_load_model, vram_bandwidths, presets, preset_bindings, profiles, active_engine) plus router_api_key_configured; it is not full config.json and never includes the key. | |
| GET | /api/schema | The knob schema (available llama-server flags), built from llama-server --help. | |
| POST | /api/save | Save per-model knob overrides into models.ini (config.set_keys). Reloads the running model if it was loaded. | |
| POST | /api/models/unregister | Remove a llama-family model from its models.ini registry without deleting the GGUF. Body: {model, backend}. Unloads it first when necessary. | |
| GET | /api/presets | List saved knob presets from config.json. | |
| POST | /api/presets/save | Save a named knob preset. | |
| POST | /api/presets/delete | Delete a named preset. | |
| POST | /api/presets/apply | Apply a saved preset's knobs to a model, same reload behavior as /api/save. | |
| POST | /api/profiles/save | Save a launch profile: {name, profile: {model, backend, preset, engine}}. engine is an install directory name under engines/llama.cpp (blank = active build). | |
| POST | /api/profiles/delete | Delete a launch profile (the model and its settings stay). | |
| POST | /api/profiles/launch | Launch a profile: switch to its pinned build if needed, apply its preset, load its model. Returns {ok, step, error, switched_engine, model}. | |
| POST | /api/profiles/export | A profile as a shareable recipe {llamaforge_recipe: 1, name, model, settings, engine}. Paths, hosts and keys are never included. | |
| GET | /api/recipes/gallery | Community recipes `{source: "github"\ | "bundled", recipes: [{id, title, hardware, author, notes, model, settings, engine, dropped, have, recipe}]}. recipe is ready to POST to /api/profiles/import. ?force=1` skips the 6 h cache. |
| POST | /api/profiles/import | Import a recipe {recipe, download?}. Creates a preset and profile, or returns {missing} when the model isn't here; download: true starts fetching it from the recipe's Hugging Face repo. Unsafe or unknown knobs come back in dropped. | |
| POST | /api/presets/bind | Bind a preset as a model's default (materializes its knobs; name: "" unbinds). Re-saving a bound preset re-syncs every model using it. | |
| GET | /api/model/metadata | GGUF metadata for a model id (query param model). | |
| GET | /api/model/diag | Diagnostic read of the router log against a model's merged ([*] + per-model) settings (query param model). | |
| POST | /api/autotune/recommend | Recommend knob values for a model given hardware constraints. Body: {model, intent} where intent is balanced, speed, context, or coding. Returns {knobs, reasons}. | |
| POST | /api/autotune/refine | Auto-generate knob recommendations, benchmark candidates with real completion requests (~200 tokens), and return the fastest config. Body: {model, intent} (knobs optional; generated if omitted). Returns {knobs, measurements: {candidates: [{knobs, tok_s}], chosen_tok_s}}. | |
| GET | /api/scan/missing | List models.ini entries whose GGUF file no longer exists on disk. | |
| POST | /api/scan | Scan directories for GGUF files. Body roots overrides saved model_dirs; explicit roots: [] selects platform defaults. | |
| POST | /api/scan/apply | Register scanned entries into models.ini and reapply ctx-size defaults. | |
| POST | /api/scan/prune | Remove models.ini sections whose file is missing (unloading first if loaded). |
Build, setup, and hardware
| Method | Path | Purpose |
|---|---|---|
| GET | /api/setup | Prerequisite tool status (git/cmake/ninja/compiler/CUDA) plus hardware recommendation. |
| POST | /api/setup/install | Install a missing prerequisite tool (Windows/macOS only). |
| GET | /api/gpus | GPU telemetry via a shared 10-second nvidia-smi cache, plus ram: {used, total} in MiB (or null). |
| GET | /api/build/info | Current commit, available updates, and recommended/saved CMake flags (per target: llamacpp or ikllama). |
| GET | /api/build/log | Tail of the build log plus builder state (phase includes done_warnings for a partial success). |
| POST | /api/build/start | Start (re)building the target engine with the given (or saved/recommended) CMake flags. |
| POST | /api/engine/switch | Point the router at llamacpp or ikllama (sets active_engine); refused if the target binary has no router mode. |
| GET | /api/engine/prebuilt | Installed official llama.cpp builds, which one is active, and the latest release on a channel (?channel=, ?force=1 to skip the cache). |
| GET | /api/engine/prebuilt/status | Progress and log tail of a prebuilt download/install. |
| POST | /api/engine/prebuilt/install | Download and install the latest official build for your GPU. Body: channel, optional variant. |
| POST | /api/engine/prebuilt/cancel | Cancel a running prebuilt install. |
| POST | /api/engine/prebuilt/use | Switch to an already-installed build (rollback). Body: dir, which must be one /api/engine/prebuilt lists. |
| GET | /api/prtry | "Try a PR": the running PR build's status and log tail, the installed PR builds with the models pinned to each, and ab: the speed comparison's phase and last result. |
| POST | /api/prtry/look | Look a llama.cpp PR up on GitHub. Body: pr (number, #N or a ggml-org/llama.cpp pull link). Returns its title, author, state and head sha. |
| POST | /api/prtry/build | Build that PR. Body: pr, sha (from look; 409 if the PR has moved since). One build at a time, never beside a source build. |
| POST | /api/prtry/cancel | Cancel the running PR build. |
| POST | /api/prtry/remove | Delete a PR build. Body: name. Unloads and unpins the models on it first. |
| POST | /api/prtry/ab | Compare speed: llama-bench on one model with your build, then with another installed build (a PR build or an official release), with every model unloaded meanwhile. Body: model, install (the build's folder name under engines/llama.cpp). 400 if that build is the one the router runs. ab.label names the other build (PR #N or the release tag). 409 while a PR build, another comparison or Auto-Forge runs. Poll GET /api/prtry for ab. |
| GET | /api/starters | Starter-model suggestions sized to your total VRAM. |
| GET | /api/feed | "New this week": architectures llama.cpp just merged, marked against your engine build, plus whether a newer LlamaForge release is out. |
| GET | /api/app/update | Progress of a LlamaForge self-update. |
| POST | /api/app/update | Update LlamaForge to a release. Body: tag. |
| POST | /api/app/restart | Restart the panel after a finished self-update. |
VRAM prediction
| Method | Path | Purpose |
|---|---|---|
| POST | /api/vram/predict | Rough estimate of whether a model quant fits your GPU and how fast it might run. JSON body: repo (HF repo id, required), quant (default q4_k_m), optional gguf_file. The size comes from the matching GGUF file (all shards). Returns {regime, tok_s, model_size_bytes, active_size_bytes}. Accounts for MoE active-vs-total parameters and GPU memory bandwidth (with Setup overrides); treat tok_s as a ballpark. |
Model Hub (download)
| Method | Path | Purpose |
|---|---|---|
| POST | /api/hub/search | Search Hugging Face for GGUF repos, annotated with what's already installed. |
| POST | /api/hub/files | List a repo's files/quantizations, sized against available VRAM. |
| GET | /api/hub/xray?repo=&path=[&ngl=][&ncmoe=][&vs=] | X-ray of a repo file from its header alone: the Model X-ray view plus vram, budget (bytes the weights may use), configured_ngl / configured_ncmoe (the placement that fits: for a MoE, all layers with the first blocks' experts on the CPU), experts, ncmoe and flags (the llama-server flags for this view). path and vs must be files the repo lists (400), unreadable header 502. |
| POST | /api/hub/download | Start downloading a model (and optional mmproj) from the Hub. |
| GET | /api/hub/progress | Current download progress. |
| POST | /api/hub/cancel / /api/hub/pause / /api/hub/resume | Control the active download. |
| POST | /api/hub/add | Register a finished manual/download-dir GGUF into models.ini. |
Network and router control
| Method | Path | Purpose | ||||
|---|---|---|---|---|---|---|
| GET | /api/network | Redacted Network Access status: canonical/legacy access scope, configured host and security/key status, key-present boolean, remediation message, configured port and LAN IP, plus router_running/listener_status. A listening port is observation only, not process-identity or authentication verification. | ||||
| POST | /api/network | Save a canonical Network Access request and restart the router. Body: `{access_scope: "local" | "lan", key_action: "keep" | "generate" | "replace" | "clear", api_key?: "..."}. replace requires a strong replacement; clear is local-only; LAN has no unauthenticated path. For one release, the old {host, api_key} shape is accepted only for canonical hosts. A generated key is returned only as generated_api_key to that explicit request. Results distinguish saved configuration from restart_status (running, starting, or failed`) and listener observation. |
| POST | /api/client/config | Deliberately generate client snippets for {model, backend}. llama-family output may contain the router key; active vLLM output does not. Unknown, ambiguous, or unloaded vLLM targets are rejected. | ||||
| GET | /api/router/log | Tail of the router's log. | ||||
| GET | /api/stats | Usage stats summary. | ||||
| POST | /api/stats/reset | Reset usage stats. | ||||
| POST | /api/config | Merge an allowlisted subset of user-facing settings into config.json (the list is in config.json Reference); other keys are refused and named in the response. Changing multi_model, slot_cap or slot_autoload restarts a running router, and the response's router field says whether it did. |
There is no GET /api/config route; current config is read via GET /api/state's config field instead.
Multi-model, per-model builds, pi, and MCP setup
| Method | Path | Purpose |
|---|---|---|
| GET | /api/slots | Multi-model pool state: enabled, multi_model, the router's running pool, restart_needed (the setting is on but the running router was started single), the main model and the loaded list. |
| GET | /api/slots/plan | Where a model would go if loaded: query model (required) and role (worker by default). Returns the verdict a load would get now (whether it fits and what would be evicted) without loading anything. |
| POST | /api/slots/main | Set the pool's main model: {model} ("" clears it). |
| POST | /api/slots/apply | Restart the router with the pool the current settings ask for. Returns {ok, restarted, error}. |
| POST | /api/model/vision | Vision on or off for one model: {model, on}. Off removes its mmproj line and keeps the path in vision_off; on restores it. The projector file is never touched. A model that is up is unloaded first. Returns {ok, vision, was_running}; 400 when the model has no projector or the parked file is gone. |
| POST | /api/model/build | Pin a model to an installed build: {model, build}; build: "" runs it on the router's own build. A model that is up is unloaded first. Returns {ok, build, changed, was_running}. |
| GET | /api/pi/status | The pi coding agent: Node.js version against the minimum, whether LlamaForge has its own copy (managed), which pi is active, whether a job is busy, and the last result. |
| POST | /api/pi/install / /api/pi/remove | Start a pi install (or update) or removal job; progress comes back through GET /api/pi/status. 409 if a job is already running. |
| GET | /api/mcp/setup | Ready-to-paste client configs for LlamaForge's MCP server, with this install's Python and script paths filled in. |
Elastic VRAM
See Setup > Elastic VRAM. The settings themselves go through POST /api/config.
| Method | Path | Purpose | ||||
|---|---|---|---|---|---|---|
| GET | /api/elastic | Elastic VRAM status: enabled, standby (Game mode has a game running), swapping (the model being resized, or null), cooldown_s (the current wait before a grow), waiting (models still to go back to full size after it was turned off), others_mib (VRAM other apps use, per GPU), models (per resizable loaded model, not one still loading: id, at (0 = the user's own size), label, gpu_layers, layers, steps, full) and events (the last 20 resizes: t, model, from, to, reason, secs, ok, kept, note). Also in GET /api/state as elastic. | ||||
| POST | /api/elastic/grow | Grow a resized model back now: {model}. Skips the wait and cooldown, not the fit check. Returns {ok: true} at once and runs the swap in the background; the outcome lands in events. 400 if the model isn't resized, 409 if it's being resized right now. | ||||
| GET | /api/idle | Sleep idle models: {supported, minutes}. minutes is 0 when off; supported is false when the llama.cpp build has no --sleep-idle-seconds (or the engine is ik_llama). | ||||
| POST | /api/idle | Set it: {minutes}, a whole number 0 to 1440 (0 turns it off). Writes sleep-idle-seconds in [*] of models.ini and asks the router to reload its presets; applies from each model's next load. Returns {ok, minutes}, plus warning if the router couldn't be told. | ||||
| GET | /api/reclaim | Free disk space: ?days= (default 90). stale (registered models not used in that many days: id, path, bytes, last_used, 0 = never), stale_bytes, dups (groups of byte-identical files: bytes, copies, ids, files [{path, ids}]), dup_bytes (what deleting all but one copy frees), since (when Stats began) and tracked_long_enough. | ||||
| POST | /api/reclaim/delete | Delete one model's GGUF file(s) from disk: {id}. Unloads it first, removes its models.ini section, returns {ok, freed} (bytes). Refuses (ok: false, nothing touched) anything that isn't a registered .gguf, a link, or a file another model uses. | ||||
| GET | /api/template/check | Template doctor: ?model=. Compares the GGUF's embedded chat template with its repo's current file (header only) and the base model's. verdict (repo_newer, base_differs, same, unknown), repo, file, base, repo_diff, base_diff (unified diffs, yours to theirs), embedded, override (the chat-template-file in use), error. Needs network. | ||||
| POST | /api/template/apply | {model, source}: repo or base saves that template from the last check to chat-templates/ and sets chat-template-file; "" removes the override. Unloads a running model; {ok, file, was_running}. | ||||
| GET | /api/samplers | Recommended samplers: ?model=. Reads generation_config.json from the GGUF's repo, else its base model. values ({knob: value} in llama-server's names), source, url, checked (repos tried), current (what models.ini sets, the model's section over [*]), inherited (knobs only [*] sets), error. Apply with POST /api/save. Needs network. | ||||
| GET | /api/reupload | Re-upload watch: the cached answer, or ?refresh=1 to check every model's HF repo now. {checked_at, stale, models: {id: {verdict, repo, path, title, date, hf_cache, error}}}; verdict is current, reuploaded, changed (newer commit, same size), gone or unknown. Needs network to refresh. | ||||
| POST | /api/reupload/fetch | {model}: re-download a reuploaded or changed model over the old file (all shards), unloading it first. Not for files in the HF cache. {ok, was_running}; progress on /api/hub/progress. | ||||
| GET | /api/forge/status | Auto-Forge state for ?model= (default: the model of the current run): stage (idle, fit, speed, sanity, done, cancelled, error), running, progress (done, total), loading, leaderboard (ranked rows: label, code, prose, prefill, vram_mib, vram_per_gpu, finalist, sanity), failed, rejected, result (winner, current, vs_current, unverified_quality, drops_vision), applied, req. See Auto-Forge. | ||||
| POST | /api/forge/start | Start a run: `{model, intent: coding\ | chat\ | long\ | agent, ctx, time: quick\ | thorough, allow_loss, drop_vision}. Unloads every model for the run and loads them back afterwards. 409 while another run is going, 400` without an NVIDIA GPU. |
| POST | /api/forge/cancel | Stop the current run; the GPUs are given back and a run with the same request resumes later. | ||||
| POST | /api/forge/apply | Write the winner into the model's models.ini section ({model}): only what Forge changed from the setup it started with, vision through the vision switch. Returns {ok, updates}. 409 while a run is going. | ||||
| POST | /api/forge/undo | Put back what the last Apply replaced ({model}). 400 if there's nothing to undo. | ||||
| GET | /api/forge/share?model= | The last result as postable data: {model, file, gpus, ctx, before, after, x, settings, lossless, markdown}, with no paths, keys or hosts. 404 with no result. | ||||
| GET | /api/claude/check | Can Claude Code run on ?model=: cli (path, version), small (the model for background requests), checks (each {level, text}), ready, recent (recent folders), install_hint. Reading the context size never loads the model. | ||||
| POST | /api/claude/pick-folder | Open the OS folder picker; returns {path} (empty if cancelled) or {unsupported: true}. | ||||
| POST | /api/claude/launch | Open a new terminal in {folder} running claude against the panel's /v1/messages, on {model}. The endpoint and key are set on that process only. Returns {launched, command} (launched: false = no terminal found, run command yourself). 409 with the failing checks if not ready. |
Embers
The Embers tab's endpoints. See Embers for what each does.
| Method | Path | Purpose |
|---|---|---|
| GET | /api/embers | Every ember as a card, the embers folder, and whether the scheduler is running. |
| GET | /api/embers/templates | The templates a new ember can start from. |
| GET | /api/embers/brief | An ember's brief: query id, optional date. |
| GET | /api/embers/pages | An ember's wiki pages and recent source snapshots (query id). |
| GET | /api/embers/page | One wiki page rendered (query id, page). |
| GET | /api/embers/raw | One source snapshot (query id, sha). |
| GET | /api/embers/log | Recent job runs and log lines (query id). |
| POST | /api/embers/create | Create an ember from a template: {template, bindings, id?}. |
| POST | /api/embers/update | Change name, bindings, jobs, model, enabled or push for {id}. |
| POST | /api/embers/delete | Remove an ember: {id}. Its wiki stays on disk; the response's kept is the folder. 409 while a job is running. |
| POST | /api/embers/run | Queue {id, job} where job is run (ingest then brief), ingest, brief or lint. |
| POST | /api/embers/cancel | Drop an ember's queued jobs: {id}. |
| POST | /api/embers/ask | Ask an ember's wiki a question: {id, question}. Uses a model that is already loaded. |
| POST | /api/embers/forge | One turn of the Forge interview that builds an ember. Uses a model that is already loaded. |
| POST | /api/embers/push/test | Send a test notification through an ember's push settings: {id}. |
| POST | /api/embers/folder | Point the panel at another embers folder: {path} ("" = the default). Nothing is moved. |
Voice (text to speech)
| Method | Path | Purpose |
|---|---|---|
| GET | /api/tts/status | Whether llama-tts and a model are present (ready, missing), the installed speech models (models) and the downloadable ones (catalog), the saved voices, languages, the model download's progress (download), busy, and needs_key for /v1/audio/speech. |
| GET | /api/tts/progress | Just the model download's progress (the download object above). |
| POST | /api/tts/speak | The Voice tab's speak call. Same body and audio as /v1/audio/speech, panel-style {"error"} on failure. |
| POST | /api/tts/get | Download a speech model from the catalog. Body: {"id"} (qwen3-tts by default, or pocket-tts-en). Qwen3-TTS goes into <tts_dir>/models, others into a subfolder; Pocket TTS also fetches a CC0 starter voice. Not registered as a chat model. |
| POST | /api/tts/voice | Save a reference clip. Body: {"name", "wav_b64"}; a 1 to 60 second WAV. |
| POST | /api/tts/voice/delete | Remove a saved voice. Body: {"name"}. |
| GET | /api/stt/status | {models}: the registry models whose mmproj has an audio encoder, i.e. the ones speech to text can use. |
| POST | /api/stt/transcribe | The panel's mic. Body: {audio} (base64 WAV, MP3 or FLAC), optional model, language. Returns {text, model}; panel-style {"error"} on failure. |
vLLM (WSL) management
Only reachable when vLLM support is available on the host (_vllm_gate).
| Method | Path | Purpose |
|---|---|---|
| GET | /api/vllm/setup | vLLM/WSL install status plus any running setup job. |
| POST | /api/vllm/setup/install | Kick off the vLLM install script inside WSL. |
| POST | /api/vllm/update | Run the vLLM update script inside WSL. |
| GET | /api/vllm/log | Tail of the vLLM process log. |
| GET | /api/vllm/schema | vLLM knob schema. |
| GET | /api/vllm/version | Installed vs. latest PyPI vLLM version. |
| POST | /api/vllm/load / /api/vllm/unload | Load/unload a registered vLLM model. |
| POST | /api/vllm/save | Save a vLLM model's settings, restarting it if currently running. |
| POST | /api/vllm/hub/search | Search Hugging Face for vLLM-servable repos. |
| POST | /api/vllm/hub/info | Repo info sized against available VRAM. |
| POST | /api/vllm/hub/download | Start a vLLM model download. |
| GET | /api/vllm/hub/progress | Download progress. |
| POST | /api/vllm/hub/register | Register a downloaded model into the vLLM registry. |
| POST | /api/vllm/delete | Delete a downloaded vLLM model and deregister it. |
Agent config and context wiki
| Method | Path | Purpose |
|---|---|---|
| POST | /api/agent/config | Deliberately preview connection config for {agent, model, backend, small, inject}. It is POST-only; the active llama-family backend is required and vLLM is rejected. |
| POST | /api/agent/apply | Write agent config on the dashboard machine using the same targeting fields (agent, model, backend, small, inject). It may use the stored key internally but never returns a key. |
| GET | /api/wiki/docs | List context-wiki documents. |
| GET | /api/wiki/doc | Read a single document (query param name). |
| POST | /api/wiki/doc | Create/update a document. |
| POST | /api/wiki/doc/delete | Delete a document. |
| GET | /api/wiki/profiles | List saved context profiles. |
| POST | /api/wiki/profile | Save a named profile (its doc list + description). |
| POST | /api/wiki/profile/delete | Delete a profile. |
| GET | /api/wiki/preview | Preview the composed text for a profile (query param profile). |
| POST | /api/wiki/active | Set the active profile for a model. |
| POST | /api/wiki/export | Export composed context (e.g. to an agent's own file format). |
Docs viewer
| Method | Path | Purpose |
|---|---|---|
| GET | /api/docs | Manifest of documentation pages (sections/order/titles). |
| GET | /api/docs/page | Rendered HTML for one page (query param slug); 404 if the slug doesn't exist. |
| GET | /docs/img/<name> | Serve an image referenced from a docs page, path-safety-checked by docs._safe_img. |
Static / UI
| Method | Path | Purpose |
|---|---|---|
| GET | / or /index.html | The dashboard's single HTML page. |
| GET | /web/js/<name>.js | A frontend ES module. Confined to web/; anything outside 404s. |
Engine-agnostic model verbs
These dispatch on the model's own backend, so the same call works for a llama.cpp GGUF and a vLLM safetensors repo. The body takes an optional backend hint ("llamacpp" / "vllm"); without it the id is looked up.
| Method | Path | Purpose |
|---|---|---|
| POST | /api/models/load | Load {model} on whichever engine owns it. |
| POST | /api/models/unload | Unload {model}. |
| POST | /api/models/save | Persist {settings} and apply them (restarting the process if that engine has no hot reload). |
| POST | /api/models/delete | Delete the model's files, where the engine supports it (vLLM only; llama.cpp returns 400). |
The engine-specific paths (/api/load, /api/vllm/load, …) remain as aliases.
Batches
The local Batch API (Batches). The /v1 routes have OpenAI's shapes and errors and the same conditional _shim_auth_ok as /v1/chat/completions; the /api/batch/* routes are what the Stats card uses, under the panel's usual guards and with no key.
| Method | Path | Purpose |
|---|---|---|
| POST | /v1/files | Upload a batch input file: an SDK's multipart/form-data (purpose, file), or JSON {filename, purpose, content}. purpose must be batch. 64 MB limit. |
| GET | /v1/files | List stored files, newest first. |
| GET | /v1/files/{id} | One file object. |
| GET | /v1/files/{id}/content | The file's bytes, application/jsonl (inputs and results alike). |
| DELETE | /v1/files/{id} | Delete a file. 400 while an open batch reads it. |
| POST | /v1/batches | Start a batch: {input_file_id, endpoint: "/v1/chat/completions", completion_window: "24h", metadata?}, plus LlamaForge's when: now (default) or idle (wait for 20 quiet minutes). The whole file is checked first; a bad line is a 400 naming it. |
| GET | /v1/batches | List batches, newest first. |
| GET | /v1/batches/{id} | One batch object, with request_counts and, once done, output_file_id / error_file_id. |
| POST | /v1/batches/{id}/cancel | Cancel. A running batch finishes the request in flight, then ends cancelled with its results so far. |
| GET | /api/batch/state | The card's view: batches (each with when and input_filename), files, quiet_minutes. |
| POST | /api/batch/run | Upload and start in one go: {filename, content, when?}. A file that fails the check is not kept. |
| POST | /api/batch/cancel | {id}: as /v1/batches/{id}/cancel. |
| POST | /api/batch/remove | {id} of a finished batch: deletes it, its results, and its input unless another batch reads it. |
| GET | /api/batch/file?id= | A file's bytes, for the card's download links. |
Request requirements
The dashboard binds 127.0.0.1, which keeps it off your network but leaves it reachable by any page in your browser. Every request is therefore checked:
Hostmust name this loopback service, andOrigin— when present — must
match it. Anything else gets 403. This blocks both cross-site requests and DNS rebinding.
- When a POST declares
Content-Type, it must beapplication/json; declared
form or other non-JSON content types get 415, helping block a cross-site <form>. This guard currently permits an absent Content-Type. The one exceptions are an SDK's multipart/form-data upload to /v1/files or /v1/audio/transcriptions with no Origin header (a browser always sends one cross-site).
- The only
DELETEis/v1/files/{id}. It is not a simple method, so a
cross-site page needs a preflight it can't pass.
- Every POST requires exactly one ASCII-decimal
Content-Length. Missing length
is 411. Malformed, duplicate, or transfer-encoded framing is 400; a short body is also 400. Rejected framing closes the connection.
- Management POSTs are limited to 4 MiB.
/v1/messages,
/v1/chat/completions, /v1/files, /v1/audio/transcriptions and /api/stt/transcribe use a 64 MiB limit. A body over its route-class limit is 413 before it is read or dispatched.
POST /api/configonly accepts an allowlist of user-facing keys (the list is in config.json Reference). See
See also config.json Reference for the settings /api/config and /api/network write, and models.ini Format for the file /api/save, /api/scan/apply, /api/scan/prune, and /api/hub/add mutate.