# LLMock — reference for coding agents > `llmock` is a local HTTP server that imitates LLM APIs (OpenAI chat completions, Anthropic Messages, Gemini generateContent, OpenAI embeddings) with generated or canned replies. Use it so an app or test suite can run without a real provider. This file is the compact reference for agents wiring llmock into a project; the human docs are at https://llmock.piyook.com/. Describes the llmock version this file ships with (read it from `node_modules/llmock/llms.txt`). Node >= 24 (tested on 26 LTS). Nothing here needs an API key or network access. ## Mental model - One server process serves **one preset** (a named entry in `.llmockrc.json` → `models`). A preset fixes the chat route, the wire format, and the reply behaviour. To mock two providers at once, run two servers on two ports. - Config is read once at startup. Any change to `.llmockrc.json` needs `stop` + `start`. Fixture files and the stored responses file are re-read on every request, so editing them needs no restart. Response rules, chaos settings and the response delay can also be changed on a running server over HTTP (see Admin API); nothing else can be changed without a restart. - No auth: API keys and headers are ignored (SDKs still need a non-empty dummy key). - Replies are text, except that a `claude` preset's response rule can reply with tool calls (`tool_use` blocks, see Tool calls). There are no tool calls in the other formats and no token counting endpoint. Failures (an HTTP error, or a stream that fails part-way through) are simulated only with `chaos` on (see Chaos). A reply ends normally unless its response rule sets `stopReason` (see Fixture replies). ## Run it ```bash npm install --save-dev llmock npx llmock start --model=claude # detached; returns once the server answers npx llmock start --model=claude --foreground # attached; use in Docker/CI or to see errors npx llmock stop # add --port= if not 8001 npx llmock start --model=claude --port=0 # any free port: prints it, and the stop command ``` - Detached `start` waits until the server answers, then exits 0. It exits 1 if the server fails to start (the server's output is not attached: rerun with `--foreground` to see the error) or if a server already answers on that port. - With `--foreground` the command never returns; poll for readiness yourself: `GET /ping` → `200 {"response":"server is running"}`. - Confirm what is live: `GET /ui-meta` → JSON with `version` (llmock version), `llmName`, `llmUrlEndpoint`, `serverPort`, `streamingStatus`, `validateRequests`, delays. - `--port=0`: the system picks a free port. Detached `start` prints `Server is running at: http://:` and `Use 'llmock stop --port='`; `--foreground` prints `SERVER UP AND RUNNING ON LOCALHOST:`. `GET /ui-meta` → `serverPort` is the real port. - In-process (TypeScript runners such as Vitest or tsx; plain `node` will not load llmock's `.ts` source from `node_modules`): `import { startServer } from 'llmock'`; `startServer({ config?, model?, port?, host?, settings? })` → `{ port, host, url, requests(filter?), lastRequest(filter?), clearRequests(), close() }` (`requests`/`lastRequest` are the recorded calls, see Recorded requests; they work with the admin API off). `config` is a file path (relative to the cwd; rule files resolve against its folder) or the config object; default `./.llmockrc.json`, else the built-in presets. `port` defaults to the config's; `host` defaults to `127.0.0.1` (not the config's). `settings` takes any CLI flag by name (`{ stream: true, delayMin: 0 }`). One server per process at a time (a second call before `close()` throws; Vitest's per-file processes can each run one); `close()` restores the env vars it set. The request log file is shared by every llmock server on the machine. - `stop` kills whatever process is listening on the port (default 8001, it does not read the config). Do not point it at a port another service uses. - Run the CLI from the folder containing `.llmockrc.json`, or set `CONFIG_PATH=` (relative to the cwd). - Scaffold alternative: `npm create llmock@latest ` creates a standalone folder (config, Docker files, npm scripts). It prompts interactively if `` is omitted or already exists. Its scripts: `npm run llmock:start`, `llmock:stop`, `llmock:`. Its Docker setup mounts only `.llmockrc.json`; templates, fixtures and a stored responses file are copied into the image, so changing them needs `npm run docker:rebuild` (or an added volume). Its `gemini` preset already uses the endpoint `v1beta/models/gemini-pro:generateContent`, so Google's SDK works there without the change described below. ## Point a client at it (default port 8001) | Client | Preset | Setting | |---|---|---| | Anthropic SDK | `claude` | `baseURL: "http://localhost:8001"`, `apiKey: "mock"` (or env `ANTHROPIC_BASE_URL`). SDK calls `POST /v1/messages`. | | OpenAI SDK / LangChain `ChatOpenAI` | `chatgpt` | `baseURL: "http://localhost:8001/chatgpt"`, `apiKey: "mock"`. SDK calls `POST /chatgpt/chat/completions`. | | OpenAI SDK `client.responses` (Responses API) | any `openai`-template preset | Same base URL as chat: served at `/v1/responses`, and at `//responses` when the chat endpoint is `/chat/completions` (stock `chatgpt`: `baseURL: "http://localhost:8001/chatgpt"` → `POST /chatgpt/responses`). See Responses API. | | OpenAI SDK, chat + embeddings on one base URL | custom | Set the preset `endpoint` to `"v1/chat/completions"` and use `baseURL: "http://localhost:8001/v1"`. | | Google Gen AI SDK (`@google/genai`) | `gemini`, endpoint changed | Set the preset `endpoint` to `"v1beta/models/gemini-pro:generateContent"`, then `new GoogleGenAI({ apiKey: "mock", httpOptions: { baseUrl: "http://localhost:8001" } })` and call `generateContent` with `model: "gemini-pro"`. SDK calls `POST /v1beta/models/:generateContent`. | | Gemini-style REST | `gemini` | `POST http://localhost:8001/models/gemini-pro:generateContent` | | Raw HTTP | any | `POST http://localhost:8001/` with `Content-Type: application/json` | Embeddings are always at `/v1/embeddings`, not under the chat prefix: with the stock `chatgpt` preset an OpenAI client using base `…/chatgpt` will 404 on embeddings. Gemini SDKs need the endpoint change above: the stock `gemini` endpoint has no `/v1beta` prefix, so a Google SDK pointed at the mock gets a 404. The model is part of the path, so the `model` the client sends must match the one in `endpoint` (one model per preset; `gemini-2.5-flash` against a `gemini-pro` endpoint is a 404). `generateContentStream` is not mocked: it calls `:streamGenerateContent?alt=sse`, which has no route, and `stream: true` sends OpenAI-style chunks that the Google SDK cannot read. Use `generateContent` in tests. ## `.llmockrc.json` Looked up in the cwd. If absent, built-in defaults are used (presets `chatgpt`, `gemini`, `streaming`, `embeddings`, `claude`; note `chatgpt`/`gemini`/`streaming` default to a 3000–5000 ms delay). If present it **replaces** the defaults entirely; nothing is merged. ```json { "defaultModel": "claude", "models": { "claude": { "name": "claude", "model": "claude-opus-5-5", "endpoint": "v1/messages", "responseType": "lorem", "maxLoremParas": 4, "validateRequests": false, "logRequests": true, "debug": false, "stream": false, "responseDelay": { "min": 0, "max": 0 }, "embeddings": { "enabled": false, "dimensions": 128 }, "responseRules": [ { "match": "Classify this ticket", "file": "fixtures/triage.json" } ] } }, "server": { "port": 8001, "host": "0.0.0.0" } } ``` Only `name` and `endpoint` are required in a preset. The Default column applies to fields omitted from a preset in your file (the built-in presets set their own values). | Field | Type | Default | Meaning | |---|---|---|---| | `defaultModel` | string | `chatgpt` | Preset used when `--model` is not given. | | `models..name` | string | required | Wire format: `"openai"`, `"gemini"`, `"claude"`, or the name of your own templates (see Custom formats). `"claude"` also switches on Anthropic-specific behaviour. | | `model` | string | `mock-model` | Informational. Only the claude format uses it (fallback when the request has no `model`). | | `endpoint` | string | required | Chat route, no leading slash. Registered for both POST and GET. Must not be `v1/embeddings` while embeddings are enabled (startup error). | | `responseType` | `"lorem"` \| `"stored"` | `lorem` | `lorem`: 1..`maxLoremParas` random sentences. `stored`: random pick from the preset's `storedResponsesFile`, or from 11 fixed strings bundled in the package when that is unset. Any other value: 1–10 lorem paragraphs. | | `maxLoremParas` | number | 8 | Max **sentences** for `lorem`. | | `validateRequests` | boolean | false | See Validation. | | `logRequests` | boolean | false | Record the most recent validated requests (shown at `GET /ui-request-log`). | | `maxLoggedRequests` | number | 10 | How many requests the log keeps. Capped at 100; a value below 1 or not a number falls back to 10. | | `debug` | boolean | false | Verbose console output. | | `stream` | boolean | false | SSE for `openai`/`gemini` formats. On `openai`, only for requests without a boolean `stream` of their own. Ignored by `claude`. | | `responseDelay.min/max` | ms | 0 / 0 | Random wait before each reply. `0`/`0` disables. | | `embeddings.enabled` | boolean | false | Registers `/v1/embeddings`. When false the route 404s. | | `embeddings.dimensions` | number | 128 | Default vector length. | | `responseRules` | `{match, file, stopReason?}[]`, `{match, files, stopReason?}[]`, `{match, text, stopReason?}[]`, `{match, content}[]` / `{match, contentFile}[]` (claude only) or `{match, fail}[]`; any rule may add `in` and `exact` | none | See Fixture replies and Tool calls. A rule with `fail` fails the calls that match it. | | `storedResponsesFile` | string | none | JSON file of texts for `responseType: "stored"`. See Stored replies. | | `recordings` | `{dir?, upstream?}` | none | Replay recorded replies from `dir` (default `recordings`, relative to the config); with `upstream` also record new ones. See Recording. | | `chaos.enabled` | boolean | false | Fail some calls. See Chaos. | | `chaos.frequency` | integer ≥ 1 | 1 | Fail 1 in this many calls. `1` fails every call. | | `chaos.mode` | `"every"` \| `"random"` | `every` | `every`: calls X, 2X, 3X… fail (deterministic). `random`: each call fails with probability 1/X. | | `chaos.status` | 400–599 | 500 | HTTP status of the error. With `kind: "stream-error"` it picks the error type sent in the stream. | | `chaos.kind` | `"http"` \| `"stream-error"` \| `"stream-drop"` \| `"stream-stall"` | `http` | What a failing call gets: an HTTP error, or a stream that starts and then errors, is cut, or goes silent. | | `chaos.afterChunks` | integer ≥ 0 | 2 | Content deltas a failing stream sends before a `stream-*` kind fails it. | | `server.port` | number | 8001 | Listen port. | | `server.host` | string | `0.0.0.0` | Bind address. Use `127.0.0.1` to keep the mock off the network; keep `0.0.0.0` in Docker and publish the port on loopback instead (`127.0.0.1:8001:8001`). | | `server.admin` | boolean | true | `false` leaves the `/admin/*` routes out (see Admin API). | | `server.uiTheme` | `"dark"` \| `"light"` | `dark` | Dashboard colours only; no effect on the API. A viewer can swap themes with the switch in the dashboard's sidebar (kept in that browser, wins over this). Also set by the `UI_THEME` environment variable (wins over the config; `--uiTheme` wins over both). Any other value means `dark`. Reported as `uiTheme` by `GET /ui-meta`. | | `server.llmspy` | `true` \| `false` \| URL | off | Print how to use the server through LLMSpy at start, and report it as `llmspy` in `GET /ui-meta` (see Seeing the calls in LLMSpy). `true` means `http://127.0.0.1:8080`; a URL is LLMSpy's address. `--llmspy[=]` wins. Any other value stops the server at startup. | CLI flags override the selected preset for one run: `--model --port --host --admin --uiTheme --llmspy --endpoint --responseType --maxLoremParas --validateRequests --logRequests --maxLoggedRequests --debug --stream --delayMin --delayMax --chaos --chaosFrequency --chaosMode --chaosStatus --chaosKind --chaosAfterChunks --embeddings --embeddingDimensions --record --recordings --foreground`. `--record=` = `recordings.upstream` (`--record=` empty turns recording off for the run, replay only); `--recordings=` = `recordings.dir`. Both `--key=value` and `--key value` work. A boolean flag is on for `true`, `1`, `on` or `yes` (any case) and off for anything else; every flag except `--foreground` needs a value. ## Fixture replies (`responseRules`) The way to get deterministic output, e.g. JSON your code parses. - A rule fires when `match` occurs as a **case-sensitive substring** of any string value anywhere in the request body: every message of every turn, the system prompt, content blocks, and also values like `model` and `role`. Keys are not searched. - First matching rule in array order wins. Put specific rules first and use distinctive markers (`"user"` would match every request). - Because earlier turns are searched (and, in the claude format, the `tools` list with its names and descriptions), a rule that matched turn 1 keeps matching all later turns of that conversation. - `"in": "last"` on a rule searches only the content of the last entry of `messages` (string or blocks, `tool_result` content included; whatever its role). The system prompt, `tools`, earlier turns and `role` are not searched, nor are the values of `id`, `tool_use_id`, `tool_call_id` and `call_id` keys in the last message. A body with no `messages` never matches. `"in": "any"` (the default) is the whole-body search above. Use `last` for the steps of a tool loop (see Tool calls). - `"exact": true` (boolean, default false): the searched text (the strings joined by `\n`) must equal `match`, not contain it. E.g. with `in: "last"`, a Claude tool_result turn whose result is `21C` reads `"tool_result\n21C"`. - The reply text is the file's bytes, verbatim, including a trailing newline. It is inserted as the assistant message text in the preset's normal envelope (and streamed if streaming applies); it is not the raw HTTP body. - `file` resolves relative to the folder containing the config file. - Random variety for one kind of request: give the rule `files` instead of `file`, e.g. `{ "match": "Summarise", "files": ["fixtures/a.txt", "fixtures/b.txt"] }`. One file is picked at random on each matching request. A rule has `file` or `files`, not both. - Inline reply: give the rule `text` instead of a file, e.g. `{ "match": "Classify", "text": "{\"priority\":\"high\"}" }`. The reply text is that string, verbatim (`""` is an empty reply). A rule has only one of `text`, `file`, `files`, `content`, `contentFile` (the last two are tool calls, see Tool calls). - No rule matched → normal `responseType` reply. - How the reply ends: add `stopReason` to a rule (with `file` or `files`) to stand for a truncated or refused reply, e.g. `{ "match": "Summarise this", "file": "fixtures/cut-off.txt", "stopReason": "max_tokens" }`. Values: `end` (the default: a normal end), `max_tokens`, `refusal`. The Responses section lists what each becomes on the wire. The reply text is still the file's bytes: llmock does not shorten it, so write a fixture that looks cut off (or, for a refusal, whatever text your app should see). - Failing one prompt: give a rule `fail` and every call that matches it fails, whether or not `chaos` is on, while every other call is left alone. E.g. `{ "match": "Summarise the Q3 report", "fail": { "kind": "stream-error", "status": 529, "afterChunks": 1 } }`. - `fail` is an object with three optional settings that mean what they mean in `chaos`: `kind` (default `http`), `status` (default 500), `afterChunks` (default 2). `"fail": {}` is an HTTP 500. The rule's settings are its own: nothing is taken from the preset's `chaos`. - What the call gets is exactly what a failing chaos call of that `kind` gets (see Chaos): the same error bodies, the same `x-llmock-chaos: true` header, the same handling of a call that did not ask for a stream. - `file`/`files` are optional on a rule with `fail`. They only matter to a `stream-*` kind, as the text the stream sends before it fails; with neither, that text is the normal `responseType` reply. `stopReason` is never reached. - The rule fails every time it matches, so a client that retries the same prompt fails again. To have a retry succeed, use `chaos` with `frequency: 2` instead. - A call failed by a rule is added to `chaosInjected` but is not counted as a chaos call, so it does not shift which of the other calls `chaos` fails. The first matching rule still wins, so put a `fail` rule above a broader rule it overlaps with. - `match` or `file` empty/non-string, `text` not a string, `files` not a non-empty array of non-empty strings, more than one of `text`, `file` and `files` set, none set on a rule without `fail`, a `stopReason` other than those three (or `tool_use` on a tool-call rule), an `in` other than `any`/`last`, an `exact` that is not a boolean, a tool-call rule that is not valid (see Tool calls), or a `fail` that is not an object or has an unknown `kind`, a `status` outside 400–599 or an `afterChunks` that is not an integer ≥ 0 → server refuses to start. File missing → startup warning (every entry of `files` is checked), and a matching request that picks it gets `500` (no fallback to lorem). - GET requests have no body, so rules never apply to them. - `GET /ui-meta` → `responseRules` lists the active rules in match order as `{"id", "source", "match", "in", "text", "content", "files": [...], "stopReason", "fail", "times"}` (`source` is `"runtime"` for a rule added through the Admin API, which is listed first and has an `id`, else `"config"` with `id: null`; `in` is `"any"` unless the rule sets `"last"`; `text` is `null` unless the rule has one; `content` is a tool-call rule's blocks as configured (ids it gives none of are not filled in), else `null`; a single `file` or a `contentFile` is reported as a one-entry `files`, no fixture as `[]`; a tool-call rule reports `stopReason` `"tool_use"`, any other rule with no `stopReason` `"end"`; `fail` is `null`, or `{"kind", "status", "afterChunks"}` with the defaults filled in). - `GET /ui-rule-file?rule=&file=` → `{"match", "file", "content"}` for file `j` of config rule `i` (both zero-based; `i` counts the config file's rules only, so rules added through the admin API do not shift it). Unknown position → `404`; unreadable file → `500` with `{"error"}`. ## Admin API (`/admin/*`) The way to set up a reply or a failure from inside a test, with no fixture file, no config edit and no restart. Changes made here live in memory only: the config file is never written and a restart drops them. | Route | Body | Returns | |---|---|---| | `GET /admin/rules` | none | `200 {"rules": [...]}`: every active rule in match order, in the `responseRules` shape of `GET /ui-meta` | | `POST /admin/rules` | `{"match", "text"?, "content"?, "in"?, "stopReason"?, "fail"?, "times"?}` | `201 {"rule": {...}}` with its `id` (`"r1"`, `"r2"`…); `400 {"error"}` if not valid (nothing is added) | | `DELETE /admin/rules/` | none | `200 {"rules": [...]}`; `404 {"error"}` for an unknown id | | `DELETE /admin/rules` | none | `200 {"rules": [...]}`: runtime rules removed, config rules kept | | `GET /admin/chaos` | none | `200 {"chaos": {"enabled", "frequency", "mode", "status", "kind", "afterChunks"}, "stats": {"calls", "injected"}}` | | `PATCH /admin/chaos` | any of `{"enabled", "frequency", "mode", "status", "kind", "afterChunks"}` | `200` in the `GET /admin/chaos` shape; `400 {"error"}` if not valid (nothing is changed) | | `GET /admin/delay` | none | `200 {"delay": {"min", "max"}}` (ms) | | `PATCH /admin/delay` | `{"min"?, "max"?}` (ms) | `200 {"delay": {"min", "max"}}`; `400 {"error"}` if not valid (nothing is changed) | | `GET /admin/settings` | none | `200 {"settings": {"responseType", "maxLoremParas", "stream", "validateRequests", "logRequests", "maxLoggedRequests"}}` | | `PATCH /admin/settings` | any of those six keys | `200` in the `GET /admin/settings` shape; `400 {"error"}` if not valid (nothing is changed) | | `GET /admin/requests` | none; query `match`, `in`, `path` (all optional) | `200 {"requests": [...]}`: the recorded calls, oldest first, that pass the filter; `400 {"error"}` for an unknown or invalid filter. See Recorded requests. | | `GET /admin/requests/` | none | `200 {"request": {...}}`; `404 {"error"}` for an id not recorded | | `DELETE /admin/requests` | none | `200 {"requests": []}` | | `GET /admin/recording` | none | `200 {"recording": {"mode", "paused", "upstream", "dir", "forwarded", "saved"}}` (the `/ui-meta` `recording` shape; see Recording) | | `PATCH /admin/recording` | `{"paused": true \| false}` (only that key) | `200` in the `GET /admin/recording` shape; `400 {"error"}` for any other body, or for `paused: true` with no upstream (mode `replay`/`off`) | | `POST /admin/reset` | none | `200 {"rules", "chaos", "stats", "delay", "settings", "recording"}`: recorded requests forgotten, runtime rules removed, chaos, delay and reply settings back to their startup values (config file plus CLI flags), recording not paused, chaos counts and recording `forwarded`/`saved` back to 0 | - `match`, `in`, `exact`, `content`, `stopReason` and `fail` mean exactly what they mean in `responseRules` (see Fixture replies and Tool calls). A rule needs `text`, `content` (claude presets only, else `400`) or `fail`; with `fail` too, the reply is what a `stream-*` failure sends before it fails. - `file`, `files` and `contentFile` are refused with `400`: the reply must be `text` or `content`. Any other key is refused too. - `times` (integer ≥ 1, runtime rules only; a `times` in the config file's `responseRules` is ignored): the rule answers that many matching requests, then removes itself. Omitted → the rule stays until removed or reset. Each matching request the rule replies to or fails (its own `fail`) counts once; a request that chaos fails does not count, so the rule is still there for the retry. A listing shows the requests left as `times` (`null` for a rule that stays and for config rules). Fail once then succeed: `{"match": "X", "fail": {"status": 529}, "times": 1}`. A reply per call in turn: add `{"match": "X", "text": "first", "times": 1}` then `{"match": "X", "text": "second", "times": 1}`. - Order: runtime rules first, in the order added, then the config rules. First match wins, so a runtime rule with the same `match` as a config rule replaces it until removed. - `PATCH /admin/chaos` changes only the keys sent. Values are held to the config file's limits (see Chaos): `enabled` a JSON boolean, `frequency` an integer ≥ 1, `mode` `every`/`random`, `status` an integer 400–599, `kind` one of the four kinds, `afterChunks` an integer ≥ 0. Strings such as `"true"` or `"429"` are refused. Unknown keys are refused. - `PATCH /admin/chaos` does not touch the call count. For a known failing call with `mode: "every"`: `POST /admin/reset`, then `PATCH /admin/chaos`; the count then starts at 0 (with `frequency: 2` the 1st call succeeds and the 2nd fails). `stats.calls` only counts calls made while chaos is enabled. - `PATCH /admin/delay`: `min` and `max` are integers from 0 to 2147483647 and the resulting `min` must not exceed `max` (a key not sent keeps its value), so send both when raising the delay from `0`/`0`. - `PATCH /admin/settings` changes only the keys sent, with the preset's names and JSON types: `responseType` `"lorem"`/`"stored"`, `maxLoremParas` an integer 1–1000, `stream`/`validateRequests`/`logRequests` JSON booleans (not `"ON"`/`"true"`), `maxLoggedRequests` an integer 1–100. Unknown keys are refused. `"stored"` uses the preset's `storedResponsesFile`, or the bundled texts if it has none; an unreadable file → `400`. `stream` is ignored by the `claude` template (the request's own `stream` decides). Not changeable while running: preset, template (`name`), `model`, `endpoint`, `embeddings`, `debug`, `storedResponsesFile`, port and host. - A change applies from the next request. `GET /ui-meta` reports the settings in use. - Send `POST /admin/rules` and the `PATCH` routes with `Content-Type: application/json`. Send the routes that take no body without a body. - An `id` is never reused, also after a reset. Config rules have `id: null` and can not be removed. - `POST /admin/reset` also drops connections a `stream-stall` is holding open and forgets the recorded requests. It does not touch the request log. - The state is per server process and shared by every client: tests that add rules to one server must not run in parallel against it unless their `match` strings differ. - Off with `"server": {"admin": false}` or `--admin=false` (the routes then 404); `0`, `off` and `no` are read as `false`, in any case. `GET /ui-meta` → `adminApi` is `ENABLED` or `DISABLED`. While it is on, a preset `endpoint` equal to `admin` or starting with `admin/` → startup error. No auth, like the rest of the server: use `server.host` `127.0.0.1` to keep it off the network (in Docker, publish the port as `127.0.0.1:8001:8001`). Turning the admin API off leaves `GET`/`DELETE /ui-request-log`, which holds the logged prompts, open to anyone who can reach the port. Per-test fixture (any runner): ```js const mock = 'http://localhost:8001'; const addRule = (rule) => fetch(`${mock}/admin/rules`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(rule), }); beforeEach(() => fetch(`${mock}/admin/reset`, { method: 'POST' })); test('high priority tickets are escalated', async () => { await addRule({ match: 'Classify this ticket', text: '{"priority":"high"}' }); // ...call the code under test, which calls the mock... }); ``` ## Recorded requests (asserting what the app sent) - Every call to the chat route (GET or POST) and to `/v1/embeddings` is kept in memory, oldest first, last 500 only. Always on (independent of `logRequests`), calls that fail validation included, never written to disk. Per server process; cleared by `DELETE /admin/requests`, `POST /admin/reset`, `clearRequests()`, a restart, and a new `startServer`. - Entry: `{"id", "time", "method", "path", "query", "headers", "body", "valid", "streamed", "rule", "failed", "forwarded"}` (`forwarded`: see Recording). `id` = the reply's `x-llmock-request-id`. `time` ISO 8601. `path` has no query string. `headers` lower-case names; `authorization`, `proxy-authorization`, `x-api-key`, `api-key`, `x-goog-api-key`, `cookie` are `"[redacted]"`; so are the query parameters `key`, `api_key`, `api-key`, `apikey`, `access_token` (any case; Gemini clients send `?key=`). `body` the parsed JSON (`null` for none). `valid` false = failed validation (400). `streamed` true/false, `null` when nothing was sent (failed validation, or a `stream-drop`/`stream-stall` of a call that asked for no stream). `rule` `{"id", "match"}` of the rule that answered or failed it (`id` null for a config rule), `null` if none, or if chaos failed the call before the rule answered. `failed` `{"by": "rule"|"chaos", "kind", "status", "afterChunks"}` (the settings in force) or `null`. - Filter: `match` (non-empty, case-sensitive substring of the body's string values, as a rule's `match`), `in` `any`/`last` (as a rule's `in`), `path` (exact route; a missing leading `/` is added). Unknown keys or bad values → `400` (HTTP) or a thrown TypeError (`requests()`). - `requests()` and `lastRequest()` return copies. ```ts await app.classify('Printer on fire'); const call = server.lastRequest({ match: 'Printer on fire' }); expect(call?.body).toMatchObject({ max_tokens: 1024 }); ``` ## Stored replies (`storedResponsesFile`) The way to get varied but realistic output: a pool of your own texts, one picked at random per request. - Only used when `responseType` is `"stored"`. A matching `responseRules` entry still wins; the pool is the reply for everything no rule matches. - The file is a JSON array of strings: `["first reply", "second reply"]`. Entries may also be objects with a string `content` (`[{ "id": 1, "content": "..." }]`, the shape of the bundled data; other keys are ignored). - The path resolves relative to the folder containing the config file. - Empty/non-string path → server refuses to start. With `responseType: "stored"`, a file that is missing, unreadable, not valid JSON, not an array, empty, or has an entry of another shape → server refuses to start. - The file is checked at startup and re-read on every request, so editing it needs no restart. If it becomes invalid while the server runs, requests get `500` (no fallback to the bundled strings). - Unset → the bundled strings are used. - GET requests to the chat route also reply from the pool (only rules need a body). - `GET /ui-meta` → `storedResponsesCount` is the size of the pool in use and `storedResponsesFile` its configured path (`null` = bundled strings; both `null` unless `responseType` is `"stored"`). - `GET /ui-stored-responses` → `{"file", "responses": [...]}`: every text in the pool in use. ## Validation With `validateRequests: true`, a POST body must be a JSON object containing these **top-level keys** (presence only; values and extra keys are not checked): | `name` | Required keys | |---|---| | `openai` | `model`, `messages` | | `claude` | `model`, `max_tokens`, `messages` | | `gemini` | `contents` | | embeddings route (any preset) | `model`, `input` | Failure → `400 text/plain` (the message names the preset, not the missing key; the detail is at `GET /ui-request-log`). The required keys are the top-level keys of the request template, so a project template changes them. Nothing inside the keys is checked, so every request of a claude tool loop passes: a `tools` array, `tool_choice`, assistant turns of `tool_use` blocks, user turns of `tool_result` blocks (string or block `content`, `is_error`). ## Responses **OpenAI format** (`name: "openai"`), non-streaming: `{"id","object":"chat.completion","created","model","usage":{…},"choices":[{"message":{"role":"assistant","content":""},"finish_reason":"stop","index":0}]}`. `id` and `created` are constants. `model` echoes the request's `model` (the preset's `model` when absent, e.g. a `GET`). `usage` is estimated at ~4 characters per token: `prompt_tokens` from the request's `messages`, `completion_tokens` from the reply text, `total_tokens` their sum. (Up to 3.13.0 `model` was always `gpt-3.5-turbo-0301` and `usage` 12/99/111.) Only the `openai` format does this; other templates (gemini, custom) are sent as written. Streaming is decided per request, as by the real API: `"stream": true` in the body gets SSE and `"stream": false` gets JSON, whatever the preset says. A request with no boolean `stream` (or a `GET`) goes by the preset's `stream` setting. Stream shape: `data: {chat.completion.chunk}` lines — one role delta, then content deltas (which concatenate to the exact text), then an empty delta with `finish_reason: "stop"`, then `data: [DONE]`. Every chunk has the echoed `model`. With `"stream_options": {"include_usage": true}` (openai format only) every chunk also has `"usage": null`, and a last chunk with `"choices": []` and the estimated `usage` comes before `data: [DONE]`; a chaos-failed stream stops before it. **OpenAI Responses API** (any preset with `name: "openai"`; not served by claude, gemini or custom presets): `POST /v1/responses`, plus `//responses` when the chat endpoint is `/chat/completions` (never the chat endpoint itself). `GET` on those routes = generated reply, no validation, no rules. Validation (if on) needs `model` and `input` (template `openai_responses_req.json`; a project copy in `request-templates/` wins). Non-streaming: `{"id":"resp_<48 hex>","object":"response","created_at","status":"completed","background":false,"error":null,"incomplete_details":null,"instructions","max_output_tokens","model","output":[{"id":"msg_<48 hex>","type":"message","status":"completed","role":"assistant","content":[{"type":"output_text","text":"","annotations":[]}]}],"parallel_tool_calls","previous_response_id","reasoning":{"effort":null,"summary":null},"store","temperature","text","tool_choice","tools","top_p","truncation","usage":{"input_tokens","input_tokens_details":{"cached_tokens":0},"output_tokens","output_tokens_details":{"reasoning_tokens":0},"total_tokens"},"user","metadata"}`. `model` and the request settings are echoed (API defaults when absent: `instructions` null, `temperature` 1, `top_p` 1, `tools` [], `tool_choice` "auto", `store` true, `metadata` {}, `text` {"format":{"type":"text"}}, `truncation` "disabled"). `input_tokens` estimated from `instructions` + `input`. The SDK's `response.output_text` is the text. Streaming per request as on the openai chat route (`stream: true`/`false`, else the preset's `stream`). SSE `event: \ndata: {"type","sequence_number",…}` (sequence from 0): `response.created` and `response.in_progress` (`response` with status `in_progress`, `output: []`, `usage: null`), `response.output_item.added` (message, `in_progress`, no content), `response.content_part.added` (empty `output_text`), `response.output_text.delta` × ~5 (`item_id`, `output_index` 0, `content_index` 0, `delta`, `logprobs: []`; they concatenate to the exact text), `response.output_text.done` (`text`), `response.content_part.done`, `response.output_item.done`, `response.completed` (the full non-streamed response). No `[DONE]`. `client.responses.stream(...).finalResponse()` rebuilds the same response (tested against `openai` 7.62). Stop reasons: `max_tokens` → `status: "incomplete"`, `incomplete_details: {"reason":"max_output_tokens"}`, message `status: "incomplete"`, last event `response.incomplete`. `refusal` → part `{"type":"refusal","refusal":""}` (so `output_text` is `""`), streamed as `response.refusal.delta` / `response.refusal.done`. Rules: `in: "last"` searches the last `input` item's `content` (or a `function_call_output`'s `output`), or `input` itself when it is a string; `content`/`contentFile` (tool calls) are claude-only, so not available. Chaos/`fail`: HTTP errors use the OpenAI error body; `stream-error` → opening events and `afterChunks` text deltas, then `event: error` `{"type":"error","code":"server_error"|"rate_limit_exceeded","message","param":null,"sequence_number":}` and the response ends; `afterChunks: 0` stops after `response.content_part.added`. Not mocked: `function_call`/reasoning output items, `previous_response_id` state, `GET /v1/responses/` and the other Responses routes. **Claude format** (`name: "claude"`), non-streaming: `{"id":"msg_","type":"message","role":"assistant","model":"","content":[{"type":"text","text":""}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens","output_tokens","cache_creation_input_tokens":0,"cache_read_input_tokens":0}}`. Token counts are estimates (~4 chars per token). Streaming is per request: SSE if and only if the body has `"stream": true`. Events, in order: `message_start`, `content_block_start`, `content_block_delta` × n (`text_delta`, concatenating to the exact text), `content_block_stop`, `message_delta` (`stop_reason: "end_turn"`), `message_stop`. One text block, except for a tool-call reply (see Tool calls). **Gemini format** (`name: "gemini"`): `{"candidates":[{"content":{"parts":[{"text":""}],"role":"model"},"finishReason":"STOP","index":0,"safetyRatings":[…]}]}`. With the preset's `stream: true` the output is OpenAI-style chunks, not Gemini's stream format. Only the preset setting decides; a `stream` field in the body is ignored. **Stop reasons**: a reply produced by a response rule with `stopReason` ends with the value below, streamed or not. Every other reply ends as shown above. | `stopReason` | `claude` | `openai` and custom names | `gemini` | |---|---|---|---| | `end` (default) | `stop_reason: "end_turn"` | `finish_reason: "stop"` | `finishReason: "STOP"` | | `max_tokens` | `stop_reason: "max_tokens"` | `finish_reason: "length"` | `finishReason: "MAX_TOKENS"` | | `refusal` | `stop_reason: "refusal"` | `finish_reason: "content_filter"` | `finishReason: "SAFETY"` | - Streamed: `claude` carries it in the `message_delta` event's `delta`; the OpenAI-style stream in the final chunk's `finish_reason`. A streamed `gemini` or custom reply is OpenAI-style, so it gets the `openai` value. - `claude` with `refusal` also sends Anthropic's `stop_details`: `{"type":"refusal","category":null,"explanation":null}`, top-level in the message and inside `delta` of `message_delta`. `category` and `explanation` are always `null` (the API's value for a refusal with no named policy area). No other reply has a `stop_details` key (the real API sends `stop_details: null` there). - `tool_use` (`stop_reason: "tool_use"`) is only for a claude tool-call rule, and is what such a rule ends with whether or not it sets it (see Tool calls). - With `end` or no `stopReason` the response template's own value is kept, so a project template that sets another one still wins. A custom template with no `choices` list has nowhere to carry the value when not streamed. **Embeddings** (`POST /v1/embeddings`, body `{"model","input": string | string[], "dimensions"?}`): `{"object":"list","data":[{"object":"embedding","index":0,"embedding":[…]}],"model":"","usage":{"prompt_tokens","total_tokens"}}`. Vectors are deterministic for a given (`model`, input) pair, values in [-1, 1], not normalised and not semantic: similarity between them is meaningless. Request `dimensions` overrides the configured size. **Timing**: every reply waits a random `responseDelay` first. Streams then add at least 50 ms between events, even with delay `0`/`0`, so a streamed reply takes roughly 0.5 s minimum. ## Chaos (`chaos`) The way to test retries, backoff and error handling: some calls fail. By default a failing call gets an HTTP error in place of a reply; `chaos.kind` makes it a stream that fails part-way through instead. - Off unless `chaos.enabled` is true (or `--chaos=true`). Example: `"chaos": { "enabled": true, "frequency": 3, "mode": "every", "status": 429 }` fails calls 3, 6, 9… with a 429. - Use `mode: "every"` for deterministic tests: the count starts at 0 when the server starts, so with `frequency: 2` the 1st call succeeds and the 2nd fails. `frequency: 1` fails every call. `mode: "random"` fails each call with probability 1/`frequency`. - One count is shared by the chat route (POST and GET) and `/v1/embeddings`. `/ping`, `/` and the `/ui-*` routes never fail and are not counted. A request that fails validation gets its usual `400` and is not counted. - The `responseDelay` wait happens first. With `kind: "http"` a failing call that asked for a stream gets the JSON error, not SSE. - Error body by preset `name` (`kind: "http"`): - `openai` and custom names: `{"error":{"message","type","param":null,"code"}}`; `type` is `rate_limit_error` (`code` `rate_limit_exceeded`) for 429, else `server_error` (`code` `null`). - `claude`: `{"type":"error","error":{"type","message"}}`; `type` is `rate_limit_error` (429), `overloaded_error` (529), else `api_error`. - `gemini`: `{"error":{"code":,"message","status"}}`; `status` is `RESOURCE_EXHAUSTED` (429), `UNAVAILABLE` (503), else `INTERNAL`. - `/v1/embeddings` always uses the `openai` shape. - Headers: every chaos error has `x-llmock-chaos: true`; 429, 503 and 529 also have `retry-after: 1`. - `chaos.kind` decides what a failing call gets. `frequency` and `mode` still decide which calls fail, with the same shared count: | `kind` | A failing call that asked for a stream | A failing call that did not | |---|---|---| | `http` (default) | The HTTP error above | The HTTP error above | | `stream-error` | Stream starts, `afterChunks` content deltas, then the in-stream error, then the response ends | The HTTP error above | | `stream-drop` | Stream starts, `afterChunks` content deltas, then the socket is destroyed | The socket is destroyed with no response | | `stream-stall` | Stream starts, `afterChunks` content deltas, then nothing; the connection stays open until the client disconnects | No response; the connection stays open until the client disconnects | - A stream that fails has status `200` and the header `x-llmock-chaos: true` (the headers go out before the failure, so there is no `retry-after`), sends its opening events and deltas as usual, and never sends its closing events: no `content_block_stop`, `message_delta` or `message_stop` for `claude`; no `finish_reason` chunk and no `data: [DONE]` for the OpenAI-style stream. - `afterChunks` (default 2) counts content deltas: `content_block_delta` events for `claude` (`text_delta` and, in a tool-call reply, `input_json_delta`, counted together across blocks; the block stop/start events between them are sent too), content chunks for the OpenAI-style stream. `0` fails straight after the opening events (`message_start` + `content_block_start`, or the role chunk). A reply with fewer deltas than `afterChunks` fails after its last delta. A reply is split into about 5 deltas, so use a fixture of several words to have something to cut. - The in-stream error of `stream-error`: - `claude`: the SSE event `event: error` with `data: {"type":"error","error":{"type","message"}}`. `type` follows `chaos.status` as in the HTTP error (429 `rate_limit_error`, 529 `overloaded_error`, else `api_error`); `message` is `llmock chaos: simulated error`. - OpenAI-style stream (`openai`, `gemini` and custom names): a `data:` line holding the `openai` error body above (also for `gemini`). - Whether a call "asked for a stream" follows the usual rule: the request's `stream: true` for `claude`; for `openai`, the request's boolean `stream` if it has one, else the preset's `stream` setting; the preset's setting for the others. `/v1/embeddings` never streams, so it takes the right-hand column. - `stream-stall` sends no keep-alive and never times out on the server side: set a timeout in the client under test. Stalled connections are dropped when the server stops, so `stop` and a `SIGTERM` still work. - In the config file, a `frequency` that is not an integer ≥ 1, a `mode` other than `every`/`random`, a `status` outside 400–599, an unknown `kind`, or an `afterChunks` that is not an integer ≥ 0 → server refuses to start. An invalid CLI flag value falls back to the default instead. - `GET /ui-meta` → `chaosStatus` (`ENABLED`/`DISABLED`), `chaosFrequency`, `chaosMode`, `chaosErrorStatus`, `chaosKind`, `chaosAfterChunks`, and `chaosInjected` (failures of any kind since startup). - Not simulated: a malformed HTTP response (for bad JSON inside the reply text, write a broken fixture), a reply that arrives late but whole (use `responseDelay`), and a failure on only some of the calls that match a prompt. To fail every call that matches a prompt, and no other, use a response rule with `fail` (see Fixture replies). ## Tool calls (`content`, claude format only) The way to test a manual tool loop (call the model, run its `tool_use` blocks, send `tool_result` blocks back, repeat until `end_turn`) against the mock. A response rule replies with content blocks in place of text: ```json { "match": "Plan a picnic in Paris", "in": "last", "content": [ { "type": "text", "text": "Let me check the weather first." }, { "type": "tool_use", "name": "get_weather", "input": { "city": "Paris" } } ] } ``` - `content`: at most one `{"type":"text","text"}` block, only as the first block; then one or more `{"type":"tool_use","name","input"?,"id"?}` blocks, returned in the order given (several = parallel calls). No other block types or keys. - `name`: non-empty string, required. `input`: a JSON object, `{}` if omitted. `id`: optional non-empty string. A tool_use with no `id` gets a new `toolu_01` + 22 alphanumerics on every reply (unique per block and reply); a given `id` is sent as is every time. - `contentFile` in place of `content`: path (relative to the config folder) of a JSON file holding the same array. Re-read on every request. Missing → startup warning, `500` when matched; present but not valid JSON or not valid blocks → server refuses to start (`500` if it turns invalid while running). Not allowed through the Admin API. - The reply ends with `stop_reason: "tool_use"`, no `stop_details`. `stopReason` may be omitted or `"tool_use"`; any other value, `"tool_use"` on a rule without tool calls, more than one of `text`/`file`/`files`/`content`/`contentFile`, or a `content`/`contentFile` rule on a preset whose `name` is not `claude` → server refuses to start (Admin API: `400`). - `fail` works with `content` (a `stream-*` kind streams the blocks before failing) as with `text`. Wire format, not streamed: the usual claude message with `content` = the blocks (text block first if any; every `tool_use` with `id`, `name`, `input`), `stop_reason: "tool_use"`, `stop_sequence: null`, `usage.output_tokens` estimated from the blocks' JSON. Streamed (`"stream": true`): ``` message_start (content [], stop_reason null) content_block_start index 0 {"type":"text","text":""} ← only with a text block content_block_delta index 0 {"type":"text_delta","text":"…"} × n content_block_stop index 0 content_block_start index 1 {"type":"tool_use","id":"toolu_01…","name":"get_weather","input":{}} content_block_delta index 1 {"type":"input_json_delta","partial_json":"…"} × 1–3 (join to JSON.stringify(input); {} is one "{}" delta) content_block_stop index 1 … (each further tool_use at the next index) message_delta {"stop_reason":"tool_use","stop_sequence":null}, usage.output_tokens message_stop ``` The Anthropic TypeScript SDK's `client.messages.stream(…).finalMessage()` rebuilds the same blocks as the non-streamed reply (checked by a test against `@anthropic-ai/sdk`), and its `inputJson` event gets the `partial_json` pieces. Chaos and `fail` count `input_json_delta` events as deltas for `afterChunks` (see Chaos). Worked two-step loop (config rules, keyed on each step's own last message): ```json "responseRules": [ { "match": "Plan a picnic in Paris", "in": "last", "content": [ { "type": "tool_use", "name": "get_weather", "input": { "city": "Paris" } } ] }, { "match": "sunny", "in": "last", "text": "Take the picnic to the park." } ] ``` 1. `messages: [user "Plan a picnic in Paris"]` → `tool_use` `get_weather`, `stop_reason: "tool_use"`. 2. `messages: [user …, assistant [tool_use], user [tool_result "sunny, 24C"]]` → the last message holds `sunny`, so rule 2 answers: text, `end_turn`. Rule 1 does not fire again: its `match` is only in an earlier turn. Without `"in": "last"`, rule 1 would also match step 2 (the question is still in the body), and a `match` on a tool name would match every request (the `tools` list is in every body). Same loop scripted from a test with runtime rules that each answer once (order between steps does not matter, since each keys on its own last message): ```js await fetch(`${mock}/admin/reset`, { method: 'POST' }); await addRule({ match: 'Plan a picnic in Paris', in: 'last', times: 1, content: [{ type: 'tool_use', name: 'get_weather', input: { city: 'Paris' } }] }); await addRule({ match: 'sunny', in: 'last', times: 1, text: 'Take the picnic to the park.' }); // run the agent; it makes two calls and ends with that text. A third call // gets the normal responseType reply, since both rules removed themselves. ``` ## Recording (`recordings`, `--record`) Record real replies once, replay them offline after. - With `recordings.upstream` (or `--record=`): a POST to the chat route or the Responses route that no rule matches (runtime, config or recorded) and that chaos does not fail is sent on to `` with the call's headers (credentials included; `host`, `content-length`, `connection`, `accept-encoding`, `transfer-encoding` dropped) and body, with `stream: true` changed to `false` and `stream_options` removed. The provider is never asked to stream. - Provider path by format, whatever the preset's `endpoint`: claude `/v1/messages`; openai `/v1/chat/completions` (stock `chatgpt/chat/completions` included); Responses `/v1/responses` (also from `/responses`); gemini `/v1beta/models/:generateContent`, model taken from `models/:` in the call's path. Custom templates keep the call's own path. If `upstream` already ends with the path's version segment (`https://api.openai.com/v1`), it isn't repeated; any other path on `upstream` is kept as a prefix. - 2xx with a readable reply → saved, then answered in llmock's own format (streamed if the call asked), the same as every later replay. Saved: `/NNNN-.txt` (reply text, verbatim) or `/NNNN-.json` (claude tool-call blocks as a `contentFile`: one leading text block made of all text blocks, then `tool_use` blocks with `name` and `input`, no `id`; thinking blocks dropped), and a rule appended to `/rules.json`: `{"match": , "in": "last", "exact": true, "file"|"contentFile": , "stopReason"?}`. `stopReason` from claude `stop_reason` `max_tokens`/`refusal`, openai `finish_reason` `length` → `max_tokens`, `content_filter` or `message.refusal` → `refusal`, Responses `status: "incomplete"` → `max_tokens` and a `refusal` part → `refusal`, gemini `MAX_TOKENS`/`SAFETY`. - Not saved, sent back as the provider sent it (status, content-type, body), with a `RECORDING: not saved ()` warning: non-2xx, not JSON, a format with nothing to read (custom templates), openai `tool_calls`, Responses `function_call` items, Gemini `functionCall` parts, or a call whose last message has no text. Exception: if the call streams (as the route decides it) and the provider's status is 2xx, the client gets `502 {"error":{"type":"llmock_upstream","message"}}` (message includes the reason) instead of the JSON body; non-2xx still passes through as it came. Upstream unreachable → `502 {"error":{"type":"llmock_upstream","message"}}`. - Match key: the last message's strings joined by `\n` (`input` itself for a string Responses `input`), as `in: "last"` sees them (ids left out, so a recorded tool loop's next turn matches although the replayed tool call has a new id). Recorded rules have `exact: true`: a recording of `Hi` does not answer `Hi there`. - `/rules.json` is read on every request and matched after the config's `responseRules`; with `recordings.dir` set and no `upstream` the server only replays. Invalid JSON or an invalid rule in it → server refuses to start; if it turns invalid while running, the recorded rules are skipped (one `WARNING` per distinct problem) until it is fixed, everything else keeps working, and a reply recorded meanwhile is passed through, not saved (the file is never written over). Fixture numbers are one more than both the rule count and the highest `NNNN-` already in the folder, so a fixture is never overwritten. A missing fixture → startup warning. An `upstream` that is not an http(s) URL, or that holds a user name or password (`fetch` can not send to such a URL), → refuses to start. - `GET /admin/rules` / `GET /ui-meta` list recorded rules after the config ones with `source: "recorded"` and `id: null`; `/ui-rule-file?rule=` counts config rules then recorded ones. Recorded requests have `forwarded: {"status", "saved"}` for a call that was sent on (`null` otherwise); with logging on, that call's request log entry gets the same as `request_validation.forwarded`. `GET /ui-meta` has `recording: {"mode": "record"|"replay"|"off", "paused", "upstream", "dir", "forwarded", "saved"}` (counts since the server started or the last `POST /admin/reset`; each `startServer` starts at 0). - Pause: `PATCH /admin/recording {"paused": true}` (or the dashboard switch, admin API on). While paused, `mode` stays `record` with `paused: true`; a call no rule matches gets the normal generated reply (nothing sent on, nothing saved, no `forwarded`); recorded rules still answer. Resumed by `{"paused": false}`, `POST /admin/reset`, a restart or a new `startServer`. - Headers are never written to disk. The fixtures hold real prompts and replies. ## Custom formats (templates) To change a response envelope or the required request keys, or to mock a provider with another JSON shape, add templates beside `.llmockrc.json`: - `request-templates/_req.json`: `[ { …example request… } ]`. Its top-level keys become the required keys. - `response-templates/_res.json`: `[ { …response… } ]`. Every string value equal to `DYNAMIC_CONTENT_HERE` is replaced by the reply text. `` is the preset's `name`. Both files are a JSON array whose first element is used. A project file overrides the built-in one of the same name (`openai`, `gemini`, `claude`); templates are read per request. A custom `name` needs the response template always, and the request template when `validateRequests` is true. Custom formats stream as OpenAI-style chunks. Scaffolded projects already contain editable copies of the built-in templates. In Docker, mount or copy both folders next to the config. ## Other routes | Route | Returns | |---|---| | `GET /ping` | `{"response":"server is running"}` | | `GET /ui-meta` | Active settings as JSON | | `GET /ui-stored-responses` | The stored responses pool in use, as JSON | | `GET /ui-rule-file?rule=&file=` | Contents of one response rule file, as JSON | | `GET /ui-request-log` | `{"file", "log"}`: `log` is an array of the most recent validated requests (the last `maxLoggedRequests`, 10 by default), newest first, each with why it passed/failed and its `request_validation.request_id` (older ones are dropped; `log` is `null` if none logged) | | `DELETE /ui-request-log` | Empties the request log; returns `{"file", "log": null}` | | `/admin/rules`, `/admin/chaos`, `/admin/delay`, `/admin/settings`, `/admin/recording`, `/admin/reset` | Change response rules, chaos, delay and reply settings, and pause recording, while running (see Admin API) | | `/admin/requests` | The calls received, for asserting what the app sent (see Recorded requests) | | `GET /` | HTML dashboard | | `GET /` | A generated reply with no validation and no rules (quick smoke test) | Every reply, streamed ones included, carries `x-llmock-request-id: `, the same id as that request's `request_id` in the request log. ## Seeing the calls in LLMSpy LLMSpy (npm `llmspy`, tested with 0.6.0, Node >= 26) is a local proxy at `http://127.0.0.1:8080` that records LLM calls and shows them in a web UI; agents can read them from its `GET /api/requests` (see its own llms.txt). Put it between the client and llmock: 1. `npx llmock start --model=claude --llmspy` (one server per preset, each on its own port). `--llmspy` (or `server.llmspy: true` in the config; `--llmspy=` when LLMSpy is not on 127.0.0.1:8080) changes nothing about the server. It prints the upstream entry and the client base URL, e.g. `LLMSPY_UPSTREAMS=llmock-claude=http://127.0.0.1:8001` and `http://127.0.0.1:8080/llmock-claude`. `GET /ui-meta` → `llmspy` has them as `{ name, upstream, entry, baseUrl, endpointUrl }` (`null` when off; `baseUrl` is `null` for gemini and custom formats: call `endpointUrl`). 2. Start LLMSpy with the entries joined by commas: `LLMSPY_UPSTREAMS=llmock-claude=http://127.0.0.1:8001,llmock-chatgpt-8002=http://127.0.0.1:8002 npx llmspy` (`npx llmspy --background` returns once it is ready; `npx llmspy stop` stops it). 3. Point the client at the printed base URL: `http://127.0.0.1:8080/llmock-claude` for `claude`, `http://127.0.0.1:8080/llmock-chatgpt-8002/chatgpt` for `chatgpt`. The upstream is the whole server, so `/v1/embeddings` etc. work under the same prefix. The upstream name is `llmock-`, plus `-` when the port is not 8001 (so two servers of one preset do not share a name: LLMSpy keeps only the last entry of a name). To match a call LLMSpy recorded to llmock's request log, use its `x-llmock-request-id` response header. LLMSpy in Docker reaches llmock at `host.docker.internal:`, not `127.0.0.1`. ## Limits to design around - CORS: only streamed replies carry `Access-Control-Allow-Origin`, and there is no `OPTIONS` handler. Browser code calling the mock cross-origin will fail preflight; call it from a server/test process or through your dev-server proxy. - Not mocked: tool/function calls in the `openai`, `gemini` and custom formats (and their `tool_calls`/`functionCall` stop reasons), server tools (`web_search` and the like), images, multiple choices (`n`), the stop reasons `stop_sequence` and `pause_turn` (and their equivalents in the other formats), Gemini's `streamGenerateContent`, `/v1/models`, `/v1/messages/count_tokens`. Claude tool calls come only from rules: the mock never picks a tool itself, and it does not check `tool_use_id`s against earlier turns. ## Recipes Test setup (Vitest or another TypeScript runner), a server per test file on a free port: ```ts import { startServer, type RunningServer } from 'llmock'; let server: RunningServer; beforeAll(async () => { server = await startServer({ config: '.llmockrc.json', model: 'claude', port: 0, settings: { delayMin: 0, delayMax: 0 } }); }); afterAll(() => server.close()); // client baseURL: server.url ``` Test setup (any runner): ```js import { execSync } from 'node:child_process'; export function startMock() { // Returns once the server answers; throws if it could not start execSync('npx llmock start --model=claude --port=8001', { stdio: 'inherit' }); } export const stopMock = () => execSync('npx llmock stop --port=8001'); ``` Fast deterministic tests: `"responseDelay": {"min": 0, "max": 0}`, one `responseRules` entry per scenario keyed on a unique marker string in the prompt. Or keep each scenario's reply next to its test: `POST /admin/rules` in the test, `POST /admin/reset` in `beforeEach` (see Admin API). Retry tests: `"chaos": {"enabled": true, "frequency": 2, "status": 429}` makes every 2nd call fail, so a client that retries once always succeeds. Run it as its own preset (or with `--chaos=true`) so other tests are unaffected, or switch it on for one test with `PATCH /admin/chaos` and `POST /admin/reset` afterwards (see Admin API). Mid-stream failure tests: `"chaos": {"enabled": true, "frequency": 1, "kind": "stream-error", "afterChunks": 1}` on a `claude` preset (or `--chaos=true --chaosKind=stream-error --chaosAfterChunks=1`) makes every streamed call start, send one delta, then fail. With the Anthropic SDK the stream throws: ```js const stream = client.messages.stream({ model: 'claude-opus-5-5', max_tokens: 256, messages }); await expect(stream.finalMessage()).rejects.toThrow(); // an APIError carrying the overloaded_error body ``` Use `stream-drop` to test a connection cut with no error event, and `stream-stall` (with a short timeout on the client) to test a reply that hangs. Set `frequency: 2` to let a client that retries once succeed. Keep the SDK's own retries in mind (`maxRetries: 0` to see the first failure). Failing one prompt and not the rest: `"responseRules": [{ "match": "", "fail": { "kind": "stream-drop" } }]`. No `chaos` needed, and the tests around it do not depend on call order. Tool loops (claude): one rule per step with `"in": "last"`, keyed on a string only that step's last message has (the question, then each `tool_result`), the steps that call tools with `content`; or the same rules added per test with `POST /admin/rules` and `times: 1` (see Tool calls). Assert on the tool names and inputs your loop runs, and on the final text. Truncated or refused replies: a rule with `"stopReason": "max_tokens"` (or `"refusal"`) and a fixture that reads like one, then assert your code checks the stop reason before using the text. Varied replies: `"responseType": "stored"` with `"storedResponsesFile": "fixtures/replies.json"` for every unmatched request, or a rule with `files` for one kind of request. Assert on membership of the pool, not on one exact text. Docker (config and fixtures mounted, no rebuild on change): ```dockerfile FROM node:26-alpine WORKDIR /app RUN npm install --omit=dev llmock EXPOSE 8001 CMD ["npx", "llmock", "start", "--foreground"] ``` Mount `.llmockrc.json` to `/app/.llmockrc.json`, with the fixtures folder (and any template folders) beside it. A `storedResponsesFile` must be mounted too, at the same path relative to the config (simplest: keep it inside the fixtures folder). `--foreground` is required or the container exits. Publish the port as `127.0.0.1:8001:8001` (`-p` or compose `ports`) unless other machines need the mock: `8001:8001` opens the admin API and the request log to the network. Containers in the same compose file reach it by service name either way.