Recording real replies
Instead of writing fixtures by hand, let LLMock record them: calls that no rule answers go to the real provider once, and its reply is saved as a fixture file and a response rule. From then on the same call is answered from the recording, with no network, no API key and no cost.
bash
# Record: send the calls no rule answers to Anthropic, and save the replies
llmock start --model=claude --record=https://api.anthropic.com
# Later, or in CI: replay only (the recordings folder is read without --record)
llmock start --model=claudeOr in the preset:
json
"claude": {
"name": "claude",
"endpoint": "v1/messages",
"recordings": { "dir": "recordings", "upstream": "https://api.anthropic.com" }
}While recording, give your client a real API key: the call is sent on with its headers, so the provider gets the key your client sent. Headers are never saved. For replaying, any key will do.
upstreamis the provider's base URL, such ashttps://api.anthropic.com,https://api.openai.comorhttps://generativelanguage.googleapis.com(a trailing/v1or/v1betais fine too). LLMock adds the provider's own path for the call, whatever the preset'sendpointis:/v1/messages(Claude),/v1/chat/completions(OpenAI, also from the stockchatgpt/chat/completions),/v1/responses(the Responses API) and/v1beta/models/<model>:generateContent(Gemini, with the model from the preset's endpoint). The call's query string is kept, and a path inupstream(a proxy's prefix) is kept in front. Anupstreamwith a user name or password in it (https://user:pass@...) stops the server from starting: give a proxy its login in a header your client sends.dir(defaultrecordings, relative to the config file) getsrules.jsonand one fixture per recording:0001-what-is-the-capital-of-france.txt, or a.jsoncontentFilefor a Claude tool-call reply. The rules are read on every request, after the config file's ownresponseRules, so you can edit, reorder or delete them, or move them into the config file. Withdirset and noupstream, LLMock only replays. If an edit leavesrules.jsonbroken while the server runs, the recorded rules are skipped (the server prints why) until you fix it, and nothing new is saved over it; a broken file stops the server from starting.- A recorded rule matches the call's whole last message, with
"in": "last"and"exact": true(see Response rules): a recording ofHianswersHi, notHi there. Ids such as atool_result'stool_use_idare left out, so the steps of a recorded tool loop replay even though each replayed tool call gets a new id. Removeexactfrom a rule to have it answer any last message that contains itsmatch. - A cut-off reply is saved with
"stopReason": "max_tokens"and a refused one with"stopReason": "refusal", so they replay the same way. A Claude tool-call reply is saved without its ids, so each replay gets new ones. Thinking blocks are left out. - LLMock always asks the provider for a whole reply, then answers in its own format, streamed if the call asked for a stream. The first call and every replay look the same to your app.
- An error status from the provider, or a reply LLMock can not read, is sent back as it came and not saved (the server prints why). OpenAI and Gemini tool (function) calls are not recorded. If the call asked for a stream, a successful reply that wasn't saved can't be sent as it came (it is one JSON body, not a stream), so the client gets a
502saying why instead. A provider that can not be reached gives a502. - Rules from the config file and the admin API still win, and chaos still fails calls before anything is sent on.
GET /admin/requestsshows which calls were sent on, asforwarded: { "status", "saved" }. So does the request log (asrequest_validation.forwarded), and the dashboard's log viewer marks those calls.- To stop sending calls on for a while without a restart, pause recording:
PATCH /admin/recordingwith{ "paused": true }, or the switch on the dashboard. While paused, a call no rule answers gets a normal mock reply and nothing is sent or saved; the recorded rules still answer.{ "paused": false },POST /admin/resetor a restart resumes it. - On the dashboard, the Recording card (Settings page) shows whether the server is recording, paused, replaying or neither, the upstream, the recordings folder, and how many calls were sent on and saved since it started or was reset. Recorded rules are listed last on the Response rules page, marked
recorded. - The recordings hold your real prompts and the provider's replies. Look at them before you commit them to a shared repository.