Streaming API Testing Guide
This guide provides manual testing steps for the streaming functionality of the mock LLM API.
Overview
The mock LLM API supports two response modes:
- Static Mode: Returns a single JSON response
- Streaming Mode: Returns OpenAI-style Server-Sent Events (SSE) stream
With the openai format the request picks the mode, as with the real API: "stream": true in the body streams and "stream": false does not. A request with no stream key (or a GET) uses the server setting (STREAM=true / false, or the preset's stream). The gemini format always uses the server setting.
Quick Setup
1. Enable Streaming Mode
Edit your .env file:
STREAM=trueOr use the provided streaming configuration:
cp .env.streaming .env2. Start the Server
npm run dev
# or for streaming-specific config:
npm run serve:streamingThe server will start on http://localhost:8001
Manual Testing
Static Mode Testing
- Send
"stream": false(or set STREAM=false and leavestreamout of the request) - Make a request:
curl -X POST http://localhost:8001/chatgpt/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Hello world"}],
"temperature": 1,
"n": 1,
"stream": false
}'- Expected Response: Single JSON object
{
"id": "chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe",
"object": "chat.completion",
"created": 1678485525,
"model": "gpt-4o",
"usage": {
"prompt_tokens": 11,
"completion_tokens": 9,
"total_tokens": 20
},
"choices": [{
"message": {
"role": "assistant",
"content": "Generated response content here..."
},
"finish_reason": "stop",
"index": 0
}]
}Streaming Mode Testing
- Send
"stream": true(or set STREAM=true and leavestreamout of the request) - Make the request:
curl -N -X POST http://localhost:8001/chatgpt/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Hello world"}],
"temperature": 1,
"n": 1,
"stream": true
}'- Expected Response: Multiple SSE events
data: {"id":"chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe-0","object":"chat.completion.chunk","created":1678485525,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe-1","object":"chat.completion.chunk","created":1678485525,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe-2","object":"chat.completion.chunk","created":1678485525,"model":"gpt-4o","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}
data: {"id":"chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe-final","object":"chat.completion.chunk","created":1678485525,"model":"gpt-4o","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]model echoes the request's model (the preset's model if the request has none) and usage is an estimate (about 4 characters per token). To get usage on a stream, send "stream_options": {"include_usage": true}, as with the OpenAI API: every chunk then carries "usage": null, and one more chunk with "choices": [] and the estimated usage comes just before data: [DONE]:
data: {"id":"chatcmpl-6sf37lXn5paUcuf8UaurpMIKRMsTe-usage","object":"chat.completion.chunk","created":1678485525,"model":"gpt-4o","choices":[],"usage":{"prompt_tokens":11,"completion_tokens":3,"total_tokens":14}}Response Headers Verification
Static Mode Headers
Content-Type: application/jsonStreaming Mode Headers
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Cache-ControlTesting Scenarios
1. Basic Functionality
- Verify static mode returns single JSON
- Verify streaming mode returns multiple SSE events
- Check that both modes use the same endpoint URL
2. Content Validation
- Compare content between static and streaming modes
- Ensure streaming chunks reconstruct the same content as static response
- Verify first chunk contains role information
- Verify final chunk has
finish_reason: "stop"and empty delta
3. Format Compliance
- Validate static response matches OpenAI chat.completion schema
- Validate streaming chunks match OpenAI chat.completion.chunk schema
- Ensure last line is always
data: [DONE]
4. Error Handling
- Test with invalid requests (should return 400 error in both modes)
- Test with missing required fields
- Verify error responses are not streamed
5. Performance Testing
- Test response delays with different
RESPONSE_DELAY_MIN/MAXvalues - Verify streaming delays are distributed across chunks
- Test with very long responses (multiple chunks)
Automated Testing
Unit Tests
npm run test:unitE2E Tests
# Test all endpoints including streaming
npm run test:e2e
# Test only streaming
npm run test:streamingCypress Tests
# Open Cypress UI
npm run cypress:open
# Run streaming tests specifically
npm run test:streamingTroubleshooting
Common Issues
No response received
- Check server is running:
curl http://localhost:8001 - Verify port in
.envmatchesSERVER_PORT - Check for console errors
- Check server is running:
Invalid request errors
- Ensure all required fields are present:
model,messages - Check request format matches
openai_req.jsontemplate - Verify
VALIDATE_REQUESTS=ONif validation is expected
- Ensure all required fields are present:
Streaming not working
- Check the request doesn't send
"stream": false(it wins over the server setting) - Confirm
STREAM=truein.envif the request has nostreamkey - Restart server after changing environment variables
- Check response headers include
text/event-stream
- Check the request doesn't send
Content differences between modes
- Both modes should generate the same base content
- Differences may occur due to random content generation
- Set
MOCK_LLM_RESPONSE_TYPE=storedfor consistent content
Debug Mode
Enable detailed logging:
DEBUG=*Request Logging
View detailed request information:
LOG_REQUESTS=ON
VALIDATE_REQUESTS=ONThen open the dashboard at http://localhost:8001 and click View request log
Integration Examples
JavaScript/Fetch
// Static mode
const response = await fetch('http://localhost:8001/chatgpt/chat/completions', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello' }],
temperature: 1,
n: 1,
stream: false
})
});
const result = await response.json();
console.log(result);
// Streaming mode
const response = await fetch('http://localhost:8001/chatgpt/chat/completions', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello' }],
temperature: 1,
n: 1,
stream: true
})
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
const lines = chunk.split('\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = line.slice(6);
if (data === '[DONE]') {
console.log('Stream completed');
return;
}
const parsed = JSON.parse(data);
console.log('Chunk:', parsed);
}
}
}Python
import requests
# Static mode
response = requests.post(
'http://localhost:8001/chatgpt/chat/completions',
json={
'model': 'gpt-4o',
'messages': [{'role': 'user', 'content': 'Hello'}],
'temperature': 1,
'n': 1,
'stream': False
}
)
print(response.json())
# Streaming mode
response = requests.post(
'http://localhost:8001/chatgpt/chat/completions',
json={
'model': 'gpt-4o',
'messages': [{'role': 'user', 'content': 'Hello'}],
'temperature': 1,
'n': 1,
'stream': True
},
stream=True
)
for line in response.iter_lines():
if line:
decoded = line.decode('utf-8')
if decoded.startswith('data: '):
data = decoded[6:] # Remove 'data: ' prefix
if data == '[DONE]':
print('Stream completed')
break
import json
chunk = json.loads(data)
print('Chunk:', chunk)