Root cause: errors from Google were being swallowed, replaced with
placeholders like 'Google API returned HTTP 400' or '[Timeout waiting
for response]', or silently converted to fake 'incomplete' responses.
Changes across all endpoints (/v1/chat/completions, /v1/responses,
/v1/gemini, /v1/search):
Error message fidelity:
- UpstreamError message now includes Google's status prefix: [STATUS] msg
- Falls back to raw body if JSON parsing fails (protobuf, HTML, etc.)
- ErrorDetail gains optional code and param fields
Timeout handling:
- poll_for_response returns UpstreamError(504, DEADLINE_EXCEEDED) on timeout
instead of '[Timeout waiting for AI response]' placeholder text
- Streaming timeouts emit proper error events, not fake content
- Sync bypass timeouts return 504 Gateway Timeout, not 200 incomplete
Missing error checks added:
- responses.rs sync bypass: added upstream_error check in polling loop
- gemini.rs sync bypass: added upstream_error check in polling loop
- gemini.rs streaming: added upstream_error check in polling loop
(was completely missing — errors only handled in sync path)
DRY helpers:
- upstream_error_message(): shared exact message extraction
- upstream_error_type(): shared Google→OpenAI error type mapping
- All streaming handlers use these instead of inline formatting
Root cause: proxy.rs eagerly pushed tool rounds via push_tool_round_calls
when intercepting Google's functionCall response. These stale rounds leaked
into LS follow-up requests, producing malformed history that Google timed
out on (60s 'no upstream response').
Changes:
- Remove push_tool_round_calls from proxy.rs response interception
- proxy.rs: use get_tool_rounds (non-destructive) instead of take_tool_rounds
so accumulated rounds persist across multiple LS requests per cascade
- responses.rs/gemini.rs: build rounds via take+push+set pattern — each
handler accumulates its own rounds from get_last_function_calls + results
- completions.rs: unchanged (set_tool_rounds replaces from messages)
- clear_tools: also clears tool_rounds to prevent stale data between sessions
- store.rs: add get_tool_rounds (non-destructive clone) method
- proxy.rs: push_tool_round_calls alongside set_last_function_calls
when Google responds with functionCall — accumulates rounds
- responses.rs: attach_tool_round_results to pair tool results with
the correct round instead of flat add_tool_result
- gemini.rs: same attach_tool_round_results integration
- store.rs: add push_tool_round_calls and attach_tool_round_results
methods for cross-request round accumulation
- Legacy add_tool_result kept for backward compat alongside new path
- store.rs: record_function_call now falls back to active_cascade_id
(matching record_usage behavior) instead of blind _latest fallback
- store.rs: add cascade-aware take_function_calls(cascade_id) method
with priority: exact match → active cascade → _latest → any key
- completions.rs: extract tool_calls from assistant messages and tool
results from tool messages, storing them for MITM injection. This was
the ROOT CAUSE — the completions handler stored tool definitions but
never extracted tool results, so modify_request couldn't rewrite the
LS conversation history with proper functionCall/functionResponse
- responses.rs: use cascade-aware take_function_calls for consistency
Move the in-flight blocking check to the top of the LLM request flow,
BEFORE request modification. This catches follow-ups on ALL connections
(the LS opens multiple parallel TLS connections). Only the very first
modified request reaches Google — all others get fake STOP responses.
Previously, each new connection independently allowed one request
through before blocking, letting 4-5 requests leak per turn.
When Google returns an error (400, 429, 500, etc.), the MITM proxy now
captures it and the API handlers return it immediately instead of
hanging until timeout.
- UpstreamError struct stored in MitmStore
- MITM proxy parses Google error JSON (message + status)
- Polling handler checks for upstream errors each cycle
- Streaming handlers emit response.failed / SSE error events
- Error status mapped to OpenAI-style types (invalid_request_error,
rate_limit_error, authentication_error, server_error, etc.)
- All handlers clear stale errors at request start
When input is [{type: 'input_image', ...}, {type: 'input_text', text: '...'}],
the code was looking for items with role: 'user' which don't exist in flat
content arrays. Now extracts text from input_text items directly first,
falling back to role-based messages only if no flat text found.
Also adds debug header dump for MITM request forwarding.
The LS silently ignores the 'images' field from our
SendUserCascadeMessageRequest proto — it never forwards image data
to Google's API.
New approach: store the image in MitmStore, then the MITM request
modifier injects it as 'inlineData' directly into the last user
message's parts array in the Google API JSON request.
Flow:
Client → Proxy (decode base64) → MitmStore.set_pending_image()
LS → Google API → MITM intercepts → inject inlineData part
→ Google receives image + text together
This works for all three API endpoints (responses, completions,
gemini).
- MitmStore: added active_cascade_id field with set/get/clear methods
- record_usage() now falls back to active_cascade_id when the heuristic
cascade hint is absent (fixes usage always going to _latest)
- All three API handlers set active cascade before send_message
- KNOWN_ISSUES: moved 3 issues to resolved:
- Request modification (already true, was stale entry)
- Cascade correlation (fixed via active_cascade_id)
- Progressive thinking streaming (fixed via MITM bypass)
- Responses API (streaming): MITM bypass path polls MitmStore directly
when custom tools are active, skipping LS step polling entirely.
Streams thinking text deltas in real-time as they arrive from the MITM.
Handles function calls, text response, and thinking/reasoning events.
- Responses API (sync): Same MITM bypass for non-streaming responses.
Polls MitmStore for function calls or completed text before falling
back to LS path.
- Gemini endpoint: MITM bypass polls MitmStore directly for tool call
responses, eliminating LS overhead.
- MitmStore: Added captured_thinking_text field with set/peek/take methods
for real-time thinking text capture from MITM SSE.
- MITM proxy: Now captures both thinking_text and response_text from
StreamingAccumulator into MitmStore when bypass mode is active.
- store.rs: Add tool context storage (active tools, tool config, pending
tool results, call_id mapping, last function calls for history rewrite)
- types.rs: Add tools/tool_choice fields to ResponsesRequest, add
build_function_call_output helper for OpenAI function_call output items
- modify.rs: Replace hardcoded get_weather with dynamic ToolContext
injection. Add openai_tools_to_gemini and openai_tool_choice_to_gemini
converters. Add conversation history rewriting for tool result turns
(replaces fake 'Tool call completed' model turn with real functionCall,
injects functionResponse before last user turn)
- proxy.rs: Build ToolContext from MitmStore before calling modify_request.
Save last_function_calls for history rewriting on subsequent turns
- responses.rs: Store client tools in MitmStore before LS call. Detect
function_call_output in input array for tool result submission. Return
captured functionCalls as OpenAI function_call output items with
generated call_ids and stringified arguments
- gemini.rs: New Gemini-native endpoint (POST /v1/gemini) with zero
format translation. Accepts functionDeclarations directly, returns
functionCall in Gemini format directly
- mod.rs: Wire /v1/gemini route, bump version to 3.3.0
When MITM strips LS tools and injects custom tools:
- Google returns functionCall → captured in MitmStore
- Follow-up LS requests are blocked with fake SSE response
- Proxy consumes captured calls and clears the flag
- Result: 1 real Google API call instead of 5+ per tool call
Flow: Client → Proxy → LS → MITM(inject tool) → Google
Google returns functionCall → MITM captures it
LS tries follow-up → MITM blocks (fake response)
Proxy reads captured functionCall → returns to client
- Subscribe to StreamCascadeReactiveUpdates for real-time cascade state diffs
- Fall back to timer-based polling if streaming RPC unavailable
- Remove StreamCascadePanelReactiveUpdates code (dead end, only has plan_status/user_settings)
- Remove debug diff file-saving code
- Add stream_reactive_rpc() helper to backend
Streaming poll: 800-1200ms → 150-250ms (5x faster)
Sync poll: 1000-1800ms → 200-400ms (4x faster)
Verified via STEP_DUMP instrumentation that the LS updates
plannerResponse.response incrementally during GENERATING status,
so faster polling yields smoother progressive text delivery.
Also restructured streaming to emit reasoning events first
when thinking content is detected in LS steps before response text.
Adds proper streaming SSE events for reasoning content:
- response.output_item.added (reasoning)
- response.reasoning_summary_part.added
- response.reasoning_summary_text.delta
- response.reasoning_summary_text.done
- response.reasoning_summary_part.done
- response.output_item.done (reasoning)
These are emitted before the message events, matching the format
that OpenAI-compatible clients expect for displaying thinking content.
The LS makes two Google API calls for thinking models. Call 2 (thinking
summary) may not have arrived by the time usage_from_poll runs after
Call 1 (response). Now we peek first, and if thinking tokens exist but
text is missing, wait up to 1s for the merge to happen.
Also adds peek_usage method to MitmStore for non-consuming reads.
The LS strips thinking/reasoning text from plannerResponse steps —
only the thinkingSignature (opaque verification blob) is preserved.
The actual thinking text flows through the MITM proxy in the raw
Google SSE response (parts with thought: true) and Anthropic SSE
(thinking_delta content blocks).
Changes:
- StreamingAccumulator now accumulates thinking text from SSE events
- ApiUsage gains thinking_text: Option<String>
- usage_from_poll returns (Usage, Option<thinking_text>)
- Thinking text priority: MITM-captured > LS-extracted (fallback)
- Reasoning output item now populated from real API data
- Removed debug dump code
Thinking content was previously returned as non-standard top-level
fields (thinking, thinking_duration). Now follows the official OpenAI
Responses API format:
- Reasoning appears as a 'type: reasoning' item in the output array
with summary[].text containing the thinking content
- Message item follows after the reasoning item
- thinking_signature kept as proxy extension (internal multi-turn data)
- Removed ResponseOutput/OutputContent structs in favor of
serde_json::Value for polymorphic output items
When the MITM can't extract a cascade ID from the intercepted request
(Content-Length: 0 / chunked encoding), usage is stored under '_latest'.
Now usage_from_poll and completions try the exact cascade_id first,
then fall back to '_latest' so MITM-captured tokens are actually used.
