feat: expose provider request ids on STT/TTS/LLM spans for debugging#5546
feat: expose provider request ids on STT/TTS/LLM spans for debugging#5546
Conversation
Adds a new span attribute 'lk.provider_context_ids' (list[str]) populated from plugin-provided STT request_ids and TTS segment_ids, so users can cross-reference traces with provider-side logs when debugging. - TTS: AudioEmitter.start_segment appends non-empty segment_ids to a deduped list and sets the attribute on the current tts_request_run span. - STT: AudioRecognition collects SpeechEvent.request_ids across a turn and writes the full list on the user_turn span at span end.
- tts base: add AudioEmitter.note_provider_context_id() for plugins where the provider id arrives after start_segment() has been called. - sarvam TTS: feed the captured server request_id via note_provider_context_id once the first WS response arrives. - sarvam STT: fall back to the session-wide server request_id when a per-message request_id is missing. - openai STT (realtime): populate SpeechEvent.request_id from item_id; track current_item_id for interim transcripts. - upliftai TTS: pass segment_id as the request id to client.synthesize() so the id is actually known to the server.
LLMStream now tracks the deduped list of ChatChunk.id values seen during each attempt and writes them to the llm_request_run span as lk.provider_context_ids, matching the per-attempt scope used for TTS. The list is reset in _main_task at the start of each attempt, populated by the monitor task as chunks arrive, and read at the end of the attempt (before the span closes). No plugin changes needed — ChatChunk.id is already the provider's response id across all LLM plugins.
Now that the attribute covers STT (request_id), LLM (ChatChunk.id, also a request/response id), and TTS (context_id / session_id), 'context_ids' was only accurate for TTS. 'request_ids' matches the terminology STT and LLM providers actually use and reads naturally for TTS context_ids too. Renames: - ATTR_PROVIDER_CONTEXT_IDS -> ATTR_PROVIDER_REQUEST_IDS - lk.provider_context_ids -> lk.provider_request_ids - AudioEmitter.note_provider_context_id() -> AudioEmitter.note_provider_request_id() - _provider_context_ids / _stt_context_ids -> _provider_request_ids / _stt_request_ids
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if ev.request_id and ev.request_id not in self._stt_request_ids: | ||
| self._stt_request_ids.append(ev.request_id) |
There was a problem hiding this comment.
🟡 Sarvam STT's JSON-encoded RECOGNITION_USAGE request_id pollutes provider_request_ids span attribute
The new _on_stt_event collection code in audio_recognition.py:684 indiscriminately collects ev.request_id from all event types. The Sarvam STT plugin emits RECOGNITION_USAGE events with request_id=json.dumps(request_data) at stt.py:1334, where request_data is a dict containing original_id and processing_latency. This JSON string (e.g. {"original_id": "abc", "processing_latency": 0.5}) gets appended to _stt_request_ids and eventually written to the user_turn span as lk.provider_request_ids. This pollutes the provider request IDs attribute with non-ID metadata strings, undermining the debugging purpose of the feature.
Example of polluted span attribute
The span's lk.provider_request_ids would contain entries like:
["actual-server-request-id", "{\"original_id\": \"actual-server-request-id\", \"processing_latency\": 0.123}"]
instead of just: ["actual-server-request-id"]
Prompt for agents
The _on_stt_event method collects ev.request_id from ALL SpeechEvent types into _stt_request_ids. However, the Sarvam STT plugin (livekit-plugins/livekit-plugins-sarvam/livekit/plugins/sarvam/stt.py, around line 1334) sets request_id on RECOGNITION_USAGE events to json.dumps(request_data) which includes processing_latency metadata - not a clean provider ID. This JSON string gets collected alongside actual request IDs and written to the span.
Two possible fixes:
1. In audio_recognition.py _on_stt_event, skip collecting request_id from RECOGNITION_USAGE events (e.g. add a check: if ev.type != stt.SpeechEventType.RECOGNITION_USAGE).
2. In the Sarvam STT plugin, stop encoding metadata into the request_id field of RECOGNITION_USAGE events and instead use just the plain request_id string.
Was this helpful? React with 👍 or 👎 to provide feedback.
| finally: | ||
| if self._provider_request_ids: | ||
| attempt_span.set_attribute( | ||
| trace_types.ATTR_PROVIDER_REQUEST_IDS, self._provider_request_ids | ||
| ) |
There was a problem hiding this comment.
🟡 LLM _provider_request_ids may be incomplete when set on attempt span due to race with monitor task
In llm.py, _provider_request_ids is populated by the _metrics_monitor_task (line 294-295) which runs as a separate asyncio task, and read in the finally block of _main_task (line 228-231). When _run() returns, the finally block executes synchronously before the event loop can schedule the monitor task. Any ChatChunk events that were send_nowait'd by _run() after its last internal await but before returning will be buffered in the tee iterator and not yet processed by the monitor task. This means the attempt_span's lk.provider_request_ids attribute could be incomplete. In practice, since LLM request IDs are typically identical across all chunks, the first processed chunk captures the ID, making this unlikely to cause missing data — but it's architecturally fragile.
Was this helpful? React with 👍 or 👎 to provide feedback.
| self.note_provider_request_id(segment_id) | ||
| return self.__start_segment(segment_id=segment_id) | ||
|
|
||
| def note_provider_request_id(self, context_id: str) -> None: |
There was a problem hiding this comment.
| def note_provider_request_id(self, context_id: str) -> None: | |
| def _note_provider_request_id(self, context_id: str) -> None: |
3 participants
Adds a new span attribute
lk.provider_request_ids(list[str], deduped) on theuser_turn(STT),tts_request_run(TTS), andllm_request_run(LLM) spans for debugging/reporting STT or TTS issues to the provider.Supported plugins (id actually known to the provider)
For LLM, the id comes from
ChatChunk.id.TTS streaming (
tts_request_runattribute populated)note_provider_request_id()STT (
user_turnattribute populated when present)metadata["request_id"]session.request_iditem_idfrom completion event