Skip to content

Together PR #3843 drops error spans when content generators raise #3884

Description

@caroger

PR #3843 moves content-generator materialization outside the Together LLM span lifecycle. A content generator that raises now produces no LLM span in either the sync or async wrapper.

Verified with Together 2.22.0, Python 3.12.12, the actual SDK transport pointed at a local HTTP fixture, and fresh Phoenix 20.16.0 instances ingesting OTLP/HTTP.

  • Base b738fcdf46a77b65d7275608404982fe9cf6ce2c: both calls raise the original RuntimeError, send zero HTTP requests, and export an ERROR Completions/AsyncCompletions child with an exception event.
  • Head 15977e03aa51d7ecc3d43f63a48e7120ed745345: the original RuntimeError still propagates and zero HTTP requests are sent, but both LLM children disappear. Only explicit enclosing CHAIN spans remain.

Minimal input, with TogetherInstrumentor enabled and an exporter configured:

def broken_content():
    yield {"type": "text", "text": "Describe the synthetic input."}
    raise RuntimeError("synthetic content failure")

client.chat.completions.create(
    model="fixture-model",
    messages=[{"role": "user", "content": broken_content()}],
)

The async call has the same result with await. The new calls to _materialize_content_iterables(kwargs) at _wrappers.py:118 and :154 execute the generator before _start_as_current_span and _finalize_error can handle it. Record the preprocessing error through an LLM span and re-raise the original exception. Add sync and async regression coverage with a generator that yields and then raises.

The PR's 17 package tests and 30 successful live scenarios passed, including multimodal content, tuples/generators, fully consumed streams, context, masking, and suppression. This issue concerns lost failure telemetry. The separate outer-messages generator path already has broken behavior on the base and is not attributed to this PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions