Repository navigation
Conversation
|
@m-messer Do you plan to actively use this? |
|
@maximiliansoelch Of course, here is the shimmy PR on using SSE: lambda-feedback/shimmy#28 In particular, take a look at: https://github.com/lambda-feedback/shimmy/pull/28/changes#diff-1ee1003a66dcdc217b95324d29e9959e0795e3f021bb886f9b2bf26c149ef59e and the files in the I am happy to answer any questions you have on the implementation. |
maximiliansoelch
left a comment
There was a problem hiding this comment.
The overall approach looks good to me 👍
A shared, opt-in SSE progress stream, especially for /chat, is a useful addition.
I left some inline comments on the wire-format examples, event names, terminal payloads, and response selection.
The PR also currently has a merge conflict.
| model: "gpt-5.2" | ||
| temperature: 0.5 | ||
| outputTokens: 143 | ||
| text/event-stream: |
There was a problem hiding this comment.
The schema and examples under text/event-stream appear to describe the JSON in a terminal frame’s data: field, rather than the complete SSE response. That response may also contain progress events and comments, so the framing is missing from the media-type example. Is the intent to describe the full response here as SSE text and document the JSON frame payloads separately?
This also applies to /evalaute.
There was a problem hiding this comment.
Updated to OpenAPI 3.2, which has support for SSE using itemSchema
| MUST tolerate values they do not recognise. Each service advertises the | ||
| stage values it emits via `supportedProgressStages` in its `/health` | ||
| capabilities. Common values: "preparing", "starting", "evaluating" | ||
| (evaluate), "thinking" (chat), and the terminal "completed" / "failed". |
There was a problem hiding this comment.
It is unclear whether completed and failed are progress stages or only terminal event names. They are listed here and in supportedProgressStages, while the operation descriptions use them for terminal frames with a different data: shape. If progress frames can use these names, clients need a way to distinguish the two payload types. If the names are terminal-only, the stage description and capability examples currently suggest otherwise.
There was a problem hiding this comment.
Removed completed and failed from the progress states and are reserved for terminal states.
|
|
||
| `steps` is always present: the ordered replay of every progress step emitted | ||
| during the stream. | ||
| allOf: |
There was a problem hiding this comment.
The description defines a completed payload with a result and a failed payload with an error, but the schema requires only steps. It therefore accepts { "steps": [] } and also a payload containing both feedback and error. If these outcomes are intended to be exclusive, separate JSON payload alternatives can express that rule, with the SSE event mapping stated in the description. feedback also repeats Feedback[] instead of referencing EvaluateResponse. The chat terminal schema similarly repeats its normal response fields.
There was a problem hiding this comment.
Reduced duplication between chat and evaluate, and split the payloads.
| `SseEvaluateTerminalFrame` whose `error` is an `ErrorResponse`. | ||
|
|
||
|
|
||
| Failures are always delivered as a terminal `failed` frame, never as an |
There was a problem hiding this comment.
The 200/failed rule seems to describe failures after the SSE stream opens, but “failures are always delivered” also reads as applying to validation or authorization failures detected beforehand. The operation still lists 400 and 403, leaving that boundary unclear. The response mode is also unspecified when a request includes both callbackUrl and Accept: text/event-stream: one may produce 202, while the other describes a 200 stream. What response-selection rule is intended for these cases?
Also applies to chat.yml.
There was a problem hiding this comment.
Errors detected before event streaming are sent as their normal status, and if callback and event streaming are both disabled a 400 error is returned.
Conflict resolutions: - chat-health.yml / evaluate-health.yml: take main (capabilities moved to /info in #34); port the streaming capability examples (supportsStreaming, supportedProgressStages) to chat-info.yml and evaluate-info.yml. - evaluate.yml: keep main's MATH matched-case JSON examples alongside the text/event-stream SSE content block. Follow-ups: - Point streaming docs (openapi.yml, SseProgressStep, LLMConfiguration) at /info instead of /health. - Fix /info examples to use supportedAPIVersions, matching the ChatCapabilities/EvaluateCapabilities schemas (was supportedVersions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oads - Bump to OpenAPI 3.2 and describe text/event-stream with `itemSchema` (per-event SseEvaluateEvent / SseChatEvent) plus raw-wire `serializedValue` examples including progress events and keep-alives. - Replace the terminal-frame schemas with exclusive payloads: SseEvaluateCompletedFrame (feedback: $ref EvaluateResponse), SseChatCompletedFrame (allOf ChatResponse), and a shared SseFailedFrame (required error). Completed frames forbid `error`; failed requires it. - Reserve `completed` / `failed` as terminal event names only; remove them from stage descriptions and supportedProgressStages examples. - Define an ordered response-selection rule: pre-stream errors use normal HTTP statuses; callbackUrl + Accept: text/event-stream -> 400; callbackUrl -> 202; streaming -> 200 SSE; otherwise 200 JSON. - Add the missing chat 202 Accepted response and ChatAcceptedResponse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the review! I've fixed the merge conflict and addressed your four comments. |
additionalProperties was set on the string enum `classification`, where it has no effect and triggers a Swagger Editor warning. Move it up to the `dataCategory` object, which is what it was meant to apply to. Closes #17 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Standardises an opt-in Server-Sent Events (SSE) progress-streaming response for
/evaluateand/chat, as proposed in #43.Previously the spec only documented
200 → application/json, whileChatCapabilities.supportsStreamingandLLMConfiguration.streamimplied streaming was in scope with no wire contract behind them. This defines that contract once, generically, so both endpoints stream identically and new endpoints get it for free.Contract:
Accept: text/event-stream. HTTP status stays200for the life of the stream, including failures.event:= stage name,data:=SseProgressStep), optional:keep-alive comments, and exactly one terminal frame.200body (event: completed) or anErrorResponseundererror(event: failed), plus asteps[]replay of every progress step.X-Request-Id/X-Api-Versionare sent once as response headers at stream open.configuration.llm.stream(token-level LLM streaming) is now explicitly documented as distinct from API-layer progress streaming.Key schema additions (shared, reused by both operations):
SseProgressStepstageis an informative string, not an enumSseTerminalStepssteps[]replay fragmentStreamingCapabilitiessupportsStreaming+supportedProgressStages, composed viaallOfintoChatCapabilitiesandEvaluateCapabilitiesSseEvaluateTerminalFrame/SseChatTerminalFramedata, reusing each operation's existing200bodyEvaluateResponse/evaluate200body extracted into a named schema so the terminal frame can reuse itChatCapabilities.supportsStreamingmoves into the shared fragment;EvaluateCapabilitiesgains streaming advertisement for the first time. Both/healthexamples now showsupportsStreaming+supportedProgressStages.Related issue
Closes #43
Scope
mainbranch, or from a fork if direct branch creation is not available.Validation
npm run lint— valid, no new warnings (8 pre-existing warnings unchanged)npm run bundle— succeeds; all new components resolveNotes for reviewers
text/event-streamcontent, new optional capability fields, description edits. The one refactor is/evaluate's200body moving from an inline array to the namedEvaluateResponsecomponent (semantically identical).SseChatTerminalFrame.outputusesanyOf: [Message, "null"]rather than the repo's usualtype: [object, "null"] + allOf: [$ref]idiom, because thefailed-frame example setsoutput: nulland must validate. Theerrorfields keep the existing repo idiom (no null example exercises them).shimmyreference implementation needs a matching PR — terminal-frame field names,/healthcapability output, its embedded spec copy, and stream tests.🤖 Generated with Claude Code