Skip to content

Add Python text and image prompt support - #524

Merged
robgruen merged 1 commit into
mainfrom
dev/robgruen/python-multimodal-517
Nov 6, 2026
Merged

robgruen merged 1 commit into
mainfrom
dev/robgruen/python-multimodal-517

Conversation

@robgruen

@robgruen robgruen commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Add backward-compatible Python text/image prompts through the existing Chat Completions model and translator-context interfaces, implementing the accepted P2 scope.

  • Export precise PromptContent, TextPromptContent, ImagePromptContent, and ImageUrl dictionary types while preserving existing string callers.
  • Share prompt normalization and structural validation across HTTP and configured-client models. Reject malformed parts before authentication or dispatch, without echoing prompt content. Preserve exact references, optional detail, ordering, and caller-owned data; images are supported only in user messages.
  • Preserve string-only translation requests, existing schema/request placement, repair roles, strict result validation, and rejected-output payloads. Keep completion results text-only with separate HTTP response typing.
  • Include the owner-approved optional temperature setting across model factories: retain 0.0 by default; None omits the parameter for reasoning-model compatibility.
  • Add deterministic offline behavior/static-type coverage and Python documentation examples. No dependency-manifest or lockfile changes, required provider SDK, image retrieval/processing, or Responses backend.

Fixes #517.

Verification

  • hatch run test on Python 3.11.18: 418 passed, 3 Python-version-dependent skips; 19 snapshots passed. Rerun before publication.
  • npm run check with Node 24.22.1: 0 errors, 0 warnings. Rerun before publication.
  • git diff --check: passed.
  • Live Azure Chat Completions smoke tests with identity authentication and temperature=None: 10 endpoint requests passed across the HTTP/token-provider model and an actual configured AsyncAzureOpenAI adapter. Covered strings, low/high-detail mixed image prompts, immediate typed image translation, and image-preserving repair with a locally injected invalid first candidate.
  • Ordinary regression tests remain offline and require no live credentials or provider SDKs.

Scope notes

The current prompt arrangement is unchanged; this does not adopt #241 or implement the separate Python Responses work in #518. Existing TypeScript image types and the contributed image example are unchanged.

Add typed prompt parts and shared validation for the HTTP and configured-client
Chat Completions paths, preserving string callers and image context during repair.
Include optional temperature configuration, regression tests, and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 06:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the backward-compatible Python text/image prompt changes.

No open issues 🎉

What changed in this PR

Adds backward-compatible Python text/image prompts to TypeChat’s existing Chat Completions models and translator context, addressing #517 without changing the prompt layout or adding a Responses backend.

Changes:

  • Exports typed prompt parts and shares structural validation across HTTP and configured-client models.
  • Adds optional temperature configuration, including omission with None, while keeping 0.0 as the default.
  • Adds offline regression tests, static-type coverage, and documentation for image prompts and repair behavior.
File Description
site/​src/​docs/​python/​basic-usage.md Documents image prompts, supported limits, and temperature configuration.
python/​tests/​test_prompt_types.py Checks public prompt types and documentation example consistency.
python/​tests/​test_multimodal_model.py Covers multimodal validation, forwarding, repair, and existing model controls.
python/​tests/​test_create_language_model.py Checks temperature forwarding across HTTP model factories.
python/​src/​typechat/​_internal/​model.py Implements prompt types, shared validation, temperature options, and text-only response checks.
python/​src/​typechat/​__init__.py Exports the new public prompt types.
python/​README.md Adds image-prompt examples and temperature guidance.

Review effort: Balanced

@robgruen
robgruen merged commit c2215ed into main Nov 6, 2026
13 checks passed
robgruen added a commit that referenced this pull request Nov 6, 2026
Integrate merged PR #524 with the explicit Responses API work. Preserve shared prompt validation, image references, repair context, and Chat Completions temperature settings. Map Responses assistant text to output_text, with deterministic and live regression verification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] Support multimodal text and image prompts

3 participants