Fix openai api history

[pan-x-c]

Description

As the title says

Checklist

Please check the following items before code is ready to be reviewed.

• Code has passed all tests
• Docstrings have been added/updated in Google Style
• Documentation has been updated
• Code is ready for review

[gemini-code-assist]

Summary of Changes

Hello @pan-x-c, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue related to the management of OpenAI API history within the system's workflow execution. It introduces a dedicated asynchronous workflow class and a new test case to thoroughly validate the correct handling of history, particularly when dealing with exceptions and the clearing of workflow states. The core fix ensures that the model's internal history is properly reset when the workflow state is cleaned, enhancing data integrity and predictability.

Highlights

• New API Workflow Test Class: Introduced a new APIWorkflow class within the test suite. This class is designed to simulate asynchronous interactions with the OpenAI API, including creating chat completions, handling intentional exceptions, and extracting experiences from the model's history.
• OpenAI History Test Case: Added test_workflow_with_openai to TestWorkflowRunner. This comprehensive test case validates the end-to-end management of OpenAI API history, covering scenarios like exception handling during task execution and ensuring proper experience extraction and history clearing mechanisms are in place.
• History Clearing Fix: Modified the clean_workflow_state method in trinity/common/models/model.py to explicitly clear self.history when the workflow state is reset. This ensures that historical data is properly purged, preventing potential state leakage or incorrect data persistence across workflow runs.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[pan-x-c]

/unittest-module-explorer

[gemini-code-assist]

Code Review

This pull request addresses an issue with OpenAI API call history not being cleared correctly between task runs. The fix in trinity/common/models/model.py to clear the history within clean_workflow_state is correct and well-tested by the new test case in tests/explorer/workflow_test.py. I have one suggestion to improve the clarity of the new test code.

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
46	45	0	1	0	0	14m 47s

Skipped

Tests	Status
tests/explorer/workflow_test.py::TestAgentScopeWorkflowAdapter::test_adapter	skipped ⏭️

Tests

Test Name	Status	Flaky	Duration
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅		1m 32s
tests/explorer/explorer_test.py::TestExplorerGSM8KRULERNoEval::test_explorer	✅		1m 39s
tests/explorer/explorer_test.py::TestExplorerGSM8k::test_explorer	✅		3m 38s
tests/explorer/explorer_test.py::ServeTest::test_serve	✅		1m 23s
tests/explorer/scheduler_test.py::SchedulerTest::test_async_workflow	✅		12.6s
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅		12.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_dynamic_timeout	✅		20.4s
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅		27.6s
tests/explorer/scheduler_test.py::SchedulerTest::test_metric_calculation_with_non_repeatable_workflow_0	✅		12.5s
tests/explorer/scheduler_test.py::SchedulerTest::test_metric_calculation_with_non_repeatable_workflow_1	✅		12.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_metric_calculation_with_repeatable_workflow_0	✅		12.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_metric_calculation_with_repeatable_workflow_1	✅		12.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_multi_step_execution	✅		12.7s
tests/explorer/scheduler_test.py::SchedulerTest::test_non_repeatable_workflow	✅		12.3s
tests/explorer/scheduler_test.py::SchedulerTest::test_over_rollout_min_wait	✅		16.5s
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅		22.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅		23.2s
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅		15.7s
tests/explorer/scheduler_test.py::SchedulerTest::test_stepwise_experience_eid	✅		32.5s
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅		15.5s
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅		20.8s
tests/explorer/scheduler_test.py::TestRunnerStateCollection::test_runner_state_collection	✅		17.5s
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_reward_propagation_workflow_0	✅		2ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_reward_propagation_workflow_1	✅		602ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_step_wise_reward_workflow_0	✅		1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_step_wise_reward_workflow_1	✅		1.0s
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_workflows_raise_error	✅		1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_workflows_stop_at_max_env_steps	✅		1.0s
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅		39ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅		28ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅		605ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_eval_workflow	✅		4ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅		14ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅		9ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅		131ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_repeatable_0	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_repeatable_1	✅		101ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable_0	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable_1	✅		201ms
tests/explorer/workflow_test.py::MultiTurnWorkflowTest_0::test_multi_turn_workflow	✅		14.8s
tests/explorer/workflow_test.py::MultiTurnWorkflowTest_1::test_multi_turn_workflow	✅		14.7s
tests/explorer/workflow_test.py::TestWorkflowStateRecording::test_workflow_state_recording	✅		4.0s
tests/explorer/workflow_test.py::TestAgentScopeWorkflowAdapter::test_adapter	⏭️		1ms
tests/explorer/workflow_test.py::TestWorkflowRunner::test_workflow_runner	✅		300ms
tests/explorer/workflow_test.py::TestWorkflowRunner::test_workflow_runner_get_state	✅		8.1s
tests/explorer/workflow_test.py::TestWorkflowRunner::test_workflow_with_openai	✅		18.4s

Github Test Reporter by CTRF 💚