Add Step-wise Workflow #130
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Author
|
/run-unittest |
Summary
Failed Tests
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/run-unittest |
Summary
Failed Tests
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/run-unittest |
Summary
Tests
Github Test Reporter by CTRF 💚 |
hiyuchang
reviewed
Jul 22, 2025
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a comprehensive step-wise workflow system for fine-grained experience management and reward calculation. The main changes enhance the framework's capability to handle step-by-step task execution with improved experience tracking and grouping functionality.
- Introduces step-wise workflow base classes that decouple task execution from the framework and enable low-cost migration from OpenAI API usage
- Restructures the experience tracking system with a new EID (Experience ID) mechanism for better grouping and identification
- Refactors the workflow runner and scheduler to support optional experience collection and pre-processing through add strategies
Reviewed Changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| trinity/common/experience.py | Major refactoring with new EID class and enhanced Experience structure |
| trinity/common/workflows/step_wise_workflow.py | New step-wise workflow base classes for step-by-step reward calculation |
| trinity/explorer/workflow_runner.py | Modified to return experiences and support configurable experience collection |
| trinity/explorer/scheduler.py | Updated to handle experience collection and return experiences alongside statuses |
| trinity/explorer/explorer.py | Enhanced with add strategy support and experience count tracking |
| trinity/algorithm/add_strategy/ | New add strategy system for pre-processing experiences before buffer storage |
Comments suppressed due to low confidence (1)
trinity/common/experience.py:2
- The module docstring describes "Workflow Runner Module" but this is the experience.py module. The docstring should be corrected to describe the experience module.
"""Experience Class."""
Collaborator
Author
|
/run-unittest |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-common |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-diff |
hiyuchang
approved these changes
Jul 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces a series of Step-wise Workflows to support step-wise reward calculations for tasks. Here are the main changes:
StepWiseRewardWorkflowandRewardPropagationWorkflowclasses as a base for all step-wise reward workflows, defining the basic workflow structure and reward calculation methods. The task execution (Agent application) part is completely decoupled from the framework, allowing users to directly use the OpenAI API to write applications with low migration costs.ExperienceStructure: The Experience structure now supports recording the current step of execution, facilitating grouping during training.WorkflowRunnerRefactoring: TheWorkflowRunnerno longer directly writes the Experience obtained from running the Workflow into theBuffer. Instead, it sends the results back to theExplorerfor aggregation and grouping before unified writing, thus supporting finer-grained management.AddStrategy: Explorer can pre-process the collected experiences before writing them into the experience buffer.Checklist
Please check the following items before code is ready to be reviewed.