Skip to content

Add AsyncPriorityQueue. #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yanxi-chen merged 6 commits into from
Jul 11, 2025
Merged

Add AsyncPriorityQueue. #115

yanxi-chen merged 6 commits into from
Jul 11, 2025

Conversation

@chenyushuo
Copy link
Collaborator

@chenyushuo chenyushuo commented Jul 10, 2025 •
edited
Loading

Description

Add AsyncPriorityQueue to support replay buffer.

Checklist

Please check the following items before code is ready to be reviewed.

  • Code has passed all tests
  • Docstrings have been added/updated in Google Style
  • Documentation has been updated
  • Code is ready for review

@chenyushuo
Copy link
Collaborator Author

chenyushuo commented Jul 10, 2025

/run-unittest

@chenyushuo chenyushuo changed the title Add replay_buffer. Add AsyncPriorityQueue. Jul 10, 2025
@chenyushuo chenyushuo requested review from pan-x-c and yanxi-chen July 10, 2025 11:49
@github-actions
Copy link

github-actions bot commented Jul 10, 2025

Summary

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
49 49 0 0 0 0 1.3s

Tests

Test Name Status Flaky Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss ✅ 1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer ✅ 3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader ✅ 1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer ✅ 1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse ✅ 3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue ✅ 5ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue ✅ 4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity ✅ 4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer ✅ 4ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid ✅ 1ms
tests/common/config_test.py::TestConfig::test_load_default_config ✅ 4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion ✅ 1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion ✅ 1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate ✅ 41ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate ✅ 40ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate ✅ 50ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate ✅ 51ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate ✅ 38ms
tests/common/vllm_test.py::TestAPIServer::test_api ✅ 24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask ✅ 1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer ✅ 1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer ✅ 82ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer ✅ 99ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations ✅ 4ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results ✅ 19ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods ✅ 14ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop ✅ 8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks ✅ 7ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all ✅ 7ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch ✅ 13ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable ✅ 1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer ✅ 1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer ✅ 250ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer ✅ 88ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer ✅ 60ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer ✅ 87ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer ✅ 42ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue ✅ 98ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue ✅ 97ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins ✅ 4ms

Github Test Reporter by CTRF 💚

@pan-x-c
Copy link
Collaborator

pan-x-c commented Jul 11, 2025

/run-unittest

@github-actions
Copy link

github-actions bot commented Jul 11, 2025

Summary

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
50 50 0 0 0 0 1.3s

Tests

Test Name Status Flaky Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss ✅ 1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss ✅ 1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer ✅ 3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader ✅ 1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer ✅ 1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse ✅ 7ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity ✅ 2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue ✅ 5ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue ✅ 4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity ✅ 4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer ✅ 4ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid ✅ 1ms
tests/common/config_test.py::TestConfig::test_load_default_config ✅ 5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion ✅ 1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion ✅ 1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate ✅ 41ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate ✅ 40ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate ✅ 51ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate ✅ 51ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate ✅ 38ms
tests/common/vllm_test.py::TestAPIServer::test_api ✅ 24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask ✅ 1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer ✅ 1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer ✅ 90ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer ✅ 82ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations ✅ 4ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results ✅ 19ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods ✅ 14ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop ✅ 8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks ✅ 7ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all ✅ 7ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch ✅ 13ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow ✅ 1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable ✅ 1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer ✅ 1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer ✅ 245ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer ✅ 95ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer ✅ 61ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer ✅ 126ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer ✅ 43ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue ✅ 84ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue ✅ 94ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins ✅ 4ms

Github Test Reporter by CTRF 💚

pan-x-c
pan-x-c approved these changes Jul 11, 2025
View reviewed changes
Copy link
Collaborator

@pan-x-c pan-x-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

yanxi-chen
yanxi-chen approved these changes Jul 11, 2025
View reviewed changes
Copy link
Collaborator

@yanxi-chen yanxi-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yanxi-chen yanxi-chen merged commit d1c56dc into modelscope:main Jul 11, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pan-x-c pan-x-c pan-x-c approved these changes

@yanxi-chen yanxi-chen yanxi-chen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@chenyushuo @pan-x-c @yanxi-chen