Refactor advantage computation (cont.)

[yanxi-chen]

Description

As the title says.

Checklist

Please check the following items before code is ready to be reviewed.

• Code has passed all tests
• Docstrings have been added/updated in Google Style
• Documentation has been updated
• Code is ready for review

[yanxi-chen]

/run-unittest

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
27	27	0	0	0	0	0	1.3s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Flaky	Duration
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅		4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅		1ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅		1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅		4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅		1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅		1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅		44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅		42ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅		53ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅		53ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅		39ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅		22ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅		1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅		1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅		113ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅		122ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅		19ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅		3ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅		1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅		1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅		430ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅		130ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅		128ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅		116ms

Github Test Reporter by CTRF 💚

[yanxi-chen]

/run-unittest

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
27	27	0	0	0	0	0	1.3s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Flaky	Duration
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅		4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅		1ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅		1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅		4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅		1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅		1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅		44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅		43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅		52ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅		53ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅		39ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅		22ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅		1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅		1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅		106ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅		139ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅		19ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅		3ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅		1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅		1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅		419ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅		121ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅		128ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅		117ms

Github Test Reporter by CTRF 💚