Refactor Buffer Reader

[pan-x-c]

Description

1. Unify the buffer reader interface
2. Support dynamic read batch size

Checklist

Please check the following items before code is ready to be reviewed.

• Code has passed all tests
• Docstrings have been added/updated in Google Style
• Documentation has been updated
• Code is ready for review

[pan-x-c]

/run-unittest

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
0	0	0	0	0	0	0	4ms

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Flaky	Duration

Github Test Reporter by CTRF 💚

[pan-x-c]

/run-unittest

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
27	27	0	0	0	0	0	872ms

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Flaky	Duration
tests/buffer/file_test.py::TestFileReader::test_file_reader	✅		1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅		4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅		2ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅		1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅		5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅		1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅		1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅		46ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅		44ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅		52ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅		54ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅		38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅		22ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅		1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅		1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅		118ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅		140ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅		19ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅		3ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅		1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅		1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅		1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅		308ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅		3ms

Github Test Reporter by CTRF 💚

[yanxi-chen]

lgtm