[Question]: Document segmentation is not effective.

[zhangshenghang]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

I set the document parsing delimiter to ##

Data:

## A

Hello, hello, hello


## B

Hello, hello, hello

Result: It is not divided into two sections of data. In fact, A and B are still in one section.

[KevinHuSh]

The chunk is controlled by both delimiter and chunk token number.
Switching chunk token number to a very small number will archive your goal.