Repository navigation
Text segmentation will vary by language #6
Description
segmentationResult: [TODO] Come up with a way to represent text segmentation.
Just a reminder that segmentation strategies can be very different across languages.
Some scripts don't separate words (at all), some do so with special wordspace characters, rather than spaces. Some don't have sentence punctuation, but separate phrases with gaps, or use punctuation in somewhat different ways, some not only don't separate words but also combine letters at the end+start of a word, etc.
So some flexibility will be needed, and it's really important to avoid the trap of relying on spaces to indicate segmentation boundaries.
Activity
wacky6 commentedon Nov 26, 2020
Here's my draft for segmentation result, please take a look and let me know what you think.
#7
In essence, it maps every point to a grapheme cluster (what's written on paper/screen). To my understanding, this is the minimal unit for editing (e.g. delete, replace, ...) a written character.
I don't think word segmentation is a goal for this API. It's a higher level linguistic concept.
r12a commentedon Nov 27, 2020
Hmm. I think you may have a problem here.
I guess the Balinese example we used earlier can also serve here (though, again, i don't need to reach far for much more complicated examples).
If you partition the drawing based on what's visually available you'll end up with things that aren't grapheme clusters.
The first glyph on the left is part of a circumgraph (ie. part of a single character, much less the grapheme cluster which is shown by the brown colour.
The black text is a grapheme cluster, but the combining part of that cluster is invisible, and what you actually see is identical to the letter KA when written on its own. The fact that this is a consonant cluster is indicated by the conjoined form of the following SA (middle glyph in brown), which doesn't actually carry anything in its grapheme cluster to indicate that there's an invisible character before it – that information is carried in its shape. If the segmentation were to treat the whole sequence as a unit, that wouldn't be a grapheme cluster either.
Grapheme cluster is actually a concept associated with the underlying code point sequence, rather than with the visual appearance, so i'm not sure it's the appropriate term to use here.
Off the top of my head, the somewhat vague term 'grapheme' might be more appropriate, since you could certainly segment the sequence into 4 graphemes. (You'd then need to apply some contextual rules to determine that the first and last graphemes map to a single code point for storage, and that the 3rd grapheme indicates the need for an invisible extra character after the consonant that precedes it.)
r12a commentedon Nov 27, 2020
There is another issue related to segmentation of visual text when the text is written in a 'cursive' script. The term 'cursive' is used here to refer to scripts that join adjacent characters to each other. The more general word cursive may also be relevant for handwritten text also, because hand written English tends to be joined-up writing too. Scripts such as Arabic, N'Ko, Adlam, Traditional Mongolian, etc. however are generally always written joined up, and typically use different glyph shapes depending on the visual context.
For example, the Arabic word for 'a little' looks like this:
rather than like the sequence of independent letters, which would be:
I don't know how character recognition is handled for Arabic, though i asked the question a couple of days ago in a Gulf States forum and the answer suggested to me that Arabic recognition is lagging behind. My guess is that the cursive nature is a contributing factor here. I imagine that similar difficulties face handwriting recognition.
I think that, short of putting strict restrictions on the user in the way they input the text, you'd either have to have a very large number of saved patterns for joined segments (with and without vowel signs, which are normally not written), or you'd need to figure out how to detect intercharacter boundaries.
Another thing that could be an issue here is that i was taught, when writing Arabic words, to write the core connected strokes in (mostly) one flow, and then go back and add the tops of the letters K and the dots that distinguish one letter from another. I don't know whether we should expect the user to complete each letter before moving on the the next one, or allow them to write more naturally – in which case, the recogniser would need to wait until a word is completed before interpreting the sequence of glyphs.
Here's an example. Basically the joined bits in black would be written first, and then the brown bits would be added afterwards.
--
If this were a W3C repository, i'd be able to add labels to these issues which would notify experts in various languages about these discussions, and maybe produce some contributions from them. It may be that the labels exist for this repository but that i don't have access rights to add them. I'll make inquiries about whether that can be changed.
wacky6 commentedon Dec 1, 2020
Updated the /pull/7 to use "grapheme".
Use the examples you gave, here's what we'd want:
As for the recognizer's implementation, not sure how ᬓ᭄ᬱᭀ is handled at the moment. But for cursive script, the recognizer do have the ability to tell each letters apart (even if their shapes changes).
As for the concern of natural flow of writing. The recognizer works perfectly fine for both cases:
Recognizer doesn't choose when it starts to interpret the handwriting. The application developer decides and will need to wait for completion.
The developers can get a prediction before that, in this case, the model may just give a wrong result. This is expected.
wacky6 commentedon Dec 8, 2021
Closing. Terminologies have been updated.