Chunk oversized JSON string writes in JsonToBinaryStream#26911
Chunk oversized JSON string writes in JsonToBinaryStream#26911Amemoyoi wants to merge 5 commits intoprotocolbuffers:mainfrom
Conversation
|
Thank you for raising this! Serializing strings above 2 GiB is not supported by protobuf in general (our length-prefix format has an I think we likely would prefer to serialize-fail if such a condition is hit rather than trying to gracefully chunk up the writes: do you think you could update the PR to do? |
|
Thanks, that makes sense. |
|
Thanks again for the review. |
|
Unfortunately looks like theres a build breakage with latest proposed change, are you able to fix? Thanks! |
|
Thanks for the heads up. The breakage was from defining the helper in the header with external linkage. I’ve updated it to be inline and pushed the fix. |
This change avoids oversized raw writes in the proto3 JSON-to-binary streaming path.
Changes:
ParseProto3Type::SetString()instead of passing asize_tlength directly intoWriteRaw(const void*, int)INT_MAX + 1boundary