Fix azureblob multipart upload regression #930
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Separate multipart completion logic for azureblob and azureblob-sdk providers. The new azureblob-sdk validation path introduced in 2.9.0 inadvertently broke the legacy azureblob provider by validating part numbers against client requests, while azureblob uses synthetic part numbers (divided by 10000). Changes: - handleCompleteMultipartUpload: azureblob uses simple listMultipartUpload iteration, while azureblob-sdk validates and deduplicates parts from XML request - handleListParts: scope the part-number aggregation to azureblob only Fixes multipart upload failures for files >5.2MB with azureblob provider.
Owner
|
Thank you for your contribution @klaudworks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
azureblobprovider multipart upload behavior to pre-2.9.0 logic, fixes Uploading files >5.2mb to Azure Blob Storage fails since 2.9.0 release #929Problem
The
azureblob-sdkmultipart upload implementation (commit fb96699) broke the legacyazureblobprovider by validating part numbers against the client's CompleteMultipartUpload XML request.The
azureblobprovider uses synthetic part numbers (10_000 * partNumber + subPartNumber) to split large S3 parts into smaller Azure blocks. When the new validation logic looked up client-provided part numbers (1, 2, 3...) in a map containing Azure block numbers (10000, 10001...), no match was found, causingINVALID_PARTerrors.Changes
handleCompleteMultipartUpload: Restoreazureblobto its original simple iteration logic;azureblob-sdkretains the new XML parsing + validation pathhandleListParts: Scope the/10_000part-number aggregation toazureblobonly, allowingazureblob-sdkto use direct part numbers. This bug did not surface yet because usually this is not used.This ensures the new
azureblob-sdkimplementation does not affect the deprecatedazureblobprovider and theazureblobpart-number magic does not affect the newazureblob-sdkprovider.