You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Symbol uploads: batch many source maps per request (cross-repo) #7
Parking an idea we measured but deliberately did not build: pack every chunk's source map into one archive (or at least one registration) instead of one symbol document per map. It needs appserver + worker + CLI + plugin work, so it wants a deliberate cross-repo sweep.
Full write-up with the measurements: docs/design/symbol-upload-batching.md.
Where we are
One symbol document per map, two round-trips each (register + presigned PUT) — a 200-chunk web build is 400 requests. bugsee-cli PR #42 made them run up to 8 at a time, taking 200 maps from 23.58 s to 4.10 s against a mock with 50 ms of latency. That is the baseline any batching has to beat.
What we found
The upload wire already allows a multi-identity archive.presigned::Metadata.uuids[] is how a multi-slice dSYM and an IL2CPP multi-ABI line-map upload as one archive today.
The read side is the blocker. The worker's get_symbol_file picks file_list[0] when no filter is given, and the JS symbolication path passes none — so a 200-map archive would hand every frame the first map in the zip. Wrong line numbers, no error. The fix exists next door (the symcache path selects members by _same_debug_id) but it is worker work plus a storage-layout decision.
Three costs survive that fix: per-map dedup disappears (today a rebuild re-sends only changed chunks — we have an e2e flow pinning that), read amplification (every symbolication pulls the whole archive to read one map), and the byte win is modest — 60 real maps from this repo's dist: per-map zstd summed 0.62 MB vs one archive 0.54 MB, i.e. 13%.
Options
request count (200 maps)
keeps per-map dedup
read side
A. One ZIP per build
2
✗
needs member-by-debug-id + re-split/cache
B. Batch the registration only (N uuids → N presigned URLs)
201
✓
unchanged
C. Today
400, 8 in flight
✓
unchanged
B is the recommendation if we act: it removes the half of the round-trips that is pure latency, keeps per-uuid dedup and the current read path, and is an appserver + small CLI change with no worker work.
Scope when we pick this up
appserver — batch-register endpoint (B), or multi-uuid symbol documents for sourcemaps (A).
worker — only for A: member selection by debug-id, ingest-time re-split or per-uuid cache, member naming.
Activity