Skip to content

fix(firestore): hold and return fields in the official emulator's order - #149

Merged
nomunomu0504 merged 1 commit into
mainfrom
fix/108-field-order
Oct 11, 2026
Merged

nomunomu0504 merged 1 commit into
mainfrom
fix/108-field-order

Conversation

@nomunomu0504

Copy link
Copy Markdown
Contributor

Closes #108.

The official emulator returns a document's fields in the order it holds them, which depends on how they were written. hidane returned them in name order. This makes hidane hold and return the same order, everywhere.

What the official emulator does

tools/oracle/field_order.py records 34 cases against v1.22.0 (crates/hidane/tests/fixtures/field_order.json). Every observation follows these rules:

Operation Order
A write without a mask (set, create) The request's order, nested maps included. A write that changes nothing (equal values, in any order) leaves the document, its order and its updateTime as they were
A masked write The masked names are visited, per level of the path, in the order of a Java HashMap; those whose value changes go into a second HashMap, and in that map's order each is removed and added again at the end. A map changed inside moves the same way. Unchanged fields stay where they are
Transforms After the update, placed the same way, with the transform paths as the mask
Read masks, query projections The two-map order: masked names visited in one HashMap, the values found put into another
Aggregation results Request order
Export / import The document's order, empty arrays first; an import reads them back in file order

The HashMap order is Java's: String.hashCode over UTF-16, h ^ (h >>> 16), 16 buckets doubling past 3/4 load (or when a bucket gets a ninth key below 64 buckets), insertion order within a bucket. Using the two maps' own sizes is what makes zeta and str (same bucket of 16) swap when a mask names fields that are missing or unchanged.

How

  • hidane-proto: Document, MapValue and AggregationResult are written by hand in src/ordered.rs around Fields, an insertion-ordered map (IndexMap), and replace the generated messages through extern_path. The wire format is unchanged; entries are encoded in order and decoded through prost's own map decoding. A second descriptor set describes these three maps as lists of entries.
  • hidane-core: hash_order models Java's HashMap iteration; field_path::apply_mask and project implement the rules above. Comparisons (order.rs) and index keys (key.rs) still use UTF-8 key order. StoredDocument::has_fields compares in canonical order.
  • hidane: writes normalise input before deciding what changed; transforms compute their results in request order and then place the changed fields like a masked update. Aggregations keep request order. REST and WebChannel JSON go through the ordered descriptors (ordered_json.rs turns objects into entry lists and back), and serde_json has preserve_order.

Tests

  • tests/field_order.rs replays every recording over REST: status and error message, the field order of every document returned at every level (79 reads), and which commit updateTime names. A second test checks gRPC.
  • Unit tests for the Java order (hash_order.rs), the mask rules (field_path.rs) and the JSON round trip (ordered_json.rs), from the recorded cases.
  • export_import.rs now also checks that reading an official export gives the official import's order.
  • Docs: the three Return document fields in the order they were written, as the official emulator does #108 rows leave docs/parity-exceptions.md; docs/export-format.md and tools/oracle/README.md are updated.

Checks: cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace (165 passed), and cargo check with Rust 1.88.0.

🤖 Generated with Claude Code

The official emulator returns a document's fields in the order it holds
them, which depends on how they were written; hidane returned them in
name order. tools/oracle/field_order.py records 34 cases, and every
observation follows these rules (#108):

- A write without a mask keeps the request's order; one that changes
  nothing (values equal, order aside) leaves the document as it was.
- A masked write visits the masked names, per level of the path, in the
  order of a Java HashMap, puts those whose value changes into a second
  HashMap, and in that map's order removes each and adds it again at the
  end. A map changed inside moves the same way; unchanged fields stay.
- Transforms then place the fields they change the same way.
- Read masks and query projections return fields in that two-map order;
  aggregation results keep the request's order.
- Exports keep the document's order, empty arrays first, and imports
  read them back in file order.

To hold an order, Document, MapValue and AggregationResult are written
by hand (hidane-proto/src/ordered.rs) around Fields, an insertion-ordered
map, and replace the generated messages through extern_path; the wire
format is the same. REST and WebChannel JSON go through descriptors in
which those maps are lists of entries, and serde_json keeps object order.
Comparisons and index keys still use UTF-8 key order.

tests/field_order.rs replays the recordings over REST (79 reads, field
order at every level and which commit updateTime names) and checks gRPC.

Closes #108

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nomunomu0504
nomunomu0504 merged commit a894bf3 into main Oct 11, 2026
9 checks passed
@nomunomu0504
nomunomu0504 deleted the fix/108-field-order branch October 11, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Return document fields in the order they were written, as the official emulator does

1 participant