Repository navigation
fix(firestore): hold and return fields in the official emulator's order - #149
Merged
Merged
Conversation
The official emulator returns a document's fields in the order it holds them, which depends on how they were written; hidane returned them in name order. tools/oracle/field_order.py records 34 cases, and every observation follows these rules (#108): - A write without a mask keeps the request's order; one that changes nothing (values equal, order aside) leaves the document as it was. - A masked write visits the masked names, per level of the path, in the order of a Java HashMap, puts those whose value changes into a second HashMap, and in that map's order removes each and adds it again at the end. A map changed inside moves the same way; unchanged fields stay. - Transforms then place the fields they change the same way. - Read masks and query projections return fields in that two-map order; aggregation results keep the request's order. - Exports keep the document's order, empty arrays first, and imports read them back in file order. To hold an order, Document, MapValue and AggregationResult are written by hand (hidane-proto/src/ordered.rs) around Fields, an insertion-ordered map, and replace the generated messages through extern_path; the wire format is the same. REST and WebChannel JSON go through descriptors in which those maps are lists of entries, and serde_json keeps object order. Comparisons and index keys still use UTF-8 key order. tests/field_order.rs replays the recordings over REST (79 reads, field order at every level and which commit updateTime names) and checks gRPC. Closes #108 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #108.
The official emulator returns a document's fields in the order it holds them, which depends on how they were written. hidane returned them in name order. This makes hidane hold and return the same order, everywhere.
What the official emulator does
tools/oracle/field_order.pyrecords 34 cases against v1.22.0 (crates/hidane/tests/fixtures/field_order.json). Every observation follows these rules:updateTimeas they wereHashMap; those whose value changes go into a secondHashMap, and in that map's order each is removed and added again at the end. A map changed inside moves the same way. Unchanged fields stay where they areHashMap, the values found put into anotherThe
HashMaporder is Java's:String.hashCodeover UTF-16,h ^ (h >>> 16), 16 buckets doubling past 3/4 load (or when a bucket gets a ninth key below 64 buckets), insertion order within a bucket. Using the two maps' own sizes is what makeszetaandstr(same bucket of 16) swap when a mask names fields that are missing or unchanged.How
Document,MapValueandAggregationResultare written by hand insrc/ordered.rsaroundFields, an insertion-ordered map (IndexMap), and replace the generated messages throughextern_path. The wire format is unchanged; entries are encoded in order and decoded through prost's own map decoding. A second descriptor set describes these three maps as lists of entries.hash_ordermodels Java'sHashMapiteration;field_path::apply_maskandprojectimplement the rules above. Comparisons (order.rs) and index keys (key.rs) still use UTF-8 key order.StoredDocument::has_fieldscompares in canonical order.ordered_json.rsturns objects into entry lists and back), andserde_jsonhaspreserve_order.Tests
tests/field_order.rsreplays every recording over REST: status and error message, the field order of every document returned at every level (79 reads), and which commitupdateTimenames. A second test checks gRPC.hash_order.rs), the mask rules (field_path.rs) and the JSON round trip (ordered_json.rs), from the recorded cases.export_import.rsnow also checks that reading an official export gives the official import's order.docs/parity-exceptions.md;docs/export-format.mdandtools/oracle/README.mdare updated.Checks:
cargo fmt --check,cargo clippy --workspace --all-targets -D warnings,cargo test --workspace(165 passed), andcargo checkwith Rust 1.88.0.🤖 Generated with Claude Code