Skip to content

Conversation

@chenzl25
Copy link
Collaborator

@chenzl25 chenzl25 commented Sep 9, 2024

No description provided.

Xuanwo and others added 30 commits December 26, 2023 08:45
* feat: Add website layout

Signed-off-by: Xuanwo <github@xuanwo.io>

* publish to rust.i.a.o

Signed-off-by: Xuanwo <github@xuanwo.io>

* Fix license

Signed-off-by: Xuanwo <github@xuanwo.io>

* Let's try mdbook action

Signed-off-by: Xuanwo <github@xuanwo.io>

* use cargo install

Signed-off-by: Xuanwo <github@xuanwo.io>

* disable section

Signed-off-by: Xuanwo <github@xuanwo.io>

* Add docs for website

Signed-off-by: Xuanwo <github@xuanwo.io>

* Fix license

Signed-off-by: Xuanwo <github@xuanwo.io>

* action approved

Signed-off-by: Xuanwo <github@xuanwo.io>

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
* feat: Expressions

* Fix comments

* Refactor expression to be more similar to iceberg model

* Fix typo
Signed-off-by: Xuanwo <github@xuanwo.io>
* feat: Add roadmap and features status in README.md

* Fix

* Fix

* Add more details according to comments

* Revert unnecessary new line break

* Nits

---------

Co-authored-by: Fokko Driesprong <fokko@apache.org>
Bumps [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) from 3.9.2 to 3.9.3.
- [Release notes](https://github.com/peaceiris/actions-gh-pages/releases)
- [Changelog](https://github.com/peaceiris/actions-gh-pages/blob/main/CHANGELOG.md)
- [Commits](peaceiris/actions-gh-pages@v3.9.2...v3.9.3)

---
updated-dependencies:
- dependency-name: peaceiris/actions-gh-pages
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Updates the requirements on [opendal](https://github.com/apache/incubator-opendal) to permit the latest version.
- [Release notes](https://github.com/apache/incubator-opendal/releases)
- [Changelog](https://github.com/apache/incubator-opendal/blob/main/CHANGELOG.md)
- [Commits](apache/opendal@v0.43.0...v0.43.0)

---
updated-dependencies:
- dependency-name: opendal
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Code complete

* Resolve

* Done

* Fix comments

* Fix comments

* Fix comments

* Fix

* Fix comment
* chore: Update reader api status

* Restore unnecessary change
* Add formatting for toml files

* Update call to taplo

* Add command to format and a command to check
Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](rust-cli/env_logger@v0.10.0...v0.10.2)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* init file writer interface

* refine

---------

Co-authored-by: ZENOTME <st810918843@gmail.com>
* fix: Manifest parsing should consider schema evolution.

* Fix ut
* Add

* Fix format

* Add license header
…175)

Updates the requirements on [derive_builder](https://github.com/colin-kiegel/rust-derive-builder) to permit the latest version.
- [Release notes](https://github.com/colin-kiegel/rust-derive-builder/releases)
- [Commits](colin-kiegel/rust-derive-builder@v0.12.0...v0.12.0)

---
updated-dependencies:
- dependency-name: derive_builder
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* add handwritten serialize

* revert expect

* remove expect
Co-authored-by: Fokko Driesprong <fokko@apache.org>
* feat: Bump version 0.2.0 to prepare for release.

* Update dependencies
Updates the requirements on [opendal](https://github.com/apache/opendal) to permit the latest version.
- [Release notes](https://github.com/apache/opendal/releases)
- [Changelog](https://github.com/apache/opendal/blob/main/CHANGELOG.md)
- [Commits](apache/opendal@v0.44.0...v0.44.2)

---
updated-dependencies:
- dependency-name: opendal
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
FANNG1 and others added 26 commits August 17, 2024 00:42
* add memory catalog

* fix style

* fix style
…ce (#512)

* feat: adds ObjectCache, to cache Manifests and ManifestLists

* refactor: change obj cache method names and use more readable default usize value

* chore: improve error message

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>

* fix: change object cache retrieval method visibility

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>

* feat: improved error message in object cache get_manifest

* test(object-cache): add unit tests for object cache manifest and manifest list retrieval

* fix: ensure that object cache insertions are weighted by size

* test: fix test typo

* fix: ensure object cache weight is that of the wrapped item, not the Arc

---------

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Signed-off-by: Xuanwo <github@xuanwo.io>
* fix: ensure that RestCatalog passes user config to FileIO

* docs: added some doc comments to clarify override order for config
Both licenses can be moved to the `allowed` section:

- **adler32** [ships](https://github.com/remram44/adler32-rs/blob/master/LICENSE) with a **zlib** license and is a category A-license
- **unicode-ident** ships with a **UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE** which is also a category A-license

The **ring** license is a bit [more involved](https://github.com/briansmith/ring/blob/main/LICENSE) and carries a lot of history, I think it is best to keep that as an exception for now, since the OpenSSL license is also not explicitly listed on the ASF page. I don't see anything alarming in the `LICENSE` file.

ASF page on the subject: https://www.apache.org/legal/resolved.html#category-a
Signed-off-by: Xuanwo <github@xuanwo.io>
)

* feat(timestamp_ns): first commit

* feat(timestamp_ns): Add mappings for timestamp_ns/timestamptz_ns

* feat(timestamp_ns): Remove unused dep

* feat(timestamp_ns): Fix unit test

* feat(timestamp_ns): Fix test_all_type_for_write()

* feat(timestamp_ns): fix test_transform_days_literal

* feat(timestamp_ns): fix math for timestamptz_nanos

* chore: formatting

* chore: formatting

* chore: Appease clippy

---------

Co-authored-by: Timothy Maloney <tmaloney@influxdata.com>
* correct partition-id to field id in PartitionSpec

* correct partition-id to field id in PartitionSpec

* correct partition-id to field id in PartitionSpec

* xx
---
updated-dependencies:
- dependency-name: typed-builder
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* bucket transform rust binding

* format

* poetry x maturin

* ignore poetry.lock in license check

* update bindings_python_ci to use makefile

* newline

* python-poetry/poetry#9135

* use hatch instead of poetry

* refactor

* revert licenserc change

* adopt review feedback

* comments

* unused dependency

* adopt review comment

* newline

* I like this approach a lot better

* more tests
* feat(scan): add row group and page index row selection filtering

* fix(row selection): off-by-one error

* feat: remove row selection to defer to a second PR

* feat: better min/max val conversion in RowGroupMetricsEvaluator

* test(row_group_filtering): first three tests

* test(row_group_filtering): next few tests

* test: add more tests for RowGroupMetricsEvaluator

* chore: refactor test assertions to silence clippy lints

* refactor: consolidate parquet stat min/max parsing in one place
* feat: SQL Catalog - namespaces

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* feat: use transaction for updates and creates

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: pull out query param builder to fn

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* feat: add drop and tests

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: String to str, remove pub and optimise query builder

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: nested match, remove ok()

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: remove pub, add set, add comments

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: refactor list_namespaces slightly

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: add default properties to all new namespaces

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: remove check for nested namespace

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* chore: add more comments to the CatalogConfig to explain bind styles

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

* fix: edit test for nested namespaces

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>

---------

Signed-off-by: callum-ryan <19956159+callum-ryan@users.noreply.github.com>
Signed-off-by: Xuanwo <github@xuanwo.io>
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](crate-ci/typos@v1.24.3...v1.24.5)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Partition compatability

* Partition compatability

* Rename compatible_with -> is_compatible_with
* SortOrder methods should take schema ref if possible

* Fix test type

* with_order_id should not take reference
@chenzl25 chenzl25 closed this Sep 9, 2024
chenzl25 pushed a commit that referenced this pull request Jan 6, 2026
…chTransformer (apache#1821)

## Which issue does this PR close?

Partially address apache#1749.

## What changes are included in this PR?

This PR adds partition spec handling to `FileScanTask` and
`RecordBatchTransformer` to correctly implement the Iceberg spec's
"Column Projection" rules for fields "not present" in data files.

### Problem Statement

Prior to this PR, `iceberg-rust`'s `FileScanTask` had no mechanism to
pass partition information to `RecordBatchTransformer`, causing two
issues:

1. **Incorrect handling of bucket partitioning**: Couldn't distinguish
identity transforms (which should use partition metadata constants) from
non-identity transforms like bucket/truncate/year/month (which must read
from data file). For example, `bucket(4, id)` stores
`id_bucket = 2` (bucket number) in partition metadata, but actual `id`
values (100, 200, 300) are only in the data file. iceberg-rust was
incorrectly treating bucket-partitioned source columns as constants,
breaking runtime filtering and returning incorrect query results.

2. **Field ID conflicts in add_files scenarios**: When importing Hive
tables via `add_files`, partition columns could have field IDs
conflicting with Parquet data columns. Example: Parquet has
field_id=1→"name", but Iceberg expects field_id=1→"id" (partition). Per
spec, the
correct field is "not present" and requires name mapping fallback.

### Iceberg Specification Requirements

Per the Iceberg spec
(https://iceberg.apache.org/spec/#column-projection), when a field ID is
"not present" in a data file, it must be resolved using these rules:

1. Return the value from partition metadata if an **Identity Transform**
exists
2. Use `schema.name-mapping.default` metadata to map field id to columns
without field id
3. Return the default value if it has a defined `initial-default`
4. Return null in all other cases

**Why this matters:**
- **Identity transforms** (e.g., `identity(dept)`) store actual column
values in partition metadata that can be used as constants without
reading the data file
- **Non-identity transforms** (e.g., `bucket(4, id)`, `day(timestamp)`)
store transformed values in partition metadata (e.g., bucket number 2,
not the actual `id` values 100, 200, 300) and must read source columns
from the data file

### Changes Made

1. **Added partition fields to `FileScanTask`** (`scan/task.rs`):
- `partition: Option<Struct>` - Partition data from manifest entry
- `partition_spec: Option<Arc<PartitionSpec>>` - For transform-aware
constant detection
- `name_mapping: Option<Arc<NameMapping>>` - Name mapping from table
metadata

2. **Implemented `constants_map()` function**
(`arrow/record_batch_transformer.rs`):
- Replicates Java's `PartitionUtil.constantsMap()` behavior
- Only includes fields where transform is `Transform::Identity`
- Used to determine which fields use partition metadata constants vs.
reading from data files

3. **Enhanced `RecordBatchTransformer`**
(`arrow/record_batch_transformer.rs`):
- Added `build_with_partition_data()` method to accept partition spec,
partition data, and name mapping
- Implements all 4 spec rules for column resolution with
identity-transform awareness
- Detects field ID conflicts by verifying both field ID AND name match
- Falls back to name mapping when field IDs are missing/conflicting
(spec rule #2)

4. **Updated `ArrowReader`** (`arrow/reader.rs`):
- Uses `build_with_partition_data()` when partition information is
available
- Falls back to `build()` when not available

5. **Updated manifest entry processing** (`scan/context.rs`):
- Populates partition fields in `FileScanTask` from manifest entry data

### Tests Added

1. **`bucket_partitioning_reads_source_column_from_file`** - Verifies
that bucket-partitioned source columns are read from data files (not
treated as constants from partition metadata)

2. **`identity_partition_uses_constant_from_metadata`** - Verifies that
identity-transformed fields correctly use partition metadata constants

3. **`test_bucket_partitioning_with_renamed_source_column`** - Verifies
field-ID-based mapping works despite column rename

4. **`add_files_partition_columns_without_field_ids`** - Verifies name
mapping resolution for Hive table imports without field IDs (spec rule
#2)

5. **`add_files_with_true_field_id_conflict`** - Verifies correct field
ID conflict detection with name mapping fallback (spec rule #2)

6. **`test_all_four_spec_rules`** - Integration test verifying all 4
spec rules work together

## Are these changes tested?

Yes, there are 6 new unit tests covering all 4 Iceberg spec rules. This
also resolved approximately 50 Iceberg Java tests when running with
DataFusion Comet's experimental
apache/datafusion-comet#2528 PR.

---------

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
nagraham pushed a commit to nagraham/iceberg-rust that referenced this pull request Jan 6, 2026
…chTransformer (apache#1821)

Partially address apache#1749.

This PR adds partition spec handling to `FileScanTask` and
`RecordBatchTransformer` to correctly implement the Iceberg spec's
"Column Projection" rules for fields "not present" in data files.

Prior to this PR, `iceberg-rust`'s `FileScanTask` had no mechanism to
pass partition information to `RecordBatchTransformer`, causing two
issues:

1. **Incorrect handling of bucket partitioning**: Couldn't distinguish
identity transforms (which should use partition metadata constants) from
non-identity transforms like bucket/truncate/year/month (which must read
from data file). For example, `bucket(4, id)` stores
`id_bucket = 2` (bucket number) in partition metadata, but actual `id`
values (100, 200, 300) are only in the data file. iceberg-rust was
incorrectly treating bucket-partitioned source columns as constants,
breaking runtime filtering and returning incorrect query results.

2. **Field ID conflicts in add_files scenarios**: When importing Hive
tables via `add_files`, partition columns could have field IDs
conflicting with Parquet data columns. Example: Parquet has
field_id=1→"name", but Iceberg expects field_id=1→"id" (partition). Per
spec, the
correct field is "not present" and requires name mapping fallback.

Per the Iceberg spec
(https://iceberg.apache.org/spec/#column-projection), when a field ID is
"not present" in a data file, it must be resolved using these rules:

1. Return the value from partition metadata if an **Identity Transform**
exists
2. Use `schema.name-mapping.default` metadata to map field id to columns
without field id
3. Return the default value if it has a defined `initial-default`
4. Return null in all other cases

**Why this matters:**
- **Identity transforms** (e.g., `identity(dept)`) store actual column
values in partition metadata that can be used as constants without
reading the data file
- **Non-identity transforms** (e.g., `bucket(4, id)`, `day(timestamp)`)
store transformed values in partition metadata (e.g., bucket number 2,
not the actual `id` values 100, 200, 300) and must read source columns
from the data file

1. **Added partition fields to `FileScanTask`** (`scan/task.rs`):
- `partition: Option<Struct>` - Partition data from manifest entry
- `partition_spec: Option<Arc<PartitionSpec>>` - For transform-aware
constant detection
- `name_mapping: Option<Arc<NameMapping>>` - Name mapping from table
metadata

2. **Implemented `constants_map()` function**
(`arrow/record_batch_transformer.rs`):
- Replicates Java's `PartitionUtil.constantsMap()` behavior
- Only includes fields where transform is `Transform::Identity`
- Used to determine which fields use partition metadata constants vs.
reading from data files

3. **Enhanced `RecordBatchTransformer`**
(`arrow/record_batch_transformer.rs`):
- Added `build_with_partition_data()` method to accept partition spec,
partition data, and name mapping
- Implements all 4 spec rules for column resolution with
identity-transform awareness
- Detects field ID conflicts by verifying both field ID AND name match
- Falls back to name mapping when field IDs are missing/conflicting
(spec rule risingwavelabs#2)

4. **Updated `ArrowReader`** (`arrow/reader.rs`):
- Uses `build_with_partition_data()` when partition information is
available
- Falls back to `build()` when not available

5. **Updated manifest entry processing** (`scan/context.rs`):
- Populates partition fields in `FileScanTask` from manifest entry data

1. **`bucket_partitioning_reads_source_column_from_file`** - Verifies
that bucket-partitioned source columns are read from data files (not
treated as constants from partition metadata)

2. **`identity_partition_uses_constant_from_metadata`** - Verifies that
identity-transformed fields correctly use partition metadata constants

3. **`test_bucket_partitioning_with_renamed_source_column`** - Verifies
field-ID-based mapping works despite column rename

4. **`add_files_partition_columns_without_field_ids`** - Verifies name
mapping resolution for Hive table imports without field IDs (spec rule

5. **`add_files_with_true_field_id_conflict`** - Verifies correct field
ID conflict detection with name mapping fallback (spec rule risingwavelabs#2)

6. **`test_all_four_spec_rules`** - Integration test verifying all 4
spec rules work together

Yes, there are 6 new unit tests covering all 4 Iceberg spec rules. This
also resolved approximately 50 Iceberg Java tests when running with
DataFusion Comet's experimental
apache/datafusion-comet#2528 PR.

---------

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
chenzl25 pushed a commit that referenced this pull request Jan 7, 2026
…chTransformer (apache#1821) (#107)

Partially address apache#1749.

This PR adds partition spec handling to `FileScanTask` and
`RecordBatchTransformer` to correctly implement the Iceberg spec's
"Column Projection" rules for fields "not present" in data files.

Prior to this PR, `iceberg-rust`'s `FileScanTask` had no mechanism to
pass partition information to `RecordBatchTransformer`, causing two
issues:

1. **Incorrect handling of bucket partitioning**: Couldn't distinguish
identity transforms (which should use partition metadata constants) from
non-identity transforms like bucket/truncate/year/month (which must read
from data file). For example, `bucket(4, id)` stores
`id_bucket = 2` (bucket number) in partition metadata, but actual `id`
values (100, 200, 300) are only in the data file. iceberg-rust was
incorrectly treating bucket-partitioned source columns as constants,
breaking runtime filtering and returning incorrect query results.

2. **Field ID conflicts in add_files scenarios**: When importing Hive
tables via `add_files`, partition columns could have field IDs
conflicting with Parquet data columns. Example: Parquet has
field_id=1→"name", but Iceberg expects field_id=1→"id" (partition). Per
spec, the
correct field is "not present" and requires name mapping fallback.

Per the Iceberg spec
(https://iceberg.apache.org/spec/#column-projection), when a field ID is
"not present" in a data file, it must be resolved using these rules:

1. Return the value from partition metadata if an **Identity Transform**
exists
2. Use `schema.name-mapping.default` metadata to map field id to columns
without field id
3. Return the default value if it has a defined `initial-default`
4. Return null in all other cases

**Why this matters:**
- **Identity transforms** (e.g., `identity(dept)`) store actual column
values in partition metadata that can be used as constants without
reading the data file
- **Non-identity transforms** (e.g., `bucket(4, id)`, `day(timestamp)`)
store transformed values in partition metadata (e.g., bucket number 2,
not the actual `id` values 100, 200, 300) and must read source columns
from the data file

1. **Added partition fields to `FileScanTask`** (`scan/task.rs`):
- `partition: Option<Struct>` - Partition data from manifest entry
- `partition_spec: Option<Arc<PartitionSpec>>` - For transform-aware
constant detection
- `name_mapping: Option<Arc<NameMapping>>` - Name mapping from table
metadata

2. **Implemented `constants_map()` function**
(`arrow/record_batch_transformer.rs`):
- Replicates Java's `PartitionUtil.constantsMap()` behavior
- Only includes fields where transform is `Transform::Identity`
- Used to determine which fields use partition metadata constants vs.
reading from data files

3. **Enhanced `RecordBatchTransformer`**
(`arrow/record_batch_transformer.rs`):
- Added `build_with_partition_data()` method to accept partition spec,
partition data, and name mapping
- Implements all 4 spec rules for column resolution with
identity-transform awareness
- Detects field ID conflicts by verifying both field ID AND name match
- Falls back to name mapping when field IDs are missing/conflicting
(spec rule #2)

4. **Updated `ArrowReader`** (`arrow/reader.rs`):
- Uses `build_with_partition_data()` when partition information is
available
- Falls back to `build()` when not available

5. **Updated manifest entry processing** (`scan/context.rs`):
- Populates partition fields in `FileScanTask` from manifest entry data

1. **`bucket_partitioning_reads_source_column_from_file`** - Verifies
that bucket-partitioned source columns are read from data files (not
treated as constants from partition metadata)

2. **`identity_partition_uses_constant_from_metadata`** - Verifies that
identity-transformed fields correctly use partition metadata constants

3. **`test_bucket_partitioning_with_renamed_source_column`** - Verifies
field-ID-based mapping works despite column rename

4. **`add_files_partition_columns_without_field_ids`** - Verifies name
mapping resolution for Hive table imports without field IDs (spec rule

5. **`add_files_with_true_field_id_conflict`** - Verifies correct field
ID conflict detection with name mapping fallback (spec rule #2)

6. **`test_all_four_spec_rules`** - Integration test verifying all 4
spec rules work together

Yes, there are 6 new unit tests covering all 4 Iceberg spec rules. This
also resolved approximately 50 Iceberg Java tests when running with
DataFusion Comet's experimental
apache/datafusion-comet#2528 PR.

---------

Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com>
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.