fix object functions & add tests for object type & fix some bugs

[shuwenwei]

Description

fix object functions & add tests for object type & fix some bugs

Test Coverage Summary

✔ Added in this release

READ_OBJECT
Added tests for reading object data with offset/length, validating correct binary retrieval

✔ Previously existing, newly verified for OBJECT type

LENGTH
Verified that LENGTH returns INT64 for OBJECT columns instead of INT32.

CAST / TRY_CAST
Verified that casting OBJECT → STRING works, and casting to OBJECT is not permitted.

COALESCE
Verified correct fallback behavior with multiple OBJECT values.

IS [NOT] NULL
Verified null evaluation correctness on OBJECT columns.

✔ Aggregation functions supporting OBJECT, validated

COUNT, FIRST, LAST, FIRST_BY, LAST_BY
Verified OBJECT compatibility and correctness of returned values.

✔ Window functions, verified with OBJECT input

FIRST_VALUE, LAST_VALUE, NTH_VALUE, LEAD, LAG
Confirmed support and correctness when applied to OBJECT data.

✔ Row pattern recognition functions, verified

RPR_FIRST, RPR_LAST, PREV, NEXT
Verified that the first argument may be OBJECT.

✔ SQL clause & syntax support, verified

FROM, WHERE, SELECT, HAVING
Confirmed these clauses can read and process OBJECT fields.

✔ Constraints validated

GROUP BY, ORDER BY, PARTITION BY / ORDER BY in MATCH_RECOGNIZE and WINDOW, OFFSET, LIMIT
Verified that these do not support OBJECT as sorting/partitioning keys.

✔ FILL behavior validated

Confirmed that only FILL PREVIOUS is supported for OBJECT values.

✔ TTL behavior validated
Verified that the expired object file can be deleted by compaction.

[codecov]

Codecov Report

❌ Patch coverage is 44.82759% with 80 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (force_ci/object_type@7249a52). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...ava/org/apache/iotdb/db/utils/ObjectTypeUtils.java	37.17%	49 Missing ⚠️
.../scalar/AbstractCastFunctionColumnTransformer.java	0.00%	5 Missing ⚠️
...ion/operator/process/function/partition/Slice.java	66.66%	4 Missing ⚠️
...execution/relational/ColumnTransformerBuilder.java	0.00%	4 Missing ⚠️
...ne/plan/relational/metadata/TableMetadataImpl.java	0.00%	4 Missing ⚠️
...ol/thrift/impl/DataNodeInternalRPCServiceImpl.java	0.00%	2 Missing ⚠️
...eryengine/plan/planner/TableOperatorGenerator.java	0.00%	2 Missing ⚠️
...nner/plan/node/write/RelationalInsertRowsNode.java	0.00%	2 Missing ⚠️
...mn/unary/scalar/CastFunctionColumnTransformer.java	0.00%	2 Missing ⚠️
...unary/scalar/TryCastFunctionColumnTransformer.java	0.00%	2 Missing ⚠️
... and 4 more

Additional details and impacted files

@@                   Coverage Diff                   @@
##             force_ci/object_type   #16829   +/-   ##
=======================================================
  Coverage                        ?   38.83%           
  Complexity                      ?      207           
=======================================================
  Files                           ?     5029           
  Lines                           ?   333887           
  Branches                        ?    42509           
=======================================================
  Hits                            ?   129681           
  Misses                          ?   204206           
  Partials                        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR fixes object type handling and adds comprehensive test coverage for object type operations in IoTDB. The changes include fixing object-related functions, refactoring the length function implementation, adding remote object file reading capabilities, and ensuring proper handling of OBJECT data types across various query operations.

• Introduces readObject RPC method and associated infrastructure for reading object files from remote data nodes
• Refactors the LENGTH function to support both STRING and OBJECT types with a unified base class
• Adds extensive test coverage including compaction tests, function tests, and integration tests for object types

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
iotdb-protocol/thrift-datanode/src/main/thrift/datanode.thrift	Adds TReadObjectReq struct and readObject RPC method for remote object file access
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/object/ObjectTypeCompactionTest.java	New test class for object type compaction with TTL handling
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/ObjectTypeFunctionTest.java	New test class for object length and read functions
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/ObjectTypeUtils.java	Adds methods for reading object content locally and remotely, including binary parsing
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/ObjectLengthColumnTransformer.java	New transformer for getting object length
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/AbstractLengthColumnTransformer.java	New abstract base class for length transformers
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/LengthColumnTransformer.java	Refactored to extend AbstractLengthColumnTransformer
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/ReadObjectColumnTransformer.java	Refactored to use ObjectTypeUtils for reading object content
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/AbstractCastFunctionColumnTransformer.java	Adds castObject method for casting OBJECT type to STRING
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/CastFunctionColumnTransformer.java	Adds OBJECT case in cast switch statement
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/unary/scalar/TryCastFunctionColumnTransformer.java	Adds OBJECT case in try_cast switch statement
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/metadata/TableMetadataImpl.java	Updates LENGTH function signature to accept OBJECT type and return INT64
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertRowsNode.java	Adds null check for OBJECT type values
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/TableOperatorGenerator.java	Adds OBJECT type to statistics and scan order checks
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/OperatorTreeGenerator.java	Adds OBJECT case for previous fill operation
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/ClusterPartitionFetcher.java	Adds getRegionReplicaSet method for fetching replica sets
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/relational/ColumnTransformerBuilder.java	Updates LENGTH and READ_OBJECT functions to use correct transformers and return types
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/RecordIterator.java	Moved from commons package and adds readObject methods
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/partition/Slice.java	Adds readObject methods to Record implementation
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/rowpattern/expression/PatternExpressionComputation.java	Adds OBJECT type handling in pattern expression computation
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/DataNodeInternalRPCServiceImpl.java	Implements readObject RPC method
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/CompactionUtils.java	Adds SuppressWarnings annotation for complexity
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/grouped/GroupedUserDefinedAggregateAccumulator.java	Updates RecordIterator import path
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/UserDefinedAggregateFunctionAccumulator.java	Removes unused RecordIterator import
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/MaskedRecordIterator.java	Removes unused RecordIterator import
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/transformation/dag/column/udf/UserDefineScalarFunctionTransformer.java	Updates RecordIterator import path
iotdb-core/confignode/src/test/resources/confignode1conf/iotdb-system.properties	Changes data_replication_factor from 3 to 1
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/access/Record.java	Adds readObject methods to Record interface with documentation
integration-test/src/test/java/org/apache/iotdb/relational/it/query/recent/IoTDBObjectTypeQueryIT.java	New integration test for object type queries
integration-test/src/test/java/org/apache/iotdb/relational/it/query/old/builtinfunction/scalar/IoTDBScalarFunctionTableIT.java	Updates error messages for LENGTH function tests

Comments suppressed due to low confidence (1)

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/RecordIterator.java:145

• The condition is inverted. The method should throw an exception when the column is NOT an object type, but currently it throws when it IS an object type.

Should be: if (getDataType(columnIndex) != Type.OBJECT)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

[sonarqubecloud]

Quality Gate failed

Failed conditions
5.4% Duplication on New Code (required ≤ 5%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE