Skip to content

Conversation

@chitralverma
Copy link
Contributor

@chitralverma chitralverma commented Oct 21, 2025

Which issue does this PR close?

Parts from #6622

Rationale for this change

Closes #6253

What changes are included in this PR?

  • Add stubs for File and AsyncFile
  • Add stubs for Operator, and AsyncOperator
  • Updated API reference

Are there any user-facing changes?

Yes.

capability() on Operator and AsyncOperator has been renamed to full_capability()

@chitralverma chitralverma changed the title Gen stubs 3 feat(bindings/python): Add stubs for remaining types Oct 21, 2025
import typing

PathBuf: TypeAlias = str | os.PathLike
from opendal import capability, exceptions, file, layers, types
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this import is auto generated and capability here was conflicting with capability method on Operator and AsyncOperator.

So had to change method name to full_capability - a user facing change.

ConcurrentLimitLayer.
Create a layer that limits the number of concurrent operations.
A layer that limits the number of concurrent operations.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor wording changes, to de duplicate heading in mkdocs

Comment on lines -27 to 29
/// Layer
///
/// Layers are used to intercept the operations on the underlying storage.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor wording changes, to de duplicate heading in mkdocs

Comment on lines +70 to +71
m.add_class::<Operator>()?;
m.add_class::<AsyncOperator>()?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initially i wanted to have operators in in their own py module, but then this would require a major breaking change from from opendal import Operator to from opendal.operator import Operator.

I tried messing with the __init__.py to get around this, but maybe this can be taken up in another PR.

Comment on lines +591 to +594
#[gen_stub(override_return_type(
type_repr="collections.abc.Awaitable[builtins.int]",
imports=("collections.abc", "builtins")
))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the generated functions are not async on py side, but the return types have been overridden to Awaitable as advised here.

same has been done on operator side


[lint.per-file-ignores]
"*.pyi" = ["PYI021", "ANN003", "RUF100"]
"*.pyi" = ["PYI021", "ANN003", "RUF100", "ANN401"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

necessary for the new/ open on Operator and AsyncOperator which take **kwargs.

Comment on lines +22 to +28
cap = operator.full_capability()
assert cap is not None
assert cap.read is not None


def test_capability_exception(service_name, operator):
cap = operator.capability()
cap = operator.full_capability()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see this comment for explanation.

@chitralverma chitralverma marked this pull request as ready for review October 21, 2025 18:17
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Oct 21, 2025
@chitralverma
Copy link
Contributor Author

@Xuanwo final one for stubs for your review, whenever you're free.


Copilot Summary:

This pull request refines the Python bindings for OpenDAL, focusing on improved documentation, more accurate type stubs, and consistent module organization. The most significant changes include enhanced docstrings and type annotations for the File and AsyncFile classes, the addition of a new PresignedRequest type, and updates to documentation and configuration files to reflect these improvements.

Python API improvements

  • Added comprehensive docstrings and accurate type annotations to the File and AsyncFile classes in file.pyi, making their usage and behavior clearer for users. Methods now include detailed parameter and return type documentation.
  • Updated the implementation and Python stub generation for the File class in src/file.rs, including improved docstrings, type overrides, and method signatures to match Python conventions and enhance usability. [1] [2] [3] [4] [5] [6] [7] [8]

New types and documentation

  • Introduced the PresignedRequest type to the Python bindings, including its stub and documentation, allowing users to access presigned request details such as URL, method, and headers. [1] [2]
  • Updated API documentation files to reflect the new module structure (e.g., opendal.file.File instead of opendal.File) and added documentation for the new PresignedRequest type. [1] [2] [3] [4]

Layer class docstring improvements

  • Improved docstrings for layer classes (ConcurrentLimitLayer, MimeGuessLayer, RetryLayer, and Layer) in layers.pyi to provide clearer descriptions of their functionality and usage. [1] [2] [3]

Configuration and dependency updates

  • Bumped the pyo3-stub-gen dependency version from 0.15 to 0.16 in Cargo.toml to ensure compatibility with the latest stub generation features.
  • Updated pyrightconfig.json and ruff.toml to adjust linting and type checking behavior for .pyi files, reflecting the expanded use of type annotations. [1] [2]

@Xuanwo Xuanwo changed the title feat(bindings/python): Add stubs for remaining types feat(bindings/python)!: Add stubs for remaining types Oct 22, 2025
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thank you for working on this!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 22, 2025
@Xuanwo Xuanwo merged commit 76f0667 into apache:main Oct 22, 2025
64 checks passed
@chitralverma chitralverma deleted the gen-stubs-3 branch October 22, 2025 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: Improve automatic stub generation for python bindings

2 participants