Skip to content

Remote Client

A pool of remote monty workers reached over gRPC instead of local subprocesses — the intended peer is monty-server. The pool holds one HTTP/2 connection to the server and each checkout opens one MontySession.Session stream on it; a stream is never reused. See running monty-server and the remote-worker trust boundary. A remote peer may be CPython rather than a Monty sandbox; the transport does not provide isolation. AsyncMontyWebsocket is the same class under its deprecated former name, and a ws:// or wss:// URL is read as http:// or https://.

checkout(profile='...') names a server-side profile to run the session under; None takes the server’s default. The client sends the name to the server unchecked; what a profile means is the server’s choice, and the monty worker ignores it.

Connection loss and shutdown

MontyRefusedError means the server refused to open the session’s stream: at capacity, with no worker to serve it, or rejecting the caller; its status_code, status_message and cause say why, and no session existed. retryable is true for RESOURCE_EXHAUSTED and UNAVAILABLE, not for credential errors. retry_after is the suggested delay in seconds from Retry-After, capped at one hour, or None if absent or invalid. The client accepts delay-seconds and HTTP dates but does not retry automatically. A server that cannot be reached, that answers with any other status (such as UNIMPLEMENTED from a URL that is no Monty server), or that is no gRPC server at all (such as a web server’s page), raises MontyConnectError. It retains status_code, status_message and cause when the server sent a status, or transport_error otherwise. Both errors subclass MontyError. If a CPU-mode server requires a finite request_timeout, an incompatible client setting raises MontyConfigurationError. Malformed request metadata names or values raise the same error before any connection attempt. MontyDisconnectError means the stream ended mid-session. A server that ended it by policy says so in the error’s status and cause (see disconnect diagnostics); a transport failure says nothing about the worker’s fate. Either way the session is lost; check out a new one. The error text includes the recorded code, cause and message, or the transport error. For example: monty worker connection closed while waiting for a reply: DEADLINE_EXCEEDED (turn_timeout): turn expired. This does not establish whether the request ran, so do not replay it automatically.

MontyShutdown means the server declined the next request because it is shutting down. That request did not run. Against a server that stores sessions the client resumes the session itself before raising, see stored sessions; the exception is raised only when that fails or is disabled. If the exception includes a dump, restore it into a fresh session using the appropriate snapshot loader. Restoring a suspended call re-announces it even if the host already executed the callback, so callback side effects can happen twice. Neither exception occurs on the local subprocess transport.

Stored sessions

A server that stores sessions gives each one an opaque ID instead of sending its state back. get_session_id() returns it; it returns None against a server that stores nothing. The session’s configuration normally travels with its first request, so called before any request, get_session_id() sends the configuration on its own and waits for the reply, which costs one round trip. The session_id property is deprecated: it is None until the server has accepted the configuration, whose reply comes back with the first request’s, even when that request fails. monty-server run without a store also refuses dump(), load_session() and load_snapshot() with MontyRuntimeError, and the session carries on; the development relay in scripts/grpc_relay.py instead passes them to its worker, which dumps and loads bytes as a local session does. Against a storing server dump() writes the session’s current state to a record that never changes and returns that record’s ID; the session continues under its own ID. The server also writes the session’s state under the session’s ID whenever it parks the session: idle for its --park-after, the client gone, or a drain. Passing either ID to the loader matching the stored state, load_session() for an idle session or load_snapshot() for one suspended mid-feed, on a fresh session, from any process, starts a new session with its own ID: a session’s ID gives the state as of that session’s last park, a dump() ID the state dumped. The record is unchanged and the session that wrote it is never resumed in place, so loading one ID twice gives two independent sessions, and a session still running elsewhere is unaffected. Loading an ID restores a dump, so what restoring does not carry applies: host objects sent before the dump, in particular, are unknown to the new session. To branch a session at a chosen point, call dump(), then load_session() with its ID once per branch; the original session can carry on as well. checkout(ephemeral=True) asks the server not to store the session on its own, so it has no ID; whether it honours dump() is the server’s choice.

When such a server drains a session, the client opens a new stream, loads the state the server named into a new session and re-sends the request, so the caller sees the result rather than MontyShutdown; get_session_id() then returns the new session’s ID, and the session’s suspension and sleep totals carry over. MontyShutdown is still raised if the server named nothing to load, if the reload fails, if auto_resume=False is passed to AsyncMontyRemote, or if the session has no ID. The new stream uses the metadata returned when the session was entered, so an expired token makes the resume fail. MontyDisconnectError is never resumed, because the request may have run.

Disconnect diagnostics

MontyDisconnectError exposes status_code, status_message, cause and transport_error. If the server ended the stream with a gRPC status, status_code holds its canonical name (DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE, …), or OK when the server simply finished the stream, and status_message its message. A monty-server names why in cause: idle_timeout, session_timeout, turn_timeout, out_of_memory, server_shutdown, capacity, caller_cap, unauthenticated, permission_denied, worker_unavailable or protocol_violation. An out_of_memory cause is raised as the same MemoryError a local out-of-memory kill raises, and a server_shutdown cause as MontyShutdown with no dump, so neither reaches this error. A connection failure or protocol failure supplies transport_error text instead, with no status. A later failure does not replace an earlier one. Missing evidence remains None, and a status message is server-supplied text, not a verified explanation. These fields do not establish whether the request ran, so the client does not replay it.

Dependencies

install_dependencies() is supported only by embedded-CPython workers. It installs PEP 508 requirements using uv, within the pool’s request_timeout. Those workers also install PEP 723 inline dependencies before running a feed. A Monty sandbox worker rejects non-empty installation requests with MontyRuntimeError and ignores PEP 723 comments. install_dependencies([]) is a no-op on either worker.

Host decode budget

AsyncMontyRemote(max_decode_bytes=...) can lower the default 1 GiB allocation budget for each decoded protocol frame. Values above 1 GiB are rejected when the client is constructed. The budget excludes wire buffers and Python conversion, and does not change sandbox memory limits. Exhaustion ends the session with a protocol error; the client does not replay the request. Separately, the transport refuses any message larger than the budget, clamped between 64 KiB and the 256 MiB frame limit, before buffering it.

API

AsyncMontyRemote

Async context manager owning a pool of remote monty workers reached over gRPC: one HTTP/2 connection to the configured URL, one stream per session. The intended peer is monty-server (the production server: one monty subprocess worker per stream, plus capacity/timeout policy and graceful drain), but any server that admits the stream and bridges it to a worker fits — e.g. the dev relay scripts/grpc_relay.py.

Like AsyncMonty, but instead of spawning local subprocesses each checkout opens a stream to the configured URL. There is no sync counterpart — remote turns are network-bound. checkout() yields the same AsyncMontySession. AsyncMontyWebsocket is the same class under its former name.

A monty-server enforces its own policy on top of the pool’s. On SIGTERM drain it answers the session’s next request with MontyShutdown, whose dump restores the session onto another server; every other server-side drop (idle, session or turn timeout, capacity) ends the stream with a status and raises MontyDisconnectError naming its cause; a stream it refuses to open raises MontyRefusedError the same way.

A server that stores sessions gives each one an ID (await session.get_session_id()) instead of handing state back: dump() writes the current state to a record that never changes and returns its ID, and load_session / load_snapshot start a new session from either ID, in any process. checkout(ephemeral=True) opts a session out. When such a server drains, the session resumes on another server without the caller noticing (see auto_resume).

async with AsyncMontyRemote('http://127.0.0.1:8799') as pool:
    async with pool.checkout() as session:
        result = await session.feed_run('1 + 1')

Methods

__new__
def __new__(
    cls,
    url: str,
    *,
    max_processes: int | None = None,
    checkout_timeout: float | None = None,
    request_timeout: float | None = 10.0,
    metadata: Callable[[], Mapping[str, str]] | None = None,
    feed_duration_limit_grace: float | None = 1.0,
    turn_duration_limit_grace: float | None = 1.0,
    auto_resume: bool = True,
    max_decode_bytes: int | None = None,
    connect_headers: Callable[[], Mapping[str, str]] | None = None,
    stream_async_calls: bool = True,
) -> Self

Configure a remote worker pool; the connection is made by the first checkout (no workers are pre-warmed).

Returns

Self

Parameters

url : str

http:///https:// URL of the server — a relay, or any server that bridges a stream to a worker. ws:///wss:// are read as those two. An https:// certificate is verified against the operating system’s trust store.

max_processes : int | None Default: None

Cap on concurrent streams (defaults to the CPU count); checkouts beyond it wait.

checkout_timeout : float | None Default: None

Seconds checkout() waits for capacity before raising TimeoutError. None waits forever.

request_timeout : float | None Default: 10.0

Hard per-turn deadline in seconds (default 10.0) — a worker that exceeds it has its connection killed and the call raises MontyCrashedError with timed_out=True. It also bounds opening the stream, including the server’s admission (30 seconds when None). CPU-mode servers require a finite value; otherwise checkout raises MontyConfigurationError. Pass None to wait indefinitely with wall-mode servers. Under contention, this wall deadline can expire before a server CPU budget. Choose it for the expected budgets and scheduling delays.

This deadline does not set the sleep cap. System sleeps return normally after at most sleep_system_max (default ten seconds), even if sandbox code asks for longer, unless max_total_sleep_secs would be exceeded. Use os_policy with sleep='call_host' and OSAccess(max_sleep=None) to handle full sleeps.

Note that install_dependencies is a turn too, so the default 10.0 is often too low for it — a real uv pip install can exceed it. Raise request_timeout when installing dependencies over the remote transport.

max_decode_bytes : int | None Default: None

Per-frame host allocation budget in bytes. None keeps the protocol’s 1 GiB ceiling. Values above 1 GiB are rejected at construction. Excludes wire buffers and Python conversion. Exhaustion ends the session with a protocol error without replay. Also caps each message’s size, clamped between 64 KiB and the 256 MiB frame limit.

stream_async_calls : bool Default: True

Let a call to an async def host function that the sandbox does not await at once go to the host without suspending the worker, saving a network round trip per call; errors from such calls are raised at the await. On by default, unlike the subprocess AsyncMonty. See starting calls without waiting.

connect_headers : Callable[[], Mapping[str, str]] | None Default: None

The former name of metadata, accepted for one release with a DeprecationWarning; passing both raises TypeError.

metadata : Callable[[], Mapping[str, str]] | None Default: None

Called once per session, as it is entered and before any wait for pool capacity, to produce extra str to str request metadata for that session’s stream — e.g. a token for infrastructure in front of the worker. It runs synchronously on the checking-out task, so it sees that task’s contextvars and must not block. Monty never interprets the values; names are lowercase ASCII and values ASCII, duplicate names are last-wins, also over the traceparent the Logfire integration adds, a user-agent entry is put in front of the default one, and a malformed name or value raises MontyConfigurationError as the session is entered.

feed_duration_limit_grace : float | None Default: 1.0

Client wall-time grace beyond max_feed_duration_secs when the server uses wall time. CPU mode uses the server’s supervisor and grace instead. None disables this client backstop. request_timeout stays independent.

turn_duration_limit_grace : float | None Default: 1.0

The same, for max_turn_duration_secs.

auto_resume : bool Default: True

When a server that stores sessions drains one, open a new stream, load the state it named into a new session and re-send the request it did not run, instead of raising MontyShutdown; the session’s suspension and sleep totals carry over. MontyShutdown is still raised when the resume fails. An ended stream is never resumed: the request may have run.

checkout
def checkout(
    *,
    script_name: str = 'main.py',
    limits: ResourceLimits | None = None,
    type_check: bool = False,
    type_check_stubs: str | None = None,
    type_check_format: TypeCheckFormat | None = None,
    type_check_color: bool = False,
    assert_message_annotations: bool | int = ...,
    print_flush_interval: float | None = None,
    os_policy: OSPolicy | None = None,
    external_modules: Mapping[str, ExternalModule | Callable[[], Awaitable[ExternalModule]]] | None = None,
    ephemeral: bool | None = None,
    profile: str | None = None,
) -> AsyncMontySession

Prepare a REPL session served by a dedicated remote stream.

Identical to AsyncMonty.checkout, except for ephemeral and profile; the stream is opened by async with on the returned session.

Returns

AsyncMontySession

Parameters

ephemeral : bool | None Default: None

Whether a server that stores sessions may store this one. True means the server never stores it on its own and it gets no session_id; False asks for it to be stored; None takes the server’s default. Servers that store nothing ignore it.

profile : str | None Default: None

The name of the server-side profile to run the session under; None takes the server’s default. Sent to the server as-is; the monty worker itself ignores it.