Durable functions

What it is

A durable function checkpoints its progress and resumes from the last completed step, so one execution can run for up to 366 days instead of the seconds a normal invocation allows. Use one for work that has to survive a restart: a multi-step order pipeline, a long agent run, a nightly batch that calls out to a slow API.

Durable functions run locally and in the cloud on runtimes that ship the durable authoring API (JavaScript, TypeScript and Python). They are a separate collection from standard functions:

• You start an execution instead of invoking a function. The start returns a handle immediately; the result arrives later.
• Each execution is a resource with its own id, status, and result.
• The two collections never accept each other's names or ids. A function's kind is fixed when it is created.

How it relates

• Belongs to a project, and reads variables, databases, and storage in that project like any function.
• Its sources live in volcano/functions/ beside standard ones. Only volcano-config.yaml says which are durable, so volcano functions deploy --all skips them and volcano cloud durable deploy --all picks them up. -f deploys one the manifest does not mention, and refuses a name the manifest declares standard — a kind cannot be changed once the function exists.
• Every region the project deploys to has to offer durable execution, or the deploy is refused up front.
• Durable execution needs the durable authoring API. A source is deployed on the newest runtime for its language that ships one, which is not always that language's default — a Python durable function needs a newer runtime than a standard one gets. A language that has no such runtime at all is refused before anything is uploaded.

version: 1
project:
  name: my-app
functions:
  - name: hello
  - name: order-pipeline
    kind: durable

A durable entry takes variable_scope and variables like any other, and durable deploy sends what the manifest declares. Declaring nothing leaves an existing function's scope alone.

CLI operations

Wherever a command takes a function, it takes the name or the id. deploy needs exactly one of --all and -f. --yes skips the confirmation prompt the three destructive commands and the two approval decisions ask for. Every approvals, messages, and callbacks command takes --json. Schedulers require SUPERAGENT and are capped at 5 per project across standard and durable functions together; a create beyond that answers 403.

The table shows cloud commands. Remove cloud to use the same commands against the local environment, for example volcano durable deploy --all. Local waits resolve immediately by default while preserving checkpoint and replay behavior. Set LOCAL_DURABLE_REAL_TIME=true before volcano start when wait timing must match the deployed function.

Examples

volcano deploy deploys the durable functions after the standard ones, together with the rest of the project.

# Deploy every function volcano-config.yaml declares durable
volcano cloud durable deploy --all

# Deploy one, and let your project's signed-in users start it
volcano cloud durable deploy -f order-pipeline --visibility authenticated

# Start an execution and follow it
volcano cloud durable start order-pipeline --input '{"order_id":4417}'
volcano cloud durable executions get order-pipeline 66666666-6666-4666-8666-666666666666

# Only the executions still running
volcano cloud durable executions list order-pipeline --status running

# End one where it is
volcano cloud durable executions stop order-pipeline 66666666-6666-4666-8666-666666666666

--status takes the status as the API spells it: pending, running, succeeded, failed, timed_out, stopped or unknown. The dashboard labels those for reading, so a pending execution appears there as "Starting".

unknown is terminal, like the four outcomes before it: the platform could not establish how the execution ended, so it carries no result and nothing will settle it later. Retry it under the same name, which picks that execution back up rather than starting a second one.

Logs

durable logs reads the same two log streams the standard collection has, on a durable function:

# Why the deploy ended in "failed"
volcano cloud durable logs order-pipeline --type build

# What the function logged while its executions ran
volcano cloud durable logs order-pipeline --type runtime --follow

--type build reads the deployment the function is on, which is the one that just failed. Name an earlier deployment to read its build logs instead: volcano cloud durable logs order-pipeline <deployment-id> --type build.

--type runtime spans every execution, including every resume: the code between context operations runs again each time, so a line logged there appears once per resume while a line inside a completed step does not. To filter runtime logs by level, text or time, or read several durable functions at once, use volcano cloud logs --kind durable.

--follow works on both, and stops at a different point for each: following build logs ends when the deploy does, following runtime logs runs until you interrupt it. A function that has never deployed has no build logs, and --type build says so rather than printing nothing.

volcano cloud functions logs does not accept a durable function's name: the two collections never accept each other's names or ids.

Scheduling executions

A scheduler ticks on a cron expression and starts an execution instead of invoking the function, so every tick produces an execution you can list, follow, and stop like any other:

# Start one execution an hour, with the same input each time
volcano cloud durable schedulers create order-pipeline --cron "0 * * * *" --input '{"scope":"hourly"}'

volcano cloud durable schedulers list order-pipeline
volcano cloud durable executions list order-pipeline

Each tick names its execution after the run, so a tick Volcano has to retry resolves to the execution it already started rather than beginning a second one. Ticks draw on the same durable execution and operation allowances and the same concurrency cap a manual start does; a tick that would exceed the cap fails that run rather than queueing.

A tick's run counts as failed when its execution fails, times out, or is stopped. schedulers list shows each scheduler's health and how many runs in a row have failed, and the project is alerted when the failures continue; see notifications.md.

schedulers disable stops the ticks and leaves the scheduler in place; executions it already started keep running. schedulers delete removes the scheduler and its run history, and also leaves running executions alone — stop those with executions stop.

Approvals

A workflow that calls ctx.waitForApproval pauses until a person approves or denies it, then resumes with the decision. See durable functions in the JavaScript SDK for the authoring side. The approvals commands are the deciding side:

# What is waiting on someone
volcano cloud durable approvals list

# Read what the workflow is asking, with the details it attached
volcano cloud durable approvals get 0b6f3c1e-8a4d-4f7e-9c2b-5d1a7e3f9b20

# Decide it; the comment reaches the workflow and stays in the history
volcano cloud durable approvals approve 0b6f3c1e-8a4d-4f7e-9c2b-5d1a7e3f9b20 --comment "Checked stock"
volcano cloud durable approvals deny 0b6f3c1e-8a4d-4f7e-9c2b-5d1a7e3f9b20

ID: 0b6f3c1e-8a4d-4f7e-9c2b-5d1a7e3f9b20
Title: Ship order 4417?
Name: ship-order
Status: approved
Workflow: order-pipeline
Execution: order-4417 (66666666-6666-4666-8666-666666666666, running)
Requested: 2026-10-06T11:00:00Z
Expires: when its execution ends
Details: {
  "order": 4417
}
Decided By: owner@example.com
Decided At: 2026-10-06T11:04:12Z
Comment: Checked stock
✓ Approved "Ship order 4417?"; the workflow resumes with the decision

list shows pending approvals unless --status says otherwise. It takes pending, approved, denied, expired, cancelled, or all. An approval ends in one of four ways:

approve and deny show the title and ask before deciding, because a decision cannot be changed. Repeating the decision an approval already has changes nothing and succeeds, so a retried command is safe. A different decision, or any decision on an expired or cancelled approval, fails with what happened, for example approval … was already approved by owner@example.com at …. An approval past its timeout counts as expired, even while it still reads pending.

Approvals are decided by a person. A command run with a project access token is refused before it asks; run volcano login with VOLCANO_TOKEN unset, then decide again.

stats counts the approvals requested in a window, 30 days unless --since says otherwise, up to 366 days:

volcano cloud durable approvals stats --since 7d

It prints the count in each status, the approval rate (approved out of approved and denied), the median time from request to decision, and the workflows that request the most.

In local mode, drop cloud: volcano durable approvals approve … decides as the local user. An approval timeout runs in real time locally, even though plain waits resolve immediately.

Messages

A workflow that calls ctx.waitForMessage pauses until a message with that name arrives, and ctx.publish sends progress out of a running execution. See durable functions in the JavaScript SDK for the authoring side. The messages commands send to an execution and read what it received and published:

# What the execution is waiting for
volcano cloud durable messages waits order-pipeline 66666666-6666-4666-8666-666666666666

# Send it; the workflow resumes with the data
volcano cloud durable messages send order-pipeline 66666666-6666-4666-8666-666666666666 \
  payment-confirmed --data '{"amount":4200}' --idempotency-key pay-4417

# Watch what it publishes until it ends
volcano cloud durable messages list order-pipeline 66666666-6666-4666-8666-666666666666 \
  --direction out --follow

ID: 8f3b6c2a-1d4e-4a7b-9c5f-2e8d1a6b3c70
Seq: 3
Direction: in
Name: payment-confirmed
Status: queued
Sender: project
Sent: 2026-10-12T09:00:00Z
Data: {
  "amount": 4200
}
✓ Sent "payment-confirmed" to execution 66666666-6666-4666-8666-666666666666

A message sent before the workflow waits for its name is queued until it does, and waits take messages oldest first. Without --data or --data-file the workflow receives null. An execution that has ended takes no messages: the send fails with execution … has ended and takes no more messages, and queued messages nobody took are marked dropped.

--idempotency-key makes a send safe to retry. Repeating it with a key the execution has already seen prints the first message and Already sent with this idempotency key; nothing changed. Without the flag, send generates a key and prints it to stderr before sending, so a send whose outcome you did not see can be retried with that key. --json prints the message as the API returns it, the same for a repeat as for the first send.

list reads both directions oldest first: in for messages sent to the execution, out for the ones it published. Each message has a seq; --after reads only the ones after it, --limit takes 1 to 500 per request, and a long list ends with the command for the rest. --follow keeps reading until the execution ends; with --json it prints one message per line. It prints each message once, with the status it had when first read, so a sent message shown as queued stays that way after a wait takes it.

Anyone with access to the project can send and read messages, including a project access token. A read-only token can list but not send.

In local mode, drop cloud: volcano durable messages send … sends to an execution running on the local engine.

Callbacks

A workflow that calls ctx.waitForCallback hands a vcb_ token to another system and pauses until that system succeeds or fails the wait. See durable callbacks for the authoring side. The callbacks commands are the token holder's side:

# Still working: restart the wait's heartbeat timeout
volcano cloud durable callbacks heartbeat vcb_0000000000000000000000000000000000000000000

# Done: the workflow resumes with the result
volcano cloud durable callbacks succeed vcb_0000000000000000000000000000000000000000000 \
  --result '{"charge_id":"ch_1"}'

# Or it failed: the workflow raises CallbackFailedError with this type and message
volcano cloud durable callbacks fail vcb_0000000000000000000000000000000000000000000 \
  --type CardDeclined --message "Insufficient funds"

✓ Succeeded the callback; the execution resumes with the result

The token is the only credential these commands send, so they need no login or project. Whoever holds the token can answer the wait, so treat it as a secret. Pass - in place of the token to read it from stdin, which keeps it out of shell history and the process list:

printf '%s' "$CALLBACK_TOKEN" | volcano cloud durable callbacks succeed - --result-file charge.json

--result or --result-file is required: any JSON value other than null, up to 64 KiB. null is refused because a wait that times out resolves with null. fail requires --type, a name the workflow can branch on; --message is optional.

succeed and fail return once the execution has the outcome. When handing it over takes longer, they report that Volcano is still delivering it. Volcano finishes the delivery on its own, so do not send it again.

A token takes one answer. A later succeed, fail, or heartbeat fails with the callback already has an outcome, including a retry of the answer that went through, so a holder retrying after a lost response can treat it as done. Once the wait times out or its execution ends, every command fails and says so; nothing will take an answer, so do not retry. A heartbeat that cannot reach the execution right now leaves the wait unchanged; retry it with backoff.

In local mode, drop cloud: volcano durable callbacks succeed … answers a callback on the local engine. A callback's timeout and heartbeat timeout run in real time locally.

Starting is asynchronous

start returns as soon as the execution is accepted, with the execution's id and name. Read the execution to get its status and, once it has finished, its result:

volcano cloud durable start order-pipeline --input order.json --name order-4417
volcano cloud durable executions get order-pipeline <execution-id>

--input takes an inline JSON object or the path to a file holding one, and is handed to the function verbatim. Omit it to start with no input at all, which is not the same as starting with {} — and not the same as --input null, which is refused, because it would arrive as neither.

--name is the execution's idempotency key. Starting again under a name that already names an execution returns the existing one instead of beginning a second, and is not charged again — so a retried start is safe. Omit it and Volcano generates one. A name is letters, digits, -, _ and ., up to 255 characters; anything else is refused.

A deploy finishes asynchronously, so a start that follows one straight away can be refused while the function is still provisioning. Wait for get to report active and start again.

Visibility

Visibility decides who can start executions of a function:

A caller a private function refuses gets 404 with durable function not found, the same as for a function that does not exist. If your app gets 404 for a durable function you deployed, check its visibility with volcano cloud durable get <name>.

A durable function is never invocable over HTTP the way a public standard function is, whatever its level: starting an execution is all the level allows. An anon key reads and stops only the executions whose client_token it sends, and a signed-in user also the ones they started. See Durable functions.

A new durable function starts private, and durable get shows the level. To change it without rebuilding, declare visibility in volcano-config.yaml and run volcano cloud config deploy:

functions:
  - name: order-pipeline
    kind: durable
    visibility: authenticated

A redeploy with --visibility also sets the level, for the one function -f names. Omit the flag and a redeploy keeps the level the function has. When a deploy creates a function without --visibility, it prints both ways to let signed-in users in.

--public is the same as --visibility public. --private is no longer accepted: it used to let signed-in users in, which is authenticated now, so pick the level you mean.

--visibility is not accepted with --all. Visibility is a per-function decision, so one flag applied to a whole manifest would take a redeploy of every function to undo. Declare levels in the manifest instead.

Stopping and deleting

executions stop requests the end of one execution. Cancellation happens behind the request, so the execution it prints back often still reads running; executions get is how you watch it reach stopped. Steps already completed are not undone. Stopping one that has already finished reports the state it is in rather than failing.

durable delete tears down the function and its execution history. It does not wait for work in flight, so stop an execution you need ended first.