Data Programs

A data program is a named, stored, agent-authored JS script that fetches, joins, filters, or aggregates data — provider APIs via providerFetch / providerFetchAll / providerSearchAll, app data via appAction, or Resources-backed workspace files — and calls emit(rows, schema) exactly once with the result. The result is cached in SQL and any chart/table component can render it, refreshed on a policy you choose instead of on every page view.

It ships automatically wherever run-code is available — there is nothing to mount or configure per app. If your app already has run-code enabled, save-data-program, preview-data-program, run-data-program, list-data-programs, get-data-program, and delete-data-program are already registered actions.

Why this exists

Without a stored data program, "build a live view of X" tends to end in one of two bad places:

• A hardcoded action per vendor/query shape. Brittle, and it can't anticipate every customer's custom properties, a cross-provider join, or an arbitrary filter you didn't think to build in.
• Re-running the same fetch/join logic from scratch on every page view. Slow, and it re-spends the same provider API quota every time someone looks at a chart.

A data program is the middle ground: write the fetch/join/aggregate logic once as ordinary run-code, save it, and let the cache and refresh policy decide how often it actually needs to re-run.

When to use one

A comparison against the other run-code-shaped options:

Authoring workflow

1. Prototype in chat first. Write the fetch/join/aggregate logic as a normal run-code script and confirm it returns the rows you expect.
2. Change it to call emit(rows, schema?) exactly once at the end instead of returning or printing the result.
3. Save it with save-data-program. This dry-runs the code with defaultParams before persisting and rejects the save with a structured error if it fails, so a broken program is never stored. On success it returns { programId, rowCount, columns, sampleRows } — treat that as proof the program produces the rows you expect before telling the user it's ready.
4. Bind a dashboard panel to it, or call run-data-program directly from your own read path. There is no dashboard-specific plumbing baked into the primitive itself — the analytics template's "program" panel source (below) is one adoption, not a required pattern.
5. Iterate without persisting via preview-data-program({ code, params? }) — same dry-run path, no stored row.

A realistic example

Suppose an app wants a live "recent high-value support tickets" table sourced from a support provider, filtered to a threshold that isn't exposed by any built-in action:

// The program's stored `code` — ordinary run-code, with one emit() call.
const minValue = Number(params.minValue || 5000);

const tickets = await providerFetchAll("acme-support", "/v1/tickets", {
  query: { status: "open" },
  itemsPath: "data",
  pagination: {
    cursorParam: "cursor",
    nextCursorPath: "next_cursor",
    maxPages: 10,
  },
});

const rows = (tickets.items || [])
  .filter((t) => Number(t.contract_value || 0) >= minValue)
  .map((t) => ({
    ticket_id: t.id,
    subject: t.subject,
    account: t.account_name,
    contract_value: Number(t.contract_value || 0),
    opened_at: t.created_at,
  }));

emit(rows, [
  { name: "ticket_id", type: "string" },
  { name: "subject", type: "string" },
  { name: "account", type: "string" },
  { name: "contract_value", type: "number" },
  { name: "opened_at", type: "string" },
]);

Saving it:

await appAction("save-data-program", {
  name: "high-value-open-tickets",
  title: "High-Value Open Tickets",
  description:
    "Open support tickets above a configurable contract-value threshold.",
  code: "/* the script above */",
  defaultParams: { minValue: 5000 },
  refreshMode: "ttl",
  refreshTtlMs: 300_000,
});

The response carries { programId, rowCount, columns, sampleRows } — the dry-run's proof-of-done. From there, run-data-program({ programId, params: { minValue: 10000 } }) re-runs it (or serves the cache) for a different threshold, and a dashboard panel can bind to programId the same way.

The sandbox surface

A data program runs through the exact same executeSandboxCode sandbox as run-code — no new sandboxing, credential, SSRF, or quota code exists for this primitive. It gets the same globals (providerFetch, providerFetchAll, providerSearchAll, appAction, webFetch, workspace* Resources helpers) plus two additions from a small framework-injected prelude:

• params — a frozen global object holding the params for this run (defaultParams merged with whatever was passed to run-data-program or the panel descriptor). Reading it is fine; mutating it throws.
• emit(rows, schema?) — call this exactly once, at the end. A second call throws. rows must be an array of plain objects. schema, when provided, is { name: string, type: string }[]; when omitted it is inferred from the first 50 rows (a column is "json" if rows disagree on primitive type across the sample, otherwise "number" / "string" / "boolean"). console.log remains completely free for debugging — the runner captures combined stdout/stderr and only strips the sentinel line that emit() writes when parsing the result.

Caps enforced on every run (not configurable per-program):

Truncation is always explicit (truncated: true on the result) — never a silent drop.

Params and dashboard panel binding

Programs declare an optional paramsSchema (JSON Schema describing accepted params) and defaultParams. A dashboard adopts the "program" panel source by putting a small JSON descriptor in the panel's query field instead of a query string:

{ "programId": "dp_a1b2c3d4e5f6", "params": { "minValue": 10000 } }

The panel renders through the exact same chart/table components as every other panel source — it only ever receives { rows, schema } back. Binding a panel to a program is just "which stored program, with which param overrides"; there's no dashboard-specific execution path duplicated anywhere else.

Refresh, TTL, and caching semantics

Every (programId, paramsHash) pair has its own run history in data_program_runs, and paramsHash folds in the viewer, not just the params (see the security section for why). What happens on each read:

Reach for background: true only for programs that routinely exceed the foreground timeout — large multi-provider joins, deep pagination. Everything else should stay foreground: it's simpler to reason about and serves fresh data faster.

Why the cache is scoped per viewer

providerFetch inside a program resolves auth using the calling viewer's own request context — never the program author's. That is true whether the call comes from the agent, a run-data-program call, or a dashboard panel being viewed by a teammate. Two different viewers running the same program with the same params can legitimately get two different results: one has a configured HubSpot key and the other doesn't, or row-level provider permissions differ per person.

Because of that, hashDataProgramParams folds the caller's identity into the cache key alongside the canonicalized params. This matters most for shared dashboards: if the cache were keyed on params alone, the first person to view a panel would silently determine what every other viewer sees, including a viewer without the credentials that produced that cached result. Per-viewer scoping means a teammate missing a credential sees their own auth error on that panel, not someone else's data.

Security model

• Server-side only. A data program's code executes exclusively inside executeSandboxCode on the server. Neither the stored source nor any credential it touches is sent to the browser or to the model — only the emitted rows/schema and compact run metadata (row count, columns, sample rows) are.
• Credentials are always the calling viewer's, never the program's original author's. See Refresh, TTL, and caching semantics above for why this also drives the cache key.
• allowPublic: false is a hard invariant, not a default. Data programs register with the sharing registry the same way Extensions do, with public sharing disabled. A public program would let any authenticated user run arbitrary stored code under their own token — sharing a program is scoped to your org, and only ever means "teammates can view its cached result and trigger reruns," never "the general internet can execute this."
• Mutating and executing actions are not tool-callable from the sandboxed extension/iframe bridge. save-data-program, preview-data-program, run-data-program, and delete-data-program are toolCallable: false — only the agent or a server-side read path can invoke them, never an extension iframe under the opener's session. list-data-programs and get-data-program are read-only and access-scoped through the same sharing helpers as every other ownable resource.
• Programs are scoped to the app that created them. Passing appId to a lookup means a program created in one app is not runnable from another app's agent chat in a shared-database deployment, even for a caller who otherwise has row-level (owner/org/share) access to it.
• Only bind to or view a program you trust. Because execution always uses the viewer's own credentials, viewing a shared program's result is exactly like opening someone else's extension — trust the code, not just the access grant.

The Risk Meeting worked example

The analytics template ships a concrete, runnable case study: the ensure-risk-meeting-dashboard action idempotently installs two data programs plus a two-panel "Risk Meeting" dashboard, demonstrating three things a hardcoded action can't anticipate for every customer.

risk-meeting-cohort.js — searches for a configurable cohort of CRM deals by an arbitrary risk_status property (an IN-search over custom values a canned action can't anticipate), resolves each deal's company through a batched association lookup, resolves each company's domain, fetches a second, unrelated provider's account sentiment, and joins the two by domain:

// params: { riskStatuses?: string[], enterpriseOnly?: boolean }
const riskStatuses = params.riskStatuses?.length
  ? params.riskStatuses
  : ["On the Radar", "Churn Risk", "Confirmed Churn", "No Save Attempted"];

// 1. Arbitrary property IN-search — not expressible through a canned filter.
const dealSearch = await providerFetchAll("crm-provider", "/deals/search", {
  method: "POST",
  body: {
    filterGroups: [
      {
        filters: [
          { propertyName: "risk_status", operator: "IN", values: riskStatuses },
        ],
      },
    ],
  },
  itemsPath: "results",
  pagination: {
    cursorBodyPath: "after",
    nextCursorPath: "paging.next.after",
    maxPages: 20,
  },
});

// 2-3. Batched deal -> company -> domain resolution (elided here; see the
// real seed file for the full batching helper).

// 4. A second, unrelated provider — zero bespoke glue action required.
const accounts = await providerFetchAll("support-provider", "/accounts", {
  itemsPath: "data",
  pagination: {
    cursorParam: "cursor",
    nextCursorPath: "pagination.cursor",
    maxPages: 20,
  },
});

// 5. Join + emit.
const rows = dealSearch.items.map((deal) => ({
  deal_id: deal.id,
  risk_status: deal.properties.risk_status,
  arr: Number(deal.properties.total_contract_value || 0),
  // ...domain lookup + sentiment join...
}));

emit(rows, [
  { name: "deal_id", type: "string" },
  { name: "risk_status", type: "string" },
  { name: "arr", type: "number" },
]);

risk-meeting-pylon-early-warning.js — the complementary view: accounts already flagged at-risk by support sentiment that have not yet shown up in the CRM cohort above. That gap — support signal outrunning CRM signal — requires excluding one provider's cohort from another's, which is exactly the kind of cross-provider set logic no single-vendor action can express.

Both programs are installed and kept current by ensure-risk-meeting-dashboard, which:

• Upserts each program by its stable name, so re-running the installer updates the same rows instead of duplicating them.
• Requires no CRM/support credentials at install time — provider auth resolves per-viewer only when a panel is actually rendered or run-data-program is called, consistent with the per-viewer cache scoping above.
• Builds a two-panel dashboard whose panels use source: "program", bound to each program's id.

See the data-programs skill for the full annotated source.

How other apps adopt the panel source

The "program" dashboard panel source is one adoption of the primitive, implemented entirely in the host app's own panel-query layer — nothing dashboard-specific is baked into @agent-native/core/data-programs itself. The analytics template's implementation is a useful reference for wiring your own:

• The panel's sql column carries a small JSON descriptor — { programId, params? } — instead of a query string.
• Rendering a panel calls runDataProgram with triggeredBy: "panel_view" and the current viewer's identity, so the per-viewer cache and credential resolution apply automatically.
• On success, the panel gets { rows, schema } — the same shape every other panel source returns, so it flows through the existing chart/table components unmodified.
• On failure, a lastGoodRun is preferred over a broken panel whenever one exists; only when there is no prior success does the panel surface a structured error (with friendlier copy for common codes like access_denied, archived, and background_pending).

Any app can implement the same adoption for its own dashboards/tables: call run-data-program (or runDataProgram directly if you're inside the same server process) from your own read path, and hand the returned rows/schema to your own rendering components.

What's next

• Extensions — the sandboxed mini-app primitive data programs share a security posture and sharing model with, but with a UI of its own.
• Actions — the run-code conventions and sandbox globals (providerFetch, appAction, etc.) data programs are built on.
• Sharing & Privacy — the org-internal, never-public sharing model data programs use.
• Security — the framework's credential-handling and access-scoping invariants.