Trust Pipeline

The trust pipeline chains import → verify → publish into a single auditable flow. Every published snapshot carries a semantic fingerprint, optional stamp, and optional checks — making it easy to detect drift and guarantee idempotency.

Quick start (60 seconds)

# 1. One-command publish: CSV → .sheet → VisiHub
vgrid pipeline publish invoices.csv \
  --repo acme/invoices --headers --stamp "Q4 Close" --json

# 2. Same data, same fingerprint → no re-upload
vgrid pipeline publish invoices.csv \
  --repo acme/invoices --headers --stamp "Q4 Close" --json
# → {"already_published": true, ...}

That’s it. The pipeline imported the CSV into a canonical .sheet file, stamped it with a fingerprint, verified integrity, and published to VisiHub — all in one call. The second run detected the fingerprint match and returned immediately.

Step-by-step (manual control)

If you need more control, use the individual commands:

# 1. Inspect — evaluate formulas against raw data
vgrid sheet inspect data.csv --headers \
  --calc "SUM(Amount)" --calc "SUM(Tax)" --json > checks.json

# 2. Import — convert to canonical .sheet with stamp
vgrid sheet import data.csv data.sheet --headers --stamp "Q4 Filing"

# 3. Verify — confirm fingerprint hasn't drifted
vgrid sheet verify data.sheet

# 4. Publish — upload with trust metadata
vgrid hub publish data.sheet --repo acme/invoices \
  --checks checks.json --message "Q4 close" --json

---

vgrid pipeline publish

End-to-end command that chains inspect → import → verify → hub publish.

vgrid pipeline publish <source> --repo <owner/slug> [options]

Option	Description
--repo	VisiHub repository in owner/slug format (required)
--headers	Treat first row as column headers
--formulas	Formula handling for XLSX: values (default), keep, recalc
--stamp [label]	Stamp with provenance fingerprint + optional label
--checks-calc <expr>	Evaluate formula as a check (repeatable)
--checks-file <path>	Pre-computed checks JSON file
--delimiter	CSV field delimiter
--sheet	Sheet to import by index or name (XLSX only)
--message	Commit message
--notes	Path to markdown notes file
--out <path>	Save .sheet file to path (default: temp, deleted after publish)
--json	Output as JSON
--dry-run	Validate locally without auth or upload
--no-wait	Return immediately after upload (skip polling)
--timeout	Poll timeout in seconds (default: 120)

Examples

# CSV with header-aware checks
vgrid pipeline publish data.csv --repo acme/payments --headers \
  --checks-calc "SUM(Amount)" --checks-calc "SUM(Tax)" --json

# XLSX with specific sheet and formula preservation
vgrid pipeline publish report.xlsx --repo acme/quarterly \
  --sheet Revenue --formulas keep --stamp "Q4 2026" --json

# Dry run — no auth, no network, just validate locally
vgrid pipeline publish data.csv --repo acme/payments --headers \
  --dry-run --json

# CI: fast, no polling
vgrid pipeline publish data.csv --repo acme/payments --headers \
  --stamp --no-wait --json

# Save .sheet file for inspection
vgrid pipeline publish data.csv --repo acme/payments --headers \
  --out report.sheet --json

---

vgrid hub publish

Publish a verified .sheet file to VisiHub. Use this when you already have a .sheet file from sheet import or sheet apply.

vgrid hub publish <file.sheet> --repo <owner/slug> [options]

Option	Description
--repo	VisiHub repository in owner/slug format (required)
--message	Commit message (default: “Publish <filename>“)
--notes	Path to markdown notes file
--checks	Path to checks JSON (output from sheet inspect --calc)
--lock	Lock snapshot immutably (requires paid tier; not yet available)
--json	Output as JSON
--dry-run	Validate + compute fingerprint locally (no auth, no network)
--no-wait	Return immediately after upload (skip polling)
--timeout	Poll timeout in seconds (default: 120)

---

CI idempotent publish

The trust pipeline is designed for CI. Fingerprint-based idempotency means repeated runs with the same data are free — no duplicate revisions, no wasted uploads.

# GitHub Actions example
- name: Publish dataset
  run: |
    vgrid login --token ${{ secrets.VISIHUB_TOKEN }}
    vgrid pipeline publish data.csv \
      --repo acme/payments --headers --stamp \
      --no-wait --json > publish.json

    # Check result
    if jq -e '.already_published' publish.json >/dev/null 2>&1; then
      echo "No changes — skipped"
    else
      echo "Published revision $(jq -r '.revision_id' publish.json)"
    fi

Key CI flags:

Flag	Why
--json	Machine-parseable output on stdout (progress on stderr)
--no-wait	Don’t block on server-side processing — return after upload
--stamp	Enable fingerprint-based idempotency
--dry-run	Preflight validation without auth (useful in PR checks)

Idempotency behavior

When the latest revision on VisiHub was published via the trust pipeline (source_metadata.type == "trust_pipeline") and its fingerprint matches the local fingerprint:

• The command returns immediately with "already_published": true
• Exit code is 0
• No revision is created, no bytes uploaded

When the fingerprint differs (data changed), or the latest revision came from a different source (non-trust publish):

• A new revision is created and uploaded normally

JSON output schema

All trust pipeline JSON outputs include schema_version: 1. Fields:

{
  "schema_version": 1,
  "ok": true,
  "repo": "acme/payments",
  "revision_id": "123",
  "version": 5,
  "fingerprint": "v2:42:abc123...",
  "stamped": true,
  "stamp_matches": true,
  "locked": false,
  "dataset_url": "https://app.visihub.app/acme/payments",
  "revision_url": "https://app.visihub.app/acme/payments/revisions/123"
}

The already_published field is only present when the publish was skipped due to fingerprint match.

---

Source metadata contract

When a .sheet file is published via the trust pipeline (hub publish or pipeline publish), the revision on VisiHub stores source_metadata with this structure:

{
  "type": "trust_pipeline",
  "fingerprint": "v2:42:abc123...",
  "timestamp": "2026-02-12T14:30:00Z",
  "trust_pipeline": {
    "stamp": {
      "expected_fingerprint": "v2:42:abc123...",
      "label": "Q4 Filing",
      "timestamp": "2026-02-12T14:30:00Z"
    },
    "checks": {
      "format": "csv",
      "sheet": "Sheet1",
      "results": [
        {"expr": "SUM(Amount)", "value": "12345.67", "value_type": "number"},
        {"expr": "SUM(Tax)", "value": "1234.57", "value_type": "number"}
      ]
    },
    "notes": "Approved by Finance team."
  }
}

Fields

Field	Always present	Description
type	Yes	Always "trust_pipeline"
fingerprint	Yes	Semantic fingerprint of the .sheet data
timestamp	Yes	ISO 8601 timestamp of the publish
trust_pipeline.stamp	When --stamp used	Provenance stamp with expected fingerprint
trust_pipeline.checks	When --checks* used	Formula evaluation results
trust_pipeline.notes	When --notes used	Free-text notes

The fingerprint is content-based: the same data re-imported from a different file (or on a different machine) produces the same fingerprint. This is what enables idempotency — it’s based on what the data is, not where it came from.

---

Exit codes

Code	Meaning
0	Published successfully (or already published)
1	Verification failed or formula error
2	Bad arguments (wrong format, missing file, invalid repo)
40	Not authenticated (run vgrid login)
42	Network error
43	Server validation error
44	Timeout waiting for processing