docs(config): explain where diagnostics come from and when a step needs diagnostic_format

[jdx]

https://entire.io/gh/jdx/hk/trails/196

hk check --sarif, the diagnostics arrays in --format json, and the MCP dashboard are empty for any step that doesn't set diagnostic_format, which includes every builtin today. The docs never said so: the only mention was a one-line field comment in Config.pkl, and the CI and agent pages promised "normalized diagnostics" without saying where they come from. A user who runs hk check --sarif on a project of builtins gets a SARIF file with zero results and no explanation.

This adds a "Diagnostics" section to the configuration guide that explains:

• diagnostics are parsed from a step's check output, and a step without diagnostic_format contributes none (it still runs, fails, and keeps its raw output);
• builtins set diagnostic_format only where the tool's default output is a supported format, so many don't;
• the four formats (gcc, sarif, eslint-json, cargo-json), what each expects, that hk parses combined stdout and stderr of check runs only, and that it never adds flags to the command;
• how diagnostic_tool and parse_warnings fit in, with a small example step.

It also expands the diagnostic_format comment in pkl/Config.pkl (the generated settings reference), and adds one sentence each to the CI and coding-agents pages pointing at the new section. The pirate variants of the three pages are updated and stamped.

Observed behavior the docs now describe, from a repo with the shellcheck builtin and a custom step with diagnostic_format = "gcc", both failing on the same file:

$ hk --format json check --all --no-fail-fast | jq '.steps | map({name, status, diagnostics: (.diagnostics | length)})'
[
  { "name": "shellcheck", "status": "failed", "diagnostics": 0 },
  { "name": "lint-gcc",   "status": "failed", "diagnostics": 1 }
]

hk check --all --no-fail-fast --sarif out.sarif writes one result, ruleId SC2006 from lint-gcc; the shellcheck step contributes none.

Docs only; no behavior change. hk pre-commit checks, including the docs build and pirate checks for the three pages, pass. docs/pirate/contributing.md is already stale on main and is untouched. The cargo-check hook step was skipped locally (HK_SKIP_STEPS=cargo-check) because of unrelated Cargo.toml warnings on main.

AI-assisted — Tool: Claude Code; model: anthropic/claude-sonnet-5-5; version: unavailable.

🤖 Generated with Claude Code

---

Note

Low Risk
Documentation and schema comments only; no changes to hk execution or diagnostic parsing logic.

Overview
This PR documents when hk produces normalized diagnostics (SARIF, JSON/JSONL diagnostics, MCP dashboard)—only steps with diagnostic_format contribute them; failing steps without it still expose raw output instead.

It adds a Diagnostics section to the configuration guide (and pirate mirror) covering opt-in parsing from check stdout/stderr, builtin defaults, the four supported formats, diagnostic_tool / parse_warnings, JSONL run_completed vs empty step_completed arrays, and a sample gcc step. The diagnostic_format field comment in pkl/Config.pkl is expanded to match.

The CI and coding agents pages now note that --sarif and structured diagnostics depend on diagnostic_format, with links to the new section. Pirate doc sourceHash stamps are refreshed; no runtime behavior changes.

^{Reviewed by Cursor Bugbot for commit a1007ce. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

• Documentation
  • Clarified when normalized diagnostics are available in SARIF, JSON, JSONL, and the MCP dashboard, and documented supported formats and parsing behavior.
  • Explained how to inspect raw step output when diagnostics are not configured or parsing produces warnings.
  • Added configuration guidance for diagnostic tool names, output retention, and a GCC-format example.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 6 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
• Review profile: CHILL
• Plan: Advanced
• Run ID: da2b4511-eeed-4623-9d3e-71b8173849f1

📥 Commits

Reviewing files that changed from the base of the PR and between 5049b97 and a1007ce.

⛔ Files ignored due to path filters (1)

• pkl/Config.pkl is excluded by !**/*.pkl

📒 Files selected for processing (6)

• docs/agents.md
• docs/ci.md
• docs/configuration.md
• docs/pirate/agents.md
• docs/pirate/ci.md
• docs/pirate/configuration.md

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 5/5

[Low risk] Documentation updates explaining diagnostics configuration.

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains.

Summary

The PR explains why steps without diagnostic_format produce no normalized diagnostics and documents the supported formats and output behavior.

• Adds configuration guidance and a Pkl schema comment.
• Links to that guidance from the CI and coding-agent pages, with corresponding pirate-page updates.

_{Reviews (5) · Last reviewed commit: "Merge origin/main into docs/diagnostics-..."}

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

[github-actions]

Instruction counts

benchmark	trend	instructions	Δ	wall (min)	Δ
builtins	▃▂█▆▁▄	1,557,480 → 1,558,626	+0.07%	2.29 → 2.19ms	-4.51%
check	—	—	—	15.30 → 15.81ms	+3.31%
usage	▆▆█▁██	4,712,152 → 4,713,430	+0.03%	3.46 → 3.53ms	+1.97%
validate	—	—	—	4.47 → 5.05ms	+13.03%

No instruction-count regression above 1%.

_{Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.}

_{Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.}

_{a1007ce3d7ae vs 5049b9741444 · measured on the runner, not pushed to the history.}