Skip to content

feat: detect the coding agent when the plugin agent is unknown - #167

Open
arorashivam96 wants to merge 1 commit into
mainfrom
u/shivamarora/detect-coding-agent-for-unknown
Open

arorashivam96 wants to merge 1 commit into
mainfrom
u/shivamarora/detect-coding-agent-for-unknown

Conversation

@arorashivam96

Copy link
Copy Markdown
Contributor

Summary

When .env has no usable agent, scripts/auth.py now reports the coding agent that actually started the process instead of agent=unknown. This is the plugin-side half of PowerPlatform-DataverseClient-Python#209. It also covers the raw-HTTP path (get_plugin_headers), which never goes through the SDK.

Why

agent=unknown is the second-largest series in plugin traffic: 1,769,098 requests from 404 users between Oct 29 and Nov 4. The fallback comes from this helper. Since v1.14.1, _current_agent() maps a missing or unrecognized DATAVERSE_PLUGIN_AGENT to unknown.

Telemetry shows the agent is often known to the same caller on a different path:

  • 32.5% of the Python-path unknown requests come from users whose MCP or CLI traffic in the same window names a standard agent. The agent was running, but its name never reached the Python process.
  • 38% of unknown arrives through get_plugin_headers (raw urllib), so an SDK-only fix cannot reach it.

What changes

_current_agent() keeps its current behaviour when .env names an allowed agent. Detection runs only when the agent is missing, unknown, or not on the allowlist. It then checks the environment variables coding agents set on the processes they start, in the GitHub CLI's order (internal/agents/detect.go):

Marker Reported agent
AI_AGENT (cross-vendor; checked first) mapped to the plugin's agent names, e.g. claude -> claude-code, github_copilot_vscode_agent -> copilot, gemini -> gemini-cli
CODEX_THREAD_ID / CODEX_SANDBOX / CODEX_CI codex
GEMINI_CLI gemini-cli
COPILOT_CLI copilot
ANTIGRAVITY_AGENT antigravity-cli
CLAUDECODE / CLAUDE_CODE claude-code
CURSOR_AGENT, or CURSOR_EXTENSION_HOST_ROLE=agent-exec cursor

The rules are deliberately narrow:

  • An agent named in .env is never overridden.
  • A /<surface> suffix is kept, so unknown/jetbrains becomes codex/jetbrains under Codex.
  • Only names already on _ALLOWED_AGENTS are reported, so the context schema stays closed. An AI_AGENT value outside the map, or Amp (AGENT=amp), stays unknown.
  • CURSOR_TRACE_ID is ignored, because it is set in every Cursor terminal, including commands a person types by hand.
  • Nothing here raises, so attribution still cannot block a data operation.

The change applies to get_client, get_plugin_headers and get_cli_context. Plugin version: 1.18.0 -> 1.19.0 in all manifests, including the two Gemini manifests.

Testing

  • python -m pytest .github/evals/test_auth.py: 80 passed (9 new cases in AgentDetectionForUnknown), with and without an agent marker in the shell.
  • python .github/evals/static_checks.py: passed.
  • python .github/evals/version_bump_check.py: passed (minor).

Live run against a Dataverse environment (aurorabapenv39ef7.crm10.dynamics.com), using this branch's auth.py and the SDK from #209. Each User-Agent is the header actually sent, captured from the request; every call returned 200:

Path Environment User-Agent sent
get_plugin_headers (urllib) no .env agent, CLAUDECODE=1 Python-urllib (app=dataverse-skills/0.0.0-livetest;skill=dv-query;agent=claude-code)
get_plugin_headers (urllib) no .env agent, no markers Python-urllib (...;agent=unknown)
get_client (SDK) .env unknown/jetbrains, CODEX_THREAD_ID DataverseSvcPythonClient:0.1.0b10 (...;agent=codex/jetbrains)
get_client (SDK) .env cursor, CLAUDECODE=1 DataverseSvcPythonClient:0.1.0b10 (...;agent=cursor)

The fix does not depend on #209. The helper sets the agent before the context reaches the SDK, so it works with any SDK version the plugin already supports.

auth.py now fills a missing, unknown or unrecognized DATAVERSE_PLUGIN_AGENT from the environment markers coding agents set on the processes they start (AI_AGENT, CLAUDECODE, CODEX_THREAD_ID, GEMINI_CLI, COPILOT_CLI, ANTIGRAVITY_AGENT, CURSOR_AGENT), following the GitHub CLI's detection order. An agent named in .env is never overridden, any /<surface> suffix is kept, and only allowlisted agents are reported. Applies to get_client, get_plugin_headers and get_cli_context. Bumps the plugin to 1.19.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant