Repository navigation
feat(mcp): search_attributes query for specific attributes - #2209
nikkikapadia wants to merge 17 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Nice shape overall: the fallbacks are well scoped, toAttributeMatches drops invented keys, and the cut-off/hint path has real tests. Two things before this goes to wider review:
- Shortlist prompt size in large orgs. In
search.ts,shortlistAttributesAgentgets every row: the base fetch plus all hint results. The endpoint returns up toexplore.trace-items.keys.max+ 1 keys per type (default 1000, across string/number/boolean, plus array behind a flag), so the orgs that triggermayBeCutOffare the same ones that send roughly 3k+ attribute lines, with descriptions, in a single prompt on everyquerycall. Those orgs are the main audience for this feature. Most of the base page is just the first ~1000 keys by name, so it's mostly noise once hints have run.datasetAttributesin search-events pages at 50 fields for this reason. Something like this would help: when the list was cut off, shortlist only the hint rows plus the recommended/convention keys from the base page, or put a hard cap on the lines sent. It would also help to log or measure the prompt token count on that path. - No model-in-the-loop coverage. Both prompts in
config.tsare what the feature depends on, but the unit tests mock both agents, and nothing inmcp-server-evalsrunssearch_attributeswith aquery. One or two evals would show that the hint and shortlist prompts work together. Use a cut-off-sized fixture where the right key only appears through a hint search (e.g. "payment provider" →checkout.payment_processor). That would cover the first point too.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1cc7198. Configure here.
| recommendedKeys.has(attribute.key), | ||
| ); | ||
| rows = mergeByKey([recommendedRows, ...hintRows]); | ||
| narrowedByHints = true; |
There was a problem hiding this comment.
Recommended keys dropped past cutoff
Medium Severity
When the attributes page looks cut off, recommended keys are taken only from that truncated first fetch. Common fields that sort after the cap never enter the hint-narrowed set, even though the fallback copy still says Sentry's common search fields are included.
Reviewed by Cursor Bugbot for commit 1cc7198. Configure here.
There was a problem hiding this comment.
this is fine imo because it's just the recommended keys and not the attributes queried from the hint keys.
lzhao-sentry
left a comment
There was a problem hiding this comment.
lgtm, i left two minor comments
| * Map the agent's keys back to fetched attributes. Drops keys that weren't | ||
| * fetched (invented or altered by the model) and repeats, then applies the limit. | ||
| */ | ||
| export function toAttributeMatches( |
There was a problem hiding this comment.
Minor thing: I noticed when querying for attributes i.e., "HTTP status code" the call usually returns the deprecated http.status_code as the best match. I don't think it's an issue because we get the alternative key to use as well, but maybe one thing to look into is to best match non deprecated attributes over deprecated ones?
There was a problem hiding this comment.
hrmm good point! this could probably be fixed in the way we ask the agent to display things. I'll take a look at either the subagent prompt or the return formatting.
| // The endpoint sorts keys by name and stops at about 1000 per type, so in a | ||
| // large org the right key can be past the cut. A substring search looks | ||
| // through every key, so search again with hints made from the query. | ||
| if (mayBeCutOff(rows)) { |
There was a problem hiding this comment.
I think this might be more of the API endpoint is weird, but in small orgs, when I search for "span duration" it doesn't successfully get span.duration. I think it's because built in attributes can be cut off for for some unholy reason?
I don't think it's a blocker for this PR, but we should file an ticket to investigate it
There was a problem hiding this comment.
def weird. The way it works rn is that we match the built in fields to what comes back in the endpoint. Seems like span.duration isn't being sent in any of their spans then. It is kind of weird behaviour tho. Maybe we can look at putting in these recommended attributes regardless in the near future?
There was a problem hiding this comment.
ah actually, might be because i didnt send any test data recently to my test org so it was outside the period 😭, lemme check on that does have it to see if its an issue
Building on top of what we currently have on the
search_attributestool; in addition to showing a list of all attributes, we now find attributes based on a user query. Users can define what attribute they're looking for and the tool will provide the best match and additional candidates.To achieve this behaviour we pass the query to a couple of subagents. The first agent will look at the user's query and provide substring hints that we can pass to the attributes endpoint. This happens because the attributes endpoint will return a limited number of attributes; so the attribute we're looking for may not be there. It will generate between 3-5 hints, query the attributes endpoint with these substrings and then merge those results with our recommended attributes. NOTE: If it notices that the total number of attributes is less than the attribute endpoints limit, it assumes those are all the attributes and will not generate these hints. This list of attributes gets formatted and passed to the second subagent. That subagent scans through all the attributes and gets max 10 candidates that the user might be talking about.
At the end we format the results and send it back to the agent that called the tool with suggested presentation but ultimately how that agent presents the information is up to them.
Some file guidance:
agents.ts-> all of the subagent callsconfig.ts-> the prompts that are put into the respective subagentsformat.ts-> output formatting functions that format the results that get handed back to the caller agentsearch.ts-> tooling decisions and returns to the caller agentCloses EXP-1353