From 7689cb4c088f02953b460b277494737e3062a125 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Tue, 20 Oct 2026 14:29:41 +0200 Subject: [PATCH 01/14] perf(trace): capture page state only for commands that change it Trace mode took a per-action capture (page source + screenshot, plus DOM on web) for reads and waits as well as actions, because the capture gate was mapCommandToAction, the timeline-rendering table. 5c54ae4 added the reads to that table so they would render as rows, which also made every read, and every poll of a waiting matcher, a capture. On iOS a capture is expensive; a reported suite made 187 page-source calls for 12 interactions and went from 3:10 to 7:10. ACTION_MAP's methods are now classified in a typed METHOD_EFFECT table (an unclassified method fails to compile) and changesPage() gates the capture in all three JS adapters. Reads keep their rows and inherit the latest earlier capture through claimAfter's fallback. The first traced command of each test is still captured so a test opening with reads has a frame. The WDIO service stamps its end-of-test capture at the test's last page change, otherwise that action's row resolves to the capture from before it. Co-Authored-By: Claude Opus 5.5 --- .changeset/capture-only-page-changes.md | 7 ++ CLAUDE.md | 2 +- packages/nightwatch-devtools/src/session.ts | 25 ++++- .../nightwatch-devtools/src/test-lifecycle.ts | 1 + .../tests/actionSnapshot.test.ts | 54 +++++++++ .../src/helpers/commandPostActions.ts | 19 +++- packages/selenium-devtools/src/session.ts | 7 ++ .../selenium-devtools/src/test-management.ts | 2 + .../tests/action-snapshot.test.ts | 51 ++++++++- packages/service/src/index.ts | 38 +++++-- .../tests/trace-action-capture.test.ts | 106 +++++++++++++++++- packages/shared/src/trace-actions.ts | 62 +++++++++- packages/shared/tests/change-page.test.ts | 37 ++++++ .../trace/tests/trace-frame-snapshots.test.ts | 19 ++++ 14 files changed, 404 insertions(+), 26 deletions(-) create mode 100644 .changeset/capture-only-page-changes.md create mode 100644 packages/shared/tests/change-page.test.ts diff --git a/.changeset/capture-only-page-changes.md b/.changeset/capture-only-page-changes.md new file mode 100644 index 000000000..944a63204 --- /dev/null +++ b/.changeset/capture-only-page-changes.md @@ -0,0 +1,7 @@ +--- +'@wdio/devtools-service': patch +'@wdio/selenium-devtools': patch +'@wdio/nightwatch-devtools': patch +--- + +Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for read and wait commands such as `getText`, `isDisplayed` or `waitForDisplayed`. Those rows still appear in the trace and now show the page as of the last command that changed it. Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). diff --git a/CLAUDE.md b/CLAUDE.md index 059c53f96..09c202845 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -387,7 +387,7 @@ Documented divergences from the conventions above. They exist today as debt to b - **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2. - A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch. - **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop. -- Residual: **a polling matcher fires one capture per poll** — each poll is a top-level mapped read — so "one capture per action" undercounts a real suite. Pre-existing in `d924a02`, unmeasured on a device, and the likeliest next lever. +- **Only a command that changes the page is captured** (`shared/trace-actions.ts` `changesPage`, read off the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit. - The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands. ### File-size (raw line counts; soft cap is 500 logic lines) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 903d10baa..6ec759919 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -16,6 +16,7 @@ import { type SessionCapturerOptions } from '@wdio/devtools-core' import { + changesPage, COLLECTOR_INJECT_SCRIPT, COLLECTOR_READY_SCRIPT } from '@wdio/devtools-shared' @@ -90,6 +91,13 @@ export class SessionCapturer extends SessionCapturerBase { log[level](message) ) + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + + resetTestCapture(): void { + this.awaitingInitialFrame = true + } + constructor( devtoolsOptions: SessionCapturerOptions = {}, browser?: NightwatchBrowser @@ -170,13 +178,18 @@ export class SessionCapturer extends SessionCapturerBase { * after the command completes, and export claims snapshots by exact equality * with the command timestamp (FrameSnapshotIndex.claimAfter / elementsAt). */ #queueActionSnapshot(command: string, timestamp: number): void { - if ( - this.traceMode !== 'trace' || - !this.#browser || - !mapCommandToAction(command) - ) { + if (this.traceMode !== 'trace' || !this.#browser) { + return + } + // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + const initialFrame = + this.awaitingInitialFrame && !!mapCommandToAction(command) + if (!changesPage(command) && !initialFrame) { return } + if (initialFrame) { + this.awaitingInitialFrame = false + } this.snapshotCaptures.push( captureActionSnapshot( this.#browser, @@ -187,6 +200,8 @@ export class SessionCapturer extends SessionCapturerBase { ).then((snap) => { if (snap) { upsertRichestSnapshot(this.actionSnapshots, snap) + } else if (initialFrame && !changesPage(command)) { + this.awaitingInitialFrame = true } }) ) diff --git a/packages/nightwatch-devtools/src/test-lifecycle.ts b/packages/nightwatch-devtools/src/test-lifecycle.ts index 4c1817043..508071e41 100644 --- a/packages/nightwatch-devtools/src/test-lifecycle.ts +++ b/packages/nightwatch-devtools/src/test-lifecycle.ts @@ -170,6 +170,7 @@ export function wrapBrowserOnce( ctx.isScriptInjected = true } ctx.browserProxy.resetTestTracking() + ctx.sessionCapturer.resetTestCapture() ctx.browserProxy.wrapBrowserCommands(browser) } diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index bcf033688..60ae95826 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -1,7 +1,17 @@ +import type * as DevtoolsCore from '@wdio/devtools-core' import { describe, it, expect, vi } from 'vitest' +import { captureActionSnapshot as coreCapture } from '@wdio/devtools-core' import { SessionCapturer } from '../src/session.js' import type { CommandLog, NightwatchBrowser } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + function makeMockBrowser(): NightwatchBrowser { return { url: vi.fn(async () => ({})), @@ -59,6 +69,50 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) + it('captures a click, but not a read or wait that follows it', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await at('waitForElementVisible', 3) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + expect(cap.commandsLog).toHaveLength(3) + cap.cleanup() + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + cap.resetTestCapture() + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 3]) + cap.cleanup() + }) + + it('retries the initial frame when the first read captures nothing', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce(null) + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await Promise.all(cap.snapshotCaptures) + await at('getText', 2) + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + cap.cleanup() + }) + it('does not capture outside trace mode', async () => { const cap = new SessionCapturer({}, makeMockBrowser()) await cap.captureCommand('click', [], undefined, undefined, { diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index d40c45132..5df0de1a1 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -11,6 +11,7 @@ import { type CapturedPerformancePayload, type RetryTracker } from '@wdio/devtools-core' +import { changesPage } from '@wdio/devtools-shared' import { ELEMENT_LOCATOR_METHODS } from '../constants.js' import { getDriverOriginals, getElementOriginals } from '../driverPatcher.js' import { captureOrReplaceCommand } from './captureOrReplaceCommand.js' @@ -362,14 +363,18 @@ function queueActionSnapshot( timestamp: number, error: unknown ): void { - if ( - ctx.options.mode !== 'trace' || - error || - !ctx.driver || - !mapCommandToAction(cmd.command) - ) { + if (ctx.options.mode !== 'trace' || error || !ctx.driver) { return } + // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + const initialFrame = + capturer.awaitingInitialFrame && !!mapCommandToAction(cmd.command) + if (!changesPage(cmd.command) && !initialFrame) { + return + } + if (initialFrame) { + capturer.awaitingInitialFrame = false + } ctx.snapshotCaptures.push( captureActionSnapshot( ctx.driver, @@ -379,6 +384,8 @@ function queueActionSnapshot( ).then((snap) => { if (snap) { upsertRichestSnapshot(ctx.actionSnapshots, snap) + } else if (initialFrame && !changesPage(cmd.command)) { + capturer.awaitingInitialFrame = true } }) ) diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index f8e68003e..4a54da528 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -55,6 +55,13 @@ export class SessionCapturer extends SessionCapturerBase { // A screenshot that reaches a driver being quit is skipped, not reported as a failure. #endingDrivers = new WeakSet() + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + + resetTestCapture(): void { + this.awaitingInitialFrame = true + } + #clientConnected = false #clientConnectedWaiters: Array<() => void> = [] #onClientDisconnected?: () => void diff --git a/packages/selenium-devtools/src/test-management.ts b/packages/selenium-devtools/src/test-management.ts index 285611487..b4df38756 100644 --- a/packages/selenium-devtools/src/test-management.ts +++ b/packages/selenium-devtools/src/test-management.ts @@ -150,6 +150,7 @@ export function startTest( } ctx.testManager!.startMarkedTest(name, resolvedMeta) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (file) { ctx.sessionCapturer?.captureSource(file).catch(() => {}) } @@ -202,6 +203,7 @@ export function startScenario( const file = meta.file ?? root?.file ?? process.cwd() ctx.suiteManager.startScenarioSuite(name, file, meta.callSource, meta.file) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (meta.file) { ctx.sessionCapturer?.captureSource(meta.file).catch(() => {}) } diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index 7e75bcd8a..0b3287698 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -1,3 +1,4 @@ +import type * as DevtoolsCore from '@wdio/devtools-core' import { afterEach, describe, expect, it, vi } from 'vitest' import { resetSignatureCounters } from '../src/helpers/utils.js' @@ -11,10 +12,21 @@ import { } from '../src/helpers/commandPostActions.js' import { captureActionSnapshot } from '../src/action-snapshot.js' import { getDriverOriginals } from '../src/driverPatcher.js' -import { RetryTracker } from '@wdio/devtools-core' +import { + RetryTracker, + captureActionSnapshot as coreCapture +} from '@wdio/devtools-core' import type { ActionSnapshot } from '@wdio/devtools-shared' import type { CapturedCommand, SeleniumDriverLike } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + const capturers: SessionCapturer[] = [] afterEach(() => { @@ -115,6 +127,43 @@ describe('selenium action-snapshot timestamp binding', () => { }) }) +describe('selenium action-snapshot capture gate', () => { + async function run(commands: string[], ctxBag = makeCtx(fakeDriver())) { + let t = 1 + for (const command of commands) { + await handleOnCommand(ctxBag.ctx, cmd({ command, timestamp: t++ })) + } + await Promise.all(ctxBag.snapshotCaptures) + return ctxBag + } + + it('captures a click, but not a read or wait that follows it', async () => { + const { actionSnapshots } = await run([ + 'click', + 'getText', + 'waitForDisplayed' + ]) + expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const bag = makeCtx(fakeDriver()) + await run(['getText', 'isDisplayed'], bag) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + bag.capturer.resetTestCapture() + await handleOnCommand(bag.ctx, cmd({ command: 'getText', timestamp: 9 })) + await Promise.all(bag.snapshotCaptures) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 9]) + }) + + it('retries the initial frame when the first read captures nothing', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce(null) + const bag = await run(['getText', 'isDisplayed', 'getText']) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + }) +}) + describe('selenium action-snapshot locator dialect', () => { it('injects the XPath text form, which By.xpath resolves', async () => { // `a*=Logout` is WebdriverIO-only syntax; a Selenium user pastes the diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index 2206b2dfe..6c0ee05d8 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -38,6 +38,7 @@ import { settleAfterLastAction } from './action-snapshot.js' import { + changesPage, isWaitCommand, type ActionSnapshot, type TestMetadataMap @@ -159,6 +160,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { /** Wall-clock ms at the current test's start, set in beforeTest/beforeScenario; * the lower bound of that test's video frame window (per-test slicing). */ #currentTestStartWallTime = 0 + #awaitingInitialCapture = true /** Map of testUid → metadata for trace group events and per-spec partitioning. */ #testMetadata: TestMetadataMap = new Map() @@ -380,6 +382,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true this.#currentStepIndex = 0 this.#currentStepUid = undefined @@ -413,6 +416,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { beforeTest(test?: { file?: string; title?: string; fullTitle?: string }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true // Track test identity for command tagging. Generate a stable UID // from file + title so commands can be partitioned across reruns. @@ -583,7 +587,11 @@ export default class DevToolsHookService implements Services.ServiceInstance { // action's has no successor and this is the only capture of it. A session // that ran no action names it FINAL_SNAPSHOT_COMMAND, which the per-test // screenshot reads as "post-teardown frame, do not use". - const lastAction = this.#lastAction() + const lastChange = this.#lastCommandWhere(changesPage) + const lastAction = + lastChange && lastChange.timestamp >= this.#currentTestStartWallTime + ? lastChange + : this.#lastAction() // A session with no action has no timestamp of its own to key on, so its // frame is recognised by the marker instead — otherwise `Date.now()` differs // between the per-test finalize and `after()` and both capture. @@ -640,20 +648,22 @@ export default class DevToolsHookService implements Services.ServiceInstance { * self-scopes, and a slot already filled at its own boundary converges on * `alreadyCaptured`. */ #lastAction(): { command: string; timestamp: number } | undefined { + return this.#lastCommandWhere(() => true) + } + + #lastCommandWhere( + predicate: (command: string) => boolean + ): { command: string; timestamp: number } | undefined { const commands = this.#sessionCapturer.commandsLog for (let i = commands.length - 1; i >= 0; i--) { const cmd = commands[i]! - if (this.#isActionCommand(cmd.command)) { + if (this.#isActionCommand(cmd.command) && predicate(cmd.command)) { return cmd } } return undefined } - #lastActionTimestamp(): number { - return this.#lastAction()?.timestamp ?? Date.now() - } - private resetStack() { this.#commandStack.reset() // No command is in flight after a reset; without this a window stranded by a @@ -766,8 +776,11 @@ export default class DevToolsHookService implements Services.ServiceInstance { // the previous action left behind. Taken HERE, before the command is issued, // because that is the one moment the driver is guaranteed idle and the app // at rest — a capture taken the instant a command returns catches whatever - // transition it started. Stamped at the previous action's end (or now, for - // the first, which makes it the initial frame). Trace mode only. + // transition it started. Stamped at the previous page-changing command's + // end (or now, for the first, which makes it the initial frame). Trace mode + // only. + // + // Reads/waits inherit the last page-changing capture; capturing them costs a page-source round trip. // // Not while a mobile-web Appium session has a document to probe — the // re-entrant probes can deadlock a serialising driver (#374). A native @@ -780,16 +793,18 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#browser, this.#sessionCapturer.currentContext ) && - this.#isActionCommand(command) + this.#isActionCommand(command) && + (changesPage(command) || this.#awaitingInitialCapture) ) { - // Stamped at the previous action's end so this capture IS that action's + // Stamped at the previous page-changing command's end so this capture IS that command's // result — except across a test boundary, where that slot already holds // its own finalize capture and a second frame at the same timestamp lets // the richer-screenshot merge replace it (with a reloadSession between // the tests, the row then replays the post-reload page). The first // capture of a test stamps now instead, the same rule the session's first // capture uses to become the initial frame. - const previousEnd = this.#lastActionTimestamp() + const previousEnd = + this.#lastCommandWhere(changesPage)?.timestamp ?? Date.now() const snap = await captureActionSnapshot( this.#browser, command, @@ -799,6 +814,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#sessionCapturer.currentContext ) if (snap) { + this.#awaitingInitialCapture = false upsertRichestSnapshot(this.#actionSnapshots, snap) } } diff --git a/packages/service/tests/trace-action-capture.test.ts b/packages/service/tests/trace-action-capture.test.ts index 188122e70..1f7b502c9 100644 --- a/packages/service/tests/trace-action-capture.test.ts +++ b/packages/service/tests/trace-action-capture.test.ts @@ -19,7 +19,7 @@ vi.mock('stack-trace', () => ({ /** * The command has to be in the log by the time `afterCommand` returns: - * `#lastActionTimestamp()` reads it to stamp the NEXT action's capture, and that + * `#lastPageChange()` reads it to stamp the NEXT action's capture, and that * stamp is what makes an action's result the following action's "before". */ const commandsLog: { command: string; timestamp: number }[] = [] @@ -202,6 +202,65 @@ describe('trace mode: one capture per action, taken before it', () => { expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) }) + describe('reads and waits do not capture', () => { + const run = async ( + service: DevToolsHookService, + commands: string[] + ): Promise => { + for (const command of commands) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + } + const started = async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + vi.mocked(captureActionSnapshot).mockClear() + return service + } + + it('captures only the page-changing commands of click, getText x3, click', async () => { + const service = await started() + await run(service, ['click', 'getText', 'getText', 'getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('click') + expect(namedAt(1)).toBe('click') + }) + + it('stamps the capture after a wait at the preceding click end', async () => { + const service = await started() + await run(service, ['click', 'waitForDisplayed', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) + }) + + it('takes an initial capture on a leading read, then one on the click', async () => { + const service = await started() + await run(service, ['getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('getText') + expect(namedAt(1)).toBe('click') + }) + + it('retries the initial capture when the first read captures nothing', async () => { + const service = await started() + vi.mocked(captureActionSnapshot).mockResolvedValueOnce(null) + await run(service, ['getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('takes the initial capture again for a new test', async () => { + const service = await started() + service.beforeTest({ file: '/spec/a.ts', title: 'one' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + service.beforeTest({ file: '/spec/a.ts', title: 'two' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + }) + it('captures the final state when the session ends', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) @@ -388,6 +447,51 @@ describe('trace mode: one capture per action, taken before it', () => { ]) }) + it('stamps the final capture at the test last page change, past later reads', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const test = { file: '/spec/a.ts', title: 'submit' } + service.beforeTest(test as never) + clock = Date.now() + 1000 + for (const command of ['click', 'getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(test as never, {} as never, {} as never) + + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + expect(namedAt(0)).toBe('click') + expect(stampedAt(0)).toBe(commandsLog[0]!.timestamp) + }) + + it('stamps the final capture at the last read when the test changed no page', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const first = { file: '/spec/a.ts', title: 'first' } + service.beforeTest(first as never) + await service.beforeCommand('click' as never, []) + await service.afterCommand('click' as never, [], undefined) + await service.afterTest(first as never, {} as never, {} as never) + + const second = { file: '/spec/a.ts', title: 'second' } + service.beforeTest(second as never) + clock = Date.now() + 1000 + for (const command of ['getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(second as never, {} as never, {} as never) + + expect(namedAt(0)).toBe('getText') + expect(stampedAt(0)).toBe(commandsLog.at(-1)!.timestamp) + }) + it('does not merge the next test first capture into the previous test slot', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index f8ed5a066..58e77dffa 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -47,7 +47,58 @@ export function mapAssertCommand(command: string): TraceAction | null { return match ? { class: ASSERT_ACTION_CLASS, method: match[1] } : null } -export const ACTION_MAP: Record = { +export type ActionEffect = 'changes' | 'reads' + +export const METHOD_EFFECT = { + navigate: 'changes', + goBack: 'changes', + goForward: 'changes', + reload: 'changes', + goto: 'changes', + click: 'changes', + dblclick: 'changes', + fill: 'changes', + selectOption: 'changes', + hover: 'changes', + scrollIntoViewIfNeeded: 'changes', + dragTo: 'changes', + clear: 'changes', + submit: 'changes', + press: 'changes', + evaluate: 'changes', + tap: 'changes', + switchToWindow: 'changes', + getText: 'reads', + getValue: 'reads', + getAttribute: 'reads', + getProperty: 'reads', + getCSSProperty: 'reads', + getTagName: 'reads', + getLocation: 'reads', + getSize: 'reads', + getRect: 'reads', + isDisplayed: 'reads', + isExisting: 'reads', + isEnabled: 'reads', + isSelected: 'reads', + isClickable: 'reads', + isFocused: 'reads', + waitForDisplayed: 'reads', + waitForHidden: 'reads', + waitForExist: 'reads', + waitForNotExist: 'reads', + waitForEnabled: 'reads', + waitForClickable: 'reads', + waitForFunction: 'reads', + getTitle: 'reads', + getUrl: 'reads', + getPageSource: 'reads' +} as const satisfies Record + +export const ACTION_MAP: Record< + string, + TraceAction & { method: keyof typeof METHOD_EFFECT } +> = { // WDIO browser-level url: { class: 'Page', method: 'navigate' }, navigateTo: { class: 'Page', method: 'navigate' }, @@ -217,3 +268,12 @@ export function isInputDispatchingCommand(command: string): boolean { const action = ACTION_MAP[command] return action !== undefined && POINTABLE_METHODS.has(action.method) } + +/** Whether a command can alter what the page shows; reads and waits don't + * warrant their own capture (cost) and inherit the latest earlier one via `claimAfter`. */ +export function changesPage(command: string): boolean { + if (!Object.prototype.hasOwnProperty.call(ACTION_MAP, command)) { + return false + } + return METHOD_EFFECT[ACTION_MAP[command].method] === 'changes' +} diff --git a/packages/shared/tests/change-page.test.ts b/packages/shared/tests/change-page.test.ts new file mode 100644 index 000000000..611b7dde9 --- /dev/null +++ b/packages/shared/tests/change-page.test.ts @@ -0,0 +1,37 @@ +import { describe, it, expect } from 'vitest' +import { ACTION_MAP, METHOD_EFFECT, changesPage } from '@wdio/devtools-shared' + +describe('changesPage', () => { + it.each([ + 'click', + 'setValue', + 'url', + 'execute', + 'keys', + 'sendKeys', + 'switchToWindow' + ])('is true for %s', (command) => { + expect(changesPage(command)).toBe(true) + }) + + it.each([ + 'getText', + 'isDisplayed', + 'waitForDisplayed', + 'waitUntil', + 'getTitle', + 'waitForElementVisible', + 'expect.toHaveText', + 'assert.equal', + 'findElement', + 'constructor' + ])('is false for %s', (command) => { + expect(changesPage(command)).toBe(false) + }) + + it('classifies the method of every ACTION_MAP entry', () => { + for (const { method } of Object.values(ACTION_MAP)) { + expect(Object.keys(METHOD_EFFECT)).toContain(method) + } + }) +}) diff --git a/packages/trace/tests/trace-frame-snapshots.test.ts b/packages/trace/tests/trace-frame-snapshots.test.ts index 26a87eace..5171bfab4 100644 --- a/packages/trace/tests/trace-frame-snapshots.test.ts +++ b/packages/trace/tests/trace-frame-snapshots.test.ts @@ -54,6 +54,25 @@ describe('FrameSnapshotIndex', () => { expect(index.refs()).toEqual([]) }) + it('resolves every read row to the capture of the latest earlier page change', () => { + const index = new FrameSnapshotIndex([ + snap({ timestamp: 100, command: 'click', screenshot: 'A1' }), + snap({ timestamp: 200, command: 'click', screenshot: 'B2' }) + ]) + expect(index.claimAfter(50, 'read@0')).toBeUndefined() + const reads: [number, string][] = [ + [150, 'A1'], + [160, 'A1'], + [250, 'B2'] + ] + for (const [ts, shot] of reads) { + const id = `read@${ts}` + expect(index.claimAfter(ts, id)).toBe(`after@${id}`) + const ref = index.refs().find((r) => r.callId === id)! + expect(ref.snapshot.screenshot).toBe(shot) + } + }) + it('ignores snapshots without a screenshot', () => { const index = new FrameSnapshotIndex([snap({ screenshot: undefined })]) expect(index.claimAfter(2000, 'call@2')).toBeUndefined() From 0fabf81c89f06009ad8ea304254eb4a100c17f3c Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Tue, 27 Oct 2026 09:32:20 +0100 Subject: [PATCH 02/14] fix(trace): retry the initial frame only when its screenshot is missing captureActionSnapshot reports a failed screenshot probe as a snapshot without a screenshot, not as null, so the retry keyed on a falsy result never fired and a test that opened with reads could end up with no frame. The flag now stays armed until a capture carries a screenshot, under one rule in all three adapters, and is retried at most once per test so a driver that cannot take screenshots does not capture every read again. Co-Authored-By: Claude Opus 5.5 --- packages/nightwatch-devtools/src/session.ts | 6 +- .../tests/actionSnapshot.test.ts | 55 ++++++++++++- .../src/helpers/commandPostActions.ts | 4 +- packages/selenium-devtools/src/session.ts | 2 + .../tests/action-snapshot.test.ts | 78 ++++++++++++++++++- packages/service/src/index.ts | 12 ++- .../tests/trace-action-capture.test.ts | 46 ++++++++++- 7 files changed, 192 insertions(+), 11 deletions(-) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 6ec759919..009d7d386 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -93,9 +93,11 @@ export class SessionCapturer extends SessionCapturerBase { /** True until the current test's first mapped command takes its capture. */ awaitingInitialFrame = true + initialFrameRetried = false resetTestCapture(): void { this.awaitingInitialFrame = true + this.initialFrameRetried = false } constructor( @@ -200,7 +202,9 @@ export class SessionCapturer extends SessionCapturerBase { ).then((snap) => { if (snap) { upsertRichestSnapshot(this.actionSnapshots, snap) - } else if (initialFrame && !changesPage(command)) { + } + if (initialFrame && !snap?.screenshot && !this.initialFrameRetried) { + this.initialFrameRetried = true this.awaitingInitialFrame = true } }) diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index 60ae95826..8acdbe680 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -99,8 +99,52 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) - it('retries the initial frame when the first read captures nothing', async () => { - vi.mocked(coreCapture).mockResolvedValueOnce(null) + it('attempts the initial frame at most twice when no capture has a screenshot', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValue({ timestamp: 1, command: 'getText' }) + const cap = makeCapturer() + for (let t = 1; t <= 4; t++) { + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: t + }) + } + await Promise.all(cap.snapshotCaptures) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + vi.mocked(coreCapture).mockReset() + cap.cleanup() + }) + + it('retries the initial frame when the first capture resolves null', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce(null) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + const cap = makeCapturer() + for (let t = 1; t <= 3; t++) { + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: t + }) + } + await Promise.all(cap.snapshotCaptures) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + cap.cleanup() + }) + + it('retries the initial frame when the first read captures no screenshot', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) const cap = makeCapturer() const at = (command: string, timestamp: number) => cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) @@ -109,7 +153,12 @@ describe('nightwatch action-snapshot timestamp binding', () => { await at('getText', 2) await at('getText', 3) await Promise.all(cap.snapshotCaptures) - expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + expect(cap.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) cap.cleanup() }) diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index 5df0de1a1..cc43f8fc9 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -384,7 +384,9 @@ function queueActionSnapshot( ).then((snap) => { if (snap) { upsertRichestSnapshot(ctx.actionSnapshots, snap) - } else if (initialFrame && !changesPage(cmd.command)) { + } + if (initialFrame && !snap?.screenshot && !capturer.initialFrameRetried) { + capturer.initialFrameRetried = true capturer.awaitingInitialFrame = true } }) diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index 4a54da528..4224e8d69 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -57,9 +57,11 @@ export class SessionCapturer extends SessionCapturerBase { /** True until the current test's first mapped command takes its capture. */ awaitingInitialFrame = true + initialFrameRetried = false resetTestCapture(): void { this.awaitingInitialFrame = true + this.initialFrameRetried = false } #clientConnected = false diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index 0b3287698..c3935aefe 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -1,5 +1,5 @@ import type * as DevtoolsCore from '@wdio/devtools-core' -import { afterEach, describe, expect, it, vi } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { resetSignatureCounters } from '../src/helpers/utils.js' import { TestManager } from '../src/helpers/testManager.js' @@ -128,6 +128,22 @@ describe('selenium action-snapshot timestamp binding', () => { }) describe('selenium action-snapshot capture gate', () => { + // The capture reads through the patcher's originals; route them to the fake + // driver so its screenshot probe is the one that can fail. + const patched = ['takeScreenshot'] as const + beforeEach(() => { + const originals = getDriverOriginals() as Record + for (const key of patched) { + originals[key] = (d: SeleniumDriverLike) => d[key]() + } + }) + afterEach(() => { + const originals = getDriverOriginals() as Record + for (const key of patched) { + delete originals[key] + } + }) + async function run(commands: string[], ctxBag = makeCtx(fakeDriver())) { let t = 1 for (const command of commands) { @@ -157,11 +173,67 @@ describe('selenium action-snapshot capture gate', () => { expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 9]) }) - it('retries the initial frame when the first read captures nothing', async () => { + it('retries the initial frame when the first read captures no screenshot', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValueOnce(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = makeCtx(driver) + for (const [i, command] of [ + 'getText', + 'isDisplayed', + 'getText' + ].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + expect(bag.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) + }) + + it('attempts the initial frame at most twice when every screenshot fails', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValue(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = makeCtx(driver) + for (const [i, command] of [ + 'getText', + 'getText', + 'getText', + 'getText' + ].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + }) + + it('retries the initial frame when the first capture resolves null', async () => { + vi.mocked(coreCapture).mockClear() vi.mocked(coreCapture).mockResolvedValueOnce(null) - const bag = await run(['getText', 'isDisplayed', 'getText']) + const bag = makeCtx(fakeDriver()) + for (const [i, command] of ['getText', 'getText', 'getText'].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) }) + + it('retries the initial frame when a page-changing first command loses its screenshot', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValueOnce(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = await run(['click', 'getText', 'getText'], makeCtx(driver)) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(bag.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) + }) }) describe('selenium action-snapshot locator dialect', () => { diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index 6c0ee05d8..45034d20f 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -80,6 +80,7 @@ export * from './types.js' export const launcher = DevToolsAppLauncher const log = logger('@wdio/devtools-service') +const INITIAL_CAPTURE_MAX_ATTEMPTS = 2 export { setupForDevtools } from './standalone.js' @@ -161,6 +162,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { * the lower bound of that test's video frame window (per-test slicing). */ #currentTestStartWallTime = 0 #awaitingInitialCapture = true + #initialCaptureAttempts = 0 /** Map of testUid → metadata for trace group events and per-spec partitioning. */ #testMetadata: TestMetadataMap = new Map() @@ -383,6 +385,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.resetStack() this.#currentTestStartWallTime = Date.now() this.#awaitingInitialCapture = true + this.#initialCaptureAttempts = 0 this.#currentStepIndex = 0 this.#currentStepUid = undefined @@ -417,6 +420,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.resetStack() this.#currentTestStartWallTime = Date.now() this.#awaitingInitialCapture = true + this.#initialCaptureAttempts = 0 // Track test identity for command tagging. Generate a stable UID // from file + title so commands can be partitioned across reruns. @@ -814,9 +818,15 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#sessionCapturer.currentContext ) if (snap) { - this.#awaitingInitialCapture = false upsertRichestSnapshot(this.#actionSnapshots, snap) } + if ( + this.#awaitingInitialCapture && + (snap?.screenshot || + ++this.#initialCaptureAttempts >= INITIAL_CAPTURE_MAX_ATTEMPTS) + ) { + this.#awaitingInitialCapture = false + } } } diff --git a/packages/service/tests/trace-action-capture.test.ts b/packages/service/tests/trace-action-capture.test.ts index 1f7b502c9..e1f90178e 100644 --- a/packages/service/tests/trace-action-capture.test.ts +++ b/packages/service/tests/trace-action-capture.test.ts @@ -243,13 +243,55 @@ describe('trace mode: one capture per action, taken before it', () => { expect(namedAt(1)).toBe('click') }) - it('retries the initial capture when the first read captures nothing', async () => { + it('retries the initial capture when the first read captures no screenshot', async () => { const service = await started() - vi.mocked(captureActionSnapshot).mockResolvedValueOnce(null) + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + await run(service, ['getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('attempts the initial capture at most twice when no screenshot lands', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ timestamp: 2, command: 'getText' }) + await run(service, ['getText', 'getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('retries the initial capture when the first capture resolves null', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce(null) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) await run(service, ['getText', 'getText', 'getText']) expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) }) + it('retries when a page-changing first command loses its screenshot', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'click' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + await run(service, ['click', 'getText', 'getText']) + expect(namedAt(1)).toBe('getText') + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + it('takes the initial capture again for a new test', async () => { const service = await started() service.beforeTest({ file: '/spec/a.ts', title: 'one' } as never) From c1d84363d7254cd060e1accb33e1ff7036861aa5 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 11:58:23 +0100 Subject: [PATCH 03/14] fix(selenium-devtools): type the fake driver's screenshot as present in the capture-gate tests takeScreenshot is optional on SeleniumDriverLike, so the tests that mock and call it failed the typecheck CI runs. Also documents in CLAUDE.md when each adapter takes its per-action capture now that only page changes take one. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 2 ++ packages/selenium-devtools/tests/action-snapshot.test.ts | 9 ++++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 09c202845..5f3539952 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,6 +91,8 @@ The test for a new helper: *would the backend ever need this to build a zip?* If The per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. +Only a page-changing command (`shared/trace-actions.ts` `changesPage`) or a test's first mapped command takes a capture, and the adapters differ in when. In WDIO the capture runs in the *next* page-changing command's `beforeCommand`, stamped at the previous page-changing command's end, so that command's row shows the page after any waits and reads that followed it. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the page-changing command itself. In every adapter a read or wait takes no capture and inherits the latest earlier one through `claimAfter`. + ### Adapters are thin and isolated Adapter packages own only: diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index c3935aefe..f1b98bdea 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -38,13 +38,16 @@ afterEach(() => { /** A driver whose probes resolve to empty results — enough for the snapshot to * be produced without a browser. The patcher's `originals` bag is empty here * (nothing was patched), so the adapter falls back to these directly. */ -function fakeDriver(): SeleniumDriverLike { +type FakeDriver = SeleniumDriverLike & + Required> + +function fakeDriver(): FakeDriver { return { executeScript: vi.fn().mockResolvedValue([]), takeScreenshot: vi.fn().mockResolvedValue('AA'), getCurrentUrl: vi.fn().mockResolvedValue('http://example.test/'), getTitle: vi.fn().mockResolvedValue('Example') - } as unknown as SeleniumDriverLike + } as unknown as FakeDriver } function makeCtx(driver: SeleniumDriverLike) { @@ -134,7 +137,7 @@ describe('selenium action-snapshot capture gate', () => { beforeEach(() => { const originals = getDriverOriginals() as Record for (const key of patched) { - originals[key] = (d: SeleniumDriverLike) => d[key]() + originals[key] = (d: FakeDriver) => d[key]() } }) afterEach(() => { From 0b59dd67bf532e026a9ab4f69665c000cbe58ff0 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 12:15:32 +0100 Subject: [PATCH 04/14] test(service): give the capture-gate tests' capturer the openWait upstream added Wait commands now open a running row through SessionCapturer.openWait, so the mocked capturer threw on the wait in the gate test once rebased onto main. Co-Authored-By: Claude Opus 5.5 --- packages/service/tests/trace-action-capture.test.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/service/tests/trace-action-capture.test.ts b/packages/service/tests/trace-action-capture.test.ts index e1f90178e..8de93bd50 100644 --- a/packages/service/tests/trace-action-capture.test.ts +++ b/packages/service/tests/trace-action-capture.test.ts @@ -46,6 +46,7 @@ const capturer = { isReportingUpstream: false, metadata: {}, setBrowser: vi.fn(), + openWait: vi.fn(() => 0), /** Set by a test to stand in for the drain having brought a document the * session had not anchored before. */ replacedDocumentInLastDrain: false From 6afc6e82c960f387c5f390ef446c8719c084c619 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 12:15:32 +0100 Subject: [PATCH 05/14] perf(selenium-devtools-py): read the element tree only for commands that change the page The Python adapter ran its two element-tree reads after every command, which the JS adapters stopped doing. The page-changing and mapped command sets are evaluated from shared's ACTION_MAP/changesPage by the wire-schema generator and emitted into _contract.py, so Python follows the same table without a copy. Each test's first mapped command still captures an initial frame, retried once when it has no screenshot. WebElement.submit() reaches execute as a marked script, so it is gated as shared's submit rather than an unmapped script. Co-Authored-By: Claude Opus 5.5 --- docs/ADAPTERS.md | 2 +- .../changes/page-changing-capture.md | 5 + .../scripts/gen_contract.py | 23 ++++ .../src/selenium_devtools/_contract.py | 2 + .../src/selenium_devtools/instrumentation.py | 67 +++++++++-- .../src/selenium_devtools/pytest_plugin.py | 3 +- .../tests/test_a11y_elements.py | 91 +++++++++++++++ .../tests/test_pytest_plugin.py | 17 ++- packages/shared/scripts/wire-schema.ts | 15 ++- packages/shared/wire-schema.json | 107 ++++++++++++++++++ 10 files changed, 319 insertions(+), 13 deletions(-) create mode 100644 packages/selenium-devtools-py/changes/page-changing-capture.md diff --git a/docs/ADAPTERS.md b/docs/ADAPTERS.md index 05512988c..aa4d59b54 100644 --- a/docs/ADAPTERS.md +++ b/docs/ADAPTERS.md @@ -169,7 +169,7 @@ All times are epoch milliseconds except the suite/test `start`/`end` ISO strings ### Generating your types -`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with: +`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`, and the command names the Python capture gate reads under `actionCommands`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with: ```sh pnpm --filter @wdio/devtools-shared gen:wire-schema diff --git a/packages/selenium-devtools-py/changes/page-changing-capture.md b/packages/selenium-devtools-py/changes/page-changing-capture.md new file mode 100644 index 000000000..02160ba61 --- /dev/null +++ b/packages/selenium-devtools-py/changes/page-changing-capture.md @@ -0,0 +1,5 @@ +--- +patch +--- + +Trace mode reads each page's element tree only after a command that changes the page, and after each test's first command, instead of after every command. Reads and waits keep their own rows and show the tree from the last page change. diff --git a/packages/selenium-devtools-py/scripts/gen_contract.py b/packages/selenium-devtools-py/scripts/gen_contract.py index f3d6af633..28a869371 100644 --- a/packages/selenium-devtools-py/scripts/gen_contract.py +++ b/packages/selenium-devtools-py/scripts/gen_contract.py @@ -203,6 +203,26 @@ def _runner_cwd_env(runner_ts: str) -> str: return m.group(1) +def _action_commands(schema_path: Path) -> dict[str, list[str]]: + """The runner-native command names the capture gate reads, evaluated from + shared's ACTION_MAP by ``wire-schema.ts`` rather than restated here. + + Generated because a name that drifts is silent: a command missing from the + page-changing set is never captured, so its row shows the page before it. + """ + if not schema_path.exists(): + raise SystemExit( + f"{schema_path} is missing: run `npx tsx scripts/wire-schema.ts` in packages/shared" + ) + sets = json.loads(schema_path.read_text()).get("actionCommands") + if not sets or "pageChanging" not in sets or "mapped" not in sets: + raise SystemExit( + "contract drift: `actionCommands` is missing from wire-schema.json; " + "regenerate it with packages/shared/scripts/wire-schema.ts" + ) + return sets + + def _test_runner_ids(types_ts: str) -> list[str]: m = re.search(r"export const TEST_RUNNER_IDS = \[(.*?)\] as const", types_ts, re.DOTALL) if not m: @@ -335,6 +355,7 @@ def main() -> int: rerun_slot = _rerun_slot(runner_ts) runner_cwd_env = _runner_cwd_env(runner_ts) reuse_env = _reuse_env(runner_ts) + action_commands = _action_commands(shared / "wire-schema.json") # Drift-guard. missing = [v for v in REQUIRED_DATA_SCOPES.values() if v not in data_keys] @@ -409,6 +430,8 @@ def main() -> int: lines += [ "", f"DATA_SCOPES = frozenset({sorted(data_keys)!r})", + f"PAGE_CHANGING_COMMANDS = frozenset({action_commands['pageChanging']!r})", + f"MAPPED_COMMANDS = frozenset({action_commands['mapped']!r})", f"CONTROL_SCOPES = frozenset({sorted(control.values())!r})", "", f'COLLECTOR_PATH = "{collector_path}"', diff --git a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py index 3023e6435..ddf904e18 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py @@ -15,6 +15,8 @@ SCOPE_REPLACE_COMMAND = "replaceCommand" DATA_SCOPES = frozenset(['actionSnapshots', 'commands', 'config', 'consoleLogs', 'logs', 'metadata', 'mutations', 'networkRequests', 'screencast', 'screencastFrames', 'sources', 'suites']) +PAGE_CHANGING_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'goBack', 'goForward', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url']) +MAPPED_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'getAttribute', 'getCSSProperty', 'getCssValue', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getLocation', 'getPageSource', 'getProperty', 'getRect', 'getSize', 'getTagName', 'getText', 'getTitle', 'getUrl', 'getValue', 'goBack', 'goForward', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) CONTROL_SCOPES = frozenset(['clearCommands', 'clearExecutionData', 'clientConnected', 'clientDisconnected', 'config', 'replaceCommand', 'testStopped']) COLLECTOR_PATH = "/api/collector" diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index e8da03ad2..3c0fda94c 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -30,6 +30,7 @@ read_value_locators, wait_tracking, ) +from ._contract import MAPPED_COMMANDS, PAGE_CHANGING_COMMANDS from .assert_tracer import ScriptAssertionTracer from .capturer import SessionCapturer from .cdp_screencast import start_push_screencast @@ -288,6 +289,9 @@ def _capture_source(capturer: SessionCapturer, call_src: Optional[str]) -> None: # is the only moment they are reachable. Run-scoped: a run may replace its # driver, and the filmstrip is the whole run. "filmstrip_frames": [], + # Re-armed per test by reset_test_capture(). + "awaiting_initial_frame": True, + "initial_frame_retried": False, } @@ -421,9 +425,18 @@ def resolved_output_dir() -> Optional[str]: return _state.get("output_dir") +_SUBMIT_SCRIPT_MARKER = "/* submitForm */" + + +def _action_capture_enabled(driver: Any) -> bool: + if not _state["trace"] or not _state["a11y"] or not _state.get("element_scripts"): + return False + return not driver_is_native_app(driver) + + def _capture_action_snapshot( driver: Any, command: str, timestamp: int, shot: Optional[str] -) -> None: +) -> Optional[ActionSnapshot]: """Read the page's element tree beside one action, for the trace's A11y tab. Trace mode only, and only with the scripts in hand — the JS adapters run @@ -437,13 +450,9 @@ def _capture_action_snapshot( Skipped entirely on a native app: both reads are page script, they run on every action, and neither can do anything but fail there. """ - if not _state["trace"] or not _state["a11y"]: - return - if driver_is_native_app(driver): - return + if not _action_capture_enabled(driver): + return None scripts = _state.get("element_scripts") - if not scripts: - return run = _guarded_execute_script(driver) snapshot: ActionSnapshot = {"timestamp": timestamp, "command": command} if shot: @@ -464,8 +473,42 @@ def _capture_action_snapshot( if isinstance(value, list) and value: snapshot[key] = value if "elements" not in snapshot and "accessibilityTree" not in snapshot: - return + return None _state["action_snapshots"].append(snapshot) + return snapshot + + +def _maybe_capture_action_snapshot( + driver: Any, command: str, timestamp: int, shot: Optional[str] +) -> None: + """Capture on a page-changing command, and on a test's first mapped command + as its initial frame. Reads and waits inherit the capture before them.""" + initial = _state["awaiting_initial_frame"] and command in MAPPED_COMMANDS + if command not in PAGE_CHANGING_COMMANDS and not initial: + return + if not _action_capture_enabled(driver): + return + if initial: + _state["awaiting_initial_frame"] = False + snapshot = _capture_action_snapshot(driver, command, timestamp, shot) + if not initial or _state["initial_frame_retried"]: + return + if snapshot is None or "screenshot" not in snapshot: + _state["initial_frame_retried"] = True + _state["awaiting_initial_frame"] = True + + +def _capture_command(driver_command: str, params: Any) -> str: + """Selenium's `WebElement.submit()` reaches `execute` as a marked script.""" + script = params.get("script") if isinstance(params, dict) else None + if isinstance(script, str) and script.startswith(_SUBMIT_SCRIPT_MARKER): + return "submit" + return driver_command + + +def reset_test_capture() -> None: + """Re-arm the initial frame for the test about to start.""" + _state.update(awaiting_initial_frame=True, initial_frame_retried=False) def _begin_screencast_run(entry: Optional[dict], shot: Optional[str] = None) -> None: @@ -1064,7 +1107,12 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, row) - _capture_action_snapshot(self, driver_command, row.get("timestamp", start), shot) + _maybe_capture_action_snapshot( + self, + _capture_command(driver_command, params), + row.get("timestamp", start), + shot, + ) if driver_command in NAVIGATION_COMMANDS: _attach_performance(capturer, self, row, params) # No per-command line here: the Actions timeline lists every command as @@ -1093,6 +1141,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN sessions=weakref.WeakKeyDictionary(), output_dir=None, default_suite=None, trace=trace, filmstrip=filmstrip, filmstrip_frames=[], a11y=a11y, element_scripts=None, page_scripts=None, action_snapshots=[], + awaiting_initial_frame=True, initial_frame_retried=False, ) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py index b89662d01..cccaa327b 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py @@ -15,7 +15,7 @@ from typing import Dict, Optional import selenium_devtools as devtools -from . import assertions, frames, read_value_locators, rerun +from . import assertions, frames, instrumentation, read_value_locators, rerun from .constants import ENV_OPT_IN, ENV_PORT, LOGGER_NAME from .utils import now_ms @@ -589,6 +589,7 @@ def pytest_runtest_logstart(nodeid, location) -> None: # noqa: ANN001 return _registry.mark_start(nodeid) read_value_locators.reset() + instrumentation.reset_test_capture() capturer = devtools.get_capturer() if capturer is None: return diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index ad0a9e47f..2104ac85b 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -6,6 +6,7 @@ same flow carried 12; this is the gap. """ +import contextlib import json import unittest from unittest import mock @@ -13,6 +14,8 @@ from selenium_devtools import element_scripts, instrumentation, trace_export from selenium_devtools._contract import ( ELEMENT_SCRIPTS_PATH, + MAPPED_COMMANDS, + PAGE_CHANGING_COMMANDS, RUNNER_ID, SCOPE_ACTION_SNAPSHOTS, ) @@ -217,6 +220,94 @@ def test_an_empty_or_failed_read_records_no_snapshot(self): self.assertEqual(self._capture(self.Driver(elements=None)), []) self.assertEqual(self._capture(self.Driver(raises=True)), []) + @contextlib.contextmanager + def _armed(self, driver=None): + with mock.patch.dict( + instrumentation._state, + { + "trace": True, + "a11y": True, + "element_scripts": SCRIPTS, + "action_snapshots": [], + "awaiting_initial_frame": True, + "initial_frame_retried": False, + }, + ): + yield driver or self.Driver(elements=[{"selector": "#go"}]) + + def _gate(self, driver, command, shot="c2hvdA=="): + instrumentation._maybe_capture_action_snapshot(driver, command, 1200, shot) + + def _captured(self): + return [snap["command"] for snap in instrumentation.action_snapshots()] + + def test_a_click_is_captured_and_the_reads_after_it_are_not(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "getElementText") + self._gate(driver, "wait") + self.assertEqual(self._captured(), ["clickElement"]) + self.assertEqual(len(driver.scripts_run), 2, "read the page for a read") + + def test_a_leading_read_is_the_initial_frame_once_until_the_next_test(self): + with self._armed() as driver: + self._gate(driver, "getElementText") + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["getElementText"]) + + instrumentation.reset_test_capture() + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["getElementText", "getElementText"]) + + def test_a_missing_initial_screenshot_is_retried_exactly_once(self): + with self._armed() as driver: + self._gate(driver, "getElementText", shot=None) + self._gate(driver, "getElementText", shot="c2hvdA==") + self._gate(driver, "getElementText", shot="c2hvdA==") + snaps = instrumentation.action_snapshots() + self.assertEqual(len(snaps), 2) + self.assertNotIn("screenshot", snaps[0]) + self.assertIn("screenshot", snaps[1]) + + def test_a_form_submit_is_captured_as_a_page_change(self): + submit = {"script": "/* submitForm */var form = arguments[0];", "args": []} + other = {"script": "return 1", "args": []} + self.assertEqual( + instrumentation._capture_command("w3cExecuteScript", submit), "submit" + ) + self.assertEqual( + instrumentation._capture_command("w3cExecuteScript", other), + "w3cExecuteScript", + ) + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", submit)) + self.assertEqual(self._captured(), ["clickElement", "submit"]) + + def test_a_second_missing_initial_screenshot_is_not_retried_again(self): + with self._armed() as driver: + self._gate(driver, "clickElement", shot=None) + self._gate(driver, "getElementText", shot=None) + self._gate(driver, "getElementText", shot="c2hvdA==") + self.assertEqual(self._captured(), ["clickElement", "getElementText"]) + + def test_the_initial_frame_waits_for_a_capture_that_can_run(self): + class NativeDriver(self.Driver): + capabilities = {"platformName": "Android", "appium:app": "/app.apk"} + + with self._armed(NativeDriver(elements=[{"selector": "#go"}])) as driver: + self._gate(driver, "getElementText") + self.assertTrue(instrumentation._state["awaiting_initial_frame"]) + self.assertEqual(driver.scripts_run, []) + + def test_the_generated_sets_agree_with_the_gate(self): + self.assertIn("get", PAGE_CHANGING_COMMANDS) + self.assertIn("clickElement", PAGE_CHANGING_COMMANDS) + self.assertNotIn("getElementText", PAGE_CHANGING_COMMANDS) + self.assertIn("getElementText", MAPPED_COMMANDS) + self.assertNotIn("findElement", MAPPED_COMMANDS) + self.assertLessEqual(PAGE_CHANGING_COMMANDS, MAPPED_COMMANDS) + class TestStreamingTheSnapshots(unittest.TestCase): def test_they_go_out_under_the_action_snapshots_scope(self): diff --git a/packages/selenium-devtools-py/tests/test_pytest_plugin.py b/packages/selenium-devtools-py/tests/test_pytest_plugin.py index d54e726b3..9bda8d391 100644 --- a/packages/selenium-devtools-py/tests/test_pytest_plugin.py +++ b/packages/selenium-devtools-py/tests/test_pytest_plugin.py @@ -11,7 +11,7 @@ import unittest from unittest import mock -from selenium_devtools import assertions +from selenium_devtools import assertions, instrumentation from selenium_devtools import pytest_plugin as plugin from selenium_devtools.pytest_plugin import _SuiteRegistry, reported_state from selenium_devtools.utils import iso @@ -550,3 +550,18 @@ def test_the_memoized_value_is_dropped_so_the_rewriter_re_reads(self): plugin._enable_assertion_pass_hook(config) self.assertIs(config.getini("enable_assertion_pass_hook"), True) + + +class TestEachTestStartsWithItsOwnInitialFrame(unittest.TestCase): + def test_a_test_starting_re_arms_the_initial_frame(self): + with mock.patch.dict( + instrumentation._state, + {"awaiting_initial_frame": False, "initial_frame_retried": True}, + ), mock.patch.object(plugin, "_opted_in", return_value=True), mock.patch.object( + plugin.devtools, "get_capturer", return_value=None + ): + plugin.pytest_runtest_logstart("tests/t.py::test_a", ("tests/t.py", 1, "test_a")) + + self.assertIs(instrumentation._state["awaiting_initial_frame"], True) + self.assertIs(instrumentation._state["initial_frame_retried"], False) + diff --git a/packages/shared/scripts/wire-schema.ts b/packages/shared/scripts/wire-schema.ts index a0d088fc2..305e9cb3b 100644 --- a/packages/shared/scripts/wire-schema.ts +++ b/packages/shared/scripts/wire-schema.ts @@ -4,6 +4,7 @@ import { realpathSync, writeFileSync } from 'node:fs' import path from 'node:path' import { fileURLToPath } from 'node:url' import ts from 'typescript' +import { ACTION_MAP, changesPage } from '../src/trace-actions.js' export type JsonSchema = { [key: string]: unknown } @@ -12,6 +13,8 @@ export interface WireSchema { $comment: string scopes: Record $defs: Record + // Runner-native command names, evaluated from ACTION_MAP rather than typed out. + actionCommands: { pageChanging: string[]; mapped: string[] } } const HERE = path.dirname(fileURLToPath(import.meta.url)) @@ -239,7 +242,17 @@ export function buildWireSchema(): WireSchema { $comment: 'GENERATED by packages/shared/scripts/wire-schema.ts from shared types. Do not edit.', scopes: sortKeys(scopes), - $defs: sortKeys(Object.fromEntries(builder.defs)) + $defs: sortKeys(Object.fromEntries(builder.defs)), + actionCommands: actionCommandSets() + } +} + +// Every ACTION_MAP key is mapped by mapCommandToAction (its fallback only adds assert.*, which no WebDriver hook sees). +function actionCommandSets(): WireSchema['actionCommands'] { + const commands = Object.keys(ACTION_MAP).sort() + return { + pageChanging: commands.filter((c) => changesPage(c)), + mapped: commands } } diff --git a/packages/shared/wire-schema.json b/packages/shared/wire-schema.json index 3616f4638..8564ce392 100644 --- a/packages/shared/wire-schema.json +++ b/packages/shared/wire-schema.json @@ -1433,5 +1433,112 @@ ], "additionalProperties": false } + }, + "actionCommands": { + "pageChanging": [ + "addValue", + "back", + "clear", + "clearElement", + "click", + "clickElement", + "doubleClick", + "dragAndDrop", + "execute", + "executeAsync", + "forward", + "get", + "goBack", + "goForward", + "keys", + "moveTo", + "navigateTo", + "newWindow", + "refresh", + "scrollIntoView", + "selectByVisibleText", + "sendKeys", + "sendKeysToElement", + "setValue", + "submit", + "switchToFrame", + "switchToParentFrame", + "switchToWindow", + "to", + "touchAction", + "url" + ], + "mapped": [ + "addValue", + "back", + "clear", + "clearElement", + "click", + "clickElement", + "doubleClick", + "dragAndDrop", + "execute", + "executeAsync", + "forward", + "get", + "getAttribute", + "getCSSProperty", + "getCssValue", + "getCurrentUrl", + "getElementAttribute", + "getElementProperty", + "getElementRect", + "getElementTagName", + "getElementText", + "getElementValueOfCssProperty", + "getLocation", + "getPageSource", + "getProperty", + "getRect", + "getSize", + "getTagName", + "getText", + "getTitle", + "getUrl", + "getValue", + "goBack", + "goForward", + "isClickable", + "isDisplayed", + "isElementEnabled", + "isElementSelected", + "isEnabled", + "isExisting", + "isFocused", + "isSelected", + "keys", + "moveTo", + "navigateTo", + "newWindow", + "refresh", + "scrollIntoView", + "selectByVisibleText", + "sendKeys", + "sendKeysToElement", + "setValue", + "submit", + "switchToFrame", + "switchToParentFrame", + "switchToWindow", + "to", + "touchAction", + "url", + "wait", + "waitForClickable", + "waitForDisplayed", + "waitForElementLocated", + "waitForElementNotPresent", + "waitForElementNotVisible", + "waitForElementPresent", + "waitForElementVisible", + "waitForEnabled", + "waitForExist", + "waitUntil" + ] } } From 3665ae402028e56b83b1c0e8228ffc76f80490a2 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 12:22:17 +0100 Subject: [PATCH 06/14] fix(trace): skip captures for Nightwatch url() reads and keep a late capture off the next test browser.url() with no string argument only reads the url (assert.urlContains reads through it), so gating it as a navigation took a capture per read. And a slow initial-frame capture that resolved after the next test had started could spend that test's one retry; a per-test generation now ignores it. CLAUDE.md scopes the beforeCommand description to the WDIO service and names the initial-frame exception. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 4 +- packages/nightwatch-devtools/src/session.ts | 27 ++++++++++--- .../tests/actionSnapshot.test.ts | 39 +++++++++++++++++++ .../src/helpers/commandPostActions.ts | 9 ++++- packages/selenium-devtools/src/session.ts | 2 + .../tests/action-snapshot.test.ts | 21 ++++++++++ 6 files changed, 94 insertions(+), 8 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5f3539952..7fcab2cbb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -89,9 +89,9 @@ The test for a new helper: *would the backend ever need this to build a zip?* If ### One DOM snapshot per action, taken before it -The per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. +In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. -Only a page-changing command (`shared/trace-actions.ts` `changesPage`) or a test's first mapped command takes a capture, and the adapters differ in when. In WDIO the capture runs in the *next* page-changing command's `beforeCommand`, stamped at the previous page-changing command's end, so that command's row shows the page after any waits and reads that followed it. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the page-changing command itself. In every adapter a read or wait takes no capture and inherits the latest earlier one through `claimAfter`. +Only a page-changing command (`shared/trace-actions.ts` `changesPage`) or a test's first mapped command takes a capture, and the adapters differ in when. In WDIO the capture runs in the *next* page-changing command's `beforeCommand`, stamped at the previous page-changing command's end, so that command's row shows the page after any waits and reads that followed it. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the page-changing command itself. In every adapter a read or wait takes no capture, unless it is the test's first mapped command, and inherits the latest earlier one through `claimAfter`. ### Adapters are thin and isolated diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 009d7d386..9f6235105 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -94,10 +94,12 @@ export class SessionCapturer extends SessionCapturerBase { /** True until the current test's first mapped command takes its capture. */ awaitingInitialFrame = true initialFrameRetried = false + testCaptureGeneration = 0 resetTestCapture(): void { this.awaitingInitialFrame = true this.initialFrameRetried = false + this.testCaptureGeneration++ } constructor( @@ -169,7 +171,7 @@ export class SessionCapturer extends SessionCapturerBase { } if (!error) { - this.#queueActionSnapshot(command, commandLogEntry.timestamp) + this.#queueActionSnapshot(command, args, commandLogEntry.timestamp) } return true @@ -179,19 +181,28 @@ export class SessionCapturer extends SessionCapturerBase { * logged command's timestamp, not the capture's own: the capture resolves * after the command completes, and export claims snapshots by exact equality * with the command timestamp (FrameSnapshotIndex.claimAfter / elementsAt). */ - #queueActionSnapshot(command: string, timestamp: number): void { + #queueActionSnapshot( + command: string, + args: unknown[], + timestamp: number + ): void { if (this.traceMode !== 'trace' || !this.#browser) { return } // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. const initialFrame = this.awaitingInitialFrame && !!mapCommandToAction(command) - if (!changesPage(command) && !initialFrame) { + // `url()` without a string argument is a getter; it does not navigate. + const pageChanging = + changesPage(command) && + !(command === 'url' && typeof args[0] !== 'string') + if (!pageChanging && !initialFrame) { return } if (initialFrame) { this.awaitingInitialFrame = false } + const generation = this.testCaptureGeneration this.snapshotCaptures.push( captureActionSnapshot( this.#browser, @@ -203,7 +214,13 @@ export class SessionCapturer extends SessionCapturerBase { if (snap) { upsertRichestSnapshot(this.actionSnapshots, snap) } - if (initialFrame && !snap?.screenshot && !this.initialFrameRetried) { + // A capture resolving after resetTestCapture() belongs to the previous test. + if ( + initialFrame && + !snap?.screenshot && + !this.initialFrameRetried && + generation === this.testCaptureGeneration + ) { this.initialFrameRetried = true this.awaitingInitialFrame = true } @@ -374,7 +391,7 @@ export class SessionCapturer extends SessionCapturerBase { entry.error = serializeError(error) entry.timestamp = timestamp if (!error) { - this.#queueActionSnapshot(entry.command, timestamp) + this.#queueActionSnapshot(entry.command, entry.args, timestamp) } this.sendReplaceCommand(oldTimestamp, entry) } diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index 8acdbe680..a3a1ef3f9 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -162,6 +162,45 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) + it('takes no snapshot for a url() read, but one for a url() navigation', async () => { + vi.mocked(coreCapture).mockClear() + const cap = makeCapturer() + const at = (command: string, args: unknown[], timestamp: number) => + cap.captureCommand(command, args, undefined, undefined, { timestamp }) + await at('click', ['#btn'], 1) + await at('url', [() => undefined], 2) + await at('url', [], 3) + await at('url', ['http://example.test/next'], 4) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 4]) + cap.cleanup() + }) + + it('does not re-arm the initial frame from a capture that resolves after the next test starts', async () => { + type Snap = Awaited> + let resolveFirst!: (snap: Snap) => void + vi.mocked(coreCapture) + .mockClear() + .mockImplementationOnce( + () => + new Promise((resolve) => { + resolveFirst = resolve + }) + ) + const cap = makeCapturer() + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: 1 + }) + cap.resetTestCapture() + resolveFirst({ timestamp: 1, command: 'getText' } as Snap) + await Promise.all(cap.snapshotCaptures) + + expect(cap.initialFrameRetried).toBe(false) + expect(cap.awaitingInitialFrame).toBe(true) + cap.cleanup() + }) + it('does not capture outside trace mode', async () => { const cap = new SessionCapturer({}, makeMockBrowser()) await cap.captureCommand('click', [], undefined, undefined, { diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index cc43f8fc9..b8df0e357 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -375,6 +375,7 @@ function queueActionSnapshot( if (initialFrame) { capturer.awaitingInitialFrame = false } + const generation = capturer.testCaptureGeneration ctx.snapshotCaptures.push( captureActionSnapshot( ctx.driver, @@ -385,7 +386,13 @@ function queueActionSnapshot( if (snap) { upsertRichestSnapshot(ctx.actionSnapshots, snap) } - if (initialFrame && !snap?.screenshot && !capturer.initialFrameRetried) { + // A capture resolving after resetTestCapture() belongs to the previous test. + if ( + initialFrame && + !snap?.screenshot && + !capturer.initialFrameRetried && + generation === capturer.testCaptureGeneration + ) { capturer.initialFrameRetried = true capturer.awaitingInitialFrame = true } diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index 4224e8d69..c659888b9 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -58,10 +58,12 @@ export class SessionCapturer extends SessionCapturerBase { /** True until the current test's first mapped command takes its capture. */ awaitingInitialFrame = true initialFrameRetried = false + testCaptureGeneration = 0 resetTestCapture(): void { this.awaitingInitialFrame = true this.initialFrameRetried = false + this.testCaptureGeneration++ } #clientConnected = false diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index f1b98bdea..ea677689c 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -237,6 +237,27 @@ describe('selenium action-snapshot capture gate', () => { true ]) }) + + it('does not re-arm the initial frame from a capture that resolves after the next test starts', async () => { + type Snap = Awaited> + let resolveFirst!: (snap: Snap) => void + vi.mocked(coreCapture) + .mockClear() + .mockImplementationOnce( + () => + new Promise((resolve) => { + resolveFirst = resolve + }) + ) + const bag = makeCtx(fakeDriver()) + await handleOnCommand(bag.ctx, cmd({ command: 'getText', timestamp: 1 })) + bag.capturer.resetTestCapture() + resolveFirst({ timestamp: 1, command: 'getText' } as Snap) + await Promise.all(bag.snapshotCaptures) + + expect(bag.capturer.initialFrameRetried).toBe(false) + expect(bag.capturer.awaitingInitialFrame).toBe(true) + }) }) describe('selenium action-snapshot locator dialect', () => { From 59ad08139b7396cfba0b811623080908b109b3c8 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 17:05:45 +0100 Subject: [PATCH 07/14] fix(trace): capture every command except known reads, and the end state after trailing reads The page-changing allowlist missed commands that change the page without being in ACTION_MAP (an executeScript or actions() click, an alert accept, Nightwatch submitForm), so the reads after them showed the old page in the Selenium, Nightwatch and Python adapters. The gate is now a deny-list: shared skipsCapture() names the reads, waits, lookups and asserts that skip a capture, and everything else captures. Python reads the list from wire-schema.json, and treats selenium's isDisplayed/getAttribute script atoms as the reads they are. Those adapters also capture right after the command, so a click that returns before its navigation commits left a test ending on reads with the old page. Each now takes one end-of-test capture when rows follow its last capture, stamped at that capture (core finalFrameStamp) and replacing it whole (trace replaceSnapshotAt), the way the WDIO service's finalize already does. Co-Authored-By: Claude Opus 5.5 --- .changeset/capture-only-page-changes.md | 4 +- CLAUDE.md | 2 +- packages/core/src/final-frame.ts | 29 ++++ packages/core/src/index.ts | 1 + packages/core/tests/final-frame.test.ts | 48 ++++++ .../src/cucumber-lifecycle.ts | 1 + packages/nightwatch-devtools/src/index.ts | 1 + packages/nightwatch-devtools/src/session.ts | 42 +++++- .../tests/actionSnapshot.test.ts | 55 +++++++ .../changes/page-changing-capture.md | 2 +- .../scripts/gen_contract.py | 11 +- .../src/selenium_devtools/_contract.py | 2 +- .../src/selenium_devtools/instrumentation.py | 114 ++++++++++++-- .../src/selenium_devtools/pytest_plugin.py | 2 + .../tests/test_a11y_elements.py | 140 +++++++++++++++++- .../tests/test_pytest_plugin.py | 27 ++++ .../src/helpers/commandPostActions.ts | 7 +- .../src/helpers/finalFrame.ts | 47 ++++++ packages/selenium-devtools/src/index.ts | 3 + packages/selenium-devtools/src/session.ts | 4 + .../tests/action-snapshot.test.ts | 50 ++++++- packages/shared/scripts/wire-schema.ts | 16 +- packages/shared/src/trace-actions.ts | 47 ++++++ packages/shared/tests/change-page.test.ts | 33 ++++- packages/shared/tests/wire-schema.test.ts | 11 ++ packages/shared/wire-schema.json | 104 ++++++++----- packages/trace/src/trace-snapshots.ts | 13 ++ packages/trace/tests/trace-snapshots.test.ts | 26 ++++ 28 files changed, 766 insertions(+), 76 deletions(-) create mode 100644 packages/core/src/final-frame.ts create mode 100644 packages/core/tests/final-frame.test.ts create mode 100644 packages/selenium-devtools/src/helpers/finalFrame.ts diff --git a/.changeset/capture-only-page-changes.md b/.changeset/capture-only-page-changes.md index 944a63204..e33551f8b 100644 --- a/.changeset/capture-only-page-changes.md +++ b/.changeset/capture-only-page-changes.md @@ -4,4 +4,6 @@ '@wdio/nightwatch-devtools': patch --- -Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for read and wait commands such as `getText`, `isDisplayed` or `waitForDisplayed`. Those rows still appear in the trace and now show the page as of the last command that changed it. Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). +Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for reads, waits and lookups such as `getText`, `isDisplayed`, `waitForDisplayed` or `findElement`. Those rows still appear in the trace and show the page as of the last command that changed it. Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). + +Selenium and Nightwatch now capture every other command, including ones that can navigate but are not in the action map (`executeScript`, `actions`, Nightwatch `submitForm`), and take one more capture at a test's end when reads follow its last capture, so a click that returns before its navigation commits no longer leaves the reads showing the page it left. The WDIO service keeps its page-changing gate. diff --git a/CLAUDE.md b/CLAUDE.md index 7fcab2cbb..0334a01ea 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,7 +91,7 @@ The test for a new helper: *would the backend ever need this to build a zip?* If In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. -Only a page-changing command (`shared/trace-actions.ts` `changesPage`) or a test's first mapped command takes a capture, and the adapters differ in when. In WDIO the capture runs in the *next* page-changing command's `beforeCommand`, stamped at the previous page-changing command's end, so that command's row shows the page after any waits and reads that followed it. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the page-changing command itself. In every adapter a read or wait takes no capture, unless it is the test's first mapped command, and inherits the latest earlier one through `claimAfter`. +Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and replacing its record whole; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. ### Adapters are thin and isolated diff --git a/packages/core/src/final-frame.ts b/packages/core/src/final-frame.ts new file mode 100644 index 000000000..ed037d457 --- /dev/null +++ b/packages/core/src/final-frame.ts @@ -0,0 +1,29 @@ +export interface FinalFrameCommand { + command: string + timestamp: number +} + +/** Rows after the last capture saw a state no capture holds; the end-of-test capture is stamped at that capture's timestamp so those rows resolve to it. */ +export function finalFrameStamp( + commands: ReadonlyArray, + testStartMs: number, + lastCapturedTimestamp: number | undefined +): FinalFrameCommand | null { + const inTest = commands.filter((c) => c.timestamp >= testStartMs) + const last = inTest[inTest.length - 1] + if (!last) { + return null + } + if ( + lastCapturedTimestamp === undefined || + lastCapturedTimestamp < testStartMs + ) { + return { timestamp: last.timestamp, command: last.command } + } + if (!inTest.some((c) => c.timestamp > lastCapturedTimestamp)) { + return null + } + const atOrBefore = inTest.filter((c) => c.timestamp <= lastCapturedTimestamp) + const owner = atOrBefore[atOrBefore.length - 1] ?? inTest[0]! + return { timestamp: lastCapturedTimestamp, command: owner.command } +} diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 540a05ccd..9d59e02fa 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -67,6 +67,7 @@ export * from './request-type.js' export * from './stack.js' export * from './terminal-throttle.js' export * from './error.js' +export * from './final-frame.js' export * from './finalize-screencast.js' export * from './input-dispatch.js' export * from './output-dir.js' diff --git a/packages/core/tests/final-frame.test.ts b/packages/core/tests/final-frame.test.ts new file mode 100644 index 000000000..fde36d525 --- /dev/null +++ b/packages/core/tests/final-frame.test.ts @@ -0,0 +1,48 @@ +import { describe, it, expect } from 'vitest' +import { finalFrameStamp } from '@wdio/devtools-core' + +const commands = [ + { command: 'url', timestamp: 90 }, + { command: 'click', timestamp: 120 }, + { command: 'getText', timestamp: 130 }, + { command: 'isDisplayed', timestamp: 140 } +] + +describe('finalFrameStamp', () => { + it('returns null when the test logged no rows', () => { + expect(finalFrameStamp([], 100, undefined)).toBeNull() + }) + + it('returns null when every row predates the test', () => { + expect(finalFrameStamp(commands, 200, 150)).toBeNull() + }) + + it('stamps the last row when no capture was taken in this test', () => { + expect(finalFrameStamp(commands, 100, 50)).toEqual({ + timestamp: 140, + command: 'isDisplayed' + }) + expect(finalFrameStamp(commands, 100, undefined)).toEqual({ + timestamp: 140, + command: 'isDisplayed' + }) + }) + + it('stamps the last capture when reads followed it', () => { + expect(finalFrameStamp(commands, 100, 120)).toEqual({ + timestamp: 120, + command: 'click' + }) + }) + + it('returns null when the test ends on the capturing command', () => { + expect(finalFrameStamp(commands, 100, 140)).toBeNull() + }) + + it('names the last row at or before the capture when no row sits on it', () => { + expect(finalFrameStamp(commands, 100, 125)).toEqual({ + timestamp: 125, + command: 'click' + }) + }) +}) diff --git a/packages/nightwatch-devtools/src/cucumber-lifecycle.ts b/packages/nightwatch-devtools/src/cucumber-lifecycle.ts index 7a6491d60..3d696523a 100644 --- a/packages/nightwatch-devtools/src/cucumber-lifecycle.ts +++ b/packages/nightwatch-devtools/src/cucumber-lifecycle.ts @@ -283,6 +283,7 @@ export async function captureCucumberScenarioBeforeQuit( ) } await ctx.sessionCapturer.captureTraceAtTestEnd(browser) + await ctx.sessionCapturer.captureFinalFrame() flushTestSlice(ctx) await ctx.emitTestArtifacts( scenario?.uid, diff --git a/packages/nightwatch-devtools/src/index.ts b/packages/nightwatch-devtools/src/index.ts index cc3fde46a..bf4d0a24b 100644 --- a/packages/nightwatch-devtools/src/index.ts +++ b/packages/nightwatch-devtools/src/index.ts @@ -572,6 +572,7 @@ class NightwatchDevToolsPlugin { ) } await this.sessionCapturer.captureTraceAtTestEnd(browser) + await this.sessionCapturer.captureFinalFrame() // Flush this test's slice before the next test overwrites its outcome. flushTestSlice(this.#getInternals()) const results = (browser.currentTest as NightwatchCurrentTest)?.results diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 9f6235105..61fba27ec 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -6,19 +6,21 @@ import { collectorDrainExpression, drainCollectorWithRecovery, errorMessage, + finalFrameStamp, loadInjectableScript, mapCommandToAction, NativeNetworkCapture, pollUntilReady, + replaceSnapshotAt, serializeError, upsertRichestSnapshot, type CapturedPerformancePayload, type SessionCapturerOptions } from '@wdio/devtools-core' import { - changesPage, COLLECTOR_INJECT_SCRIPT, - COLLECTOR_READY_SCRIPT + COLLECTOR_READY_SCRIPT, + skipsCapture } from '@wdio/devtools-shared' import { captureActionSnapshot } from './action-snapshot.js' import { nightwatchRunnerId } from './helpers/utils.js' @@ -95,11 +97,15 @@ export class SessionCapturer extends SessionCapturerBase { awaitingInitialFrame = true initialFrameRetried = false testCaptureGeneration = 0 + /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ + lastCapturedTimestamp: number | undefined = undefined + testStartWallTime = 0 resetTestCapture(): void { this.awaitingInitialFrame = true this.initialFrameRetried = false this.testCaptureGeneration++ + this.testStartWallTime = Date.now() } constructor( @@ -189,12 +195,12 @@ export class SessionCapturer extends SessionCapturerBase { if (this.traceMode !== 'trace' || !this.#browser) { return } - // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + // Reads, waits and lookups inherit the earlier capture; only the test's first mapped command adds an initial frame. const initialFrame = this.awaitingInitialFrame && !!mapCommandToAction(command) // `url()` without a string argument is a getter; it does not navigate. const pageChanging = - changesPage(command) && + !skipsCapture(command) && !(command === 'url' && typeof args[0] !== 'string') if (!pageChanging && !initialFrame) { return @@ -202,6 +208,7 @@ export class SessionCapturer extends SessionCapturerBase { if (initialFrame) { this.awaitingInitialFrame = false } + this.lastCapturedTimestamp = timestamp const generation = this.testCaptureGeneration this.snapshotCaptures.push( captureActionSnapshot( @@ -626,6 +633,33 @@ export class SessionCapturer extends SessionCapturerBase { } /** At a test or scenario end, before its slice is written: the proxy drain there is never throttled. */ + /** Rows after the last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ + async captureFinalFrame(): Promise { + await Promise.all(this.snapshotCaptures) + const stamp = finalFrameStamp( + this.commandsLog, + this.testStartWallTime, + this.lastCapturedTimestamp + ) + if (this.traceMode !== 'trace' || !this.#browser || !stamp) { + return + } + try { + const snap = await captureActionSnapshot( + this.#browser, + stamp.command, + stamp.timestamp, + this.runner, + this.isNativeAppSession + ) + if (snap) { + replaceSnapshotAt(this.actionSnapshots, snap) + } + } catch (err) { + log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) + } + } + async captureTraceAtTestEnd(browser: NightwatchBrowser) { if (this.isNativeAppSession) { await this.nativeNetwork.forward() diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index a3a1ef3f9..a1c7c794f 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -210,4 +210,59 @@ describe('nightwatch action-snapshot timestamp binding', () => { expect(cap.actionSnapshots).toHaveLength(0) cap.cleanup() }) + + it('captures an unmapped command that can change the page, but not a lookup', async () => { + const cap = makeCapturer() + const at = (command: string, args: unknown[], timestamp: number) => + cap.captureCommand(command, args, undefined, undefined, { timestamp }) + await at('submitForm', ['#form'], 1) + await at('findElement', ['#el'], 2) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + cap.cleanup() + }) + + it('takes an end-of-test frame that replaces the last capture when reads follow it', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + type Snap = Awaited> + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'FINAL' + } as Snap) + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]!.screenshot).toBe('FINAL') + cap.cleanup() + }) + + it('takes no end-of-test frame when the test ends on a capturing command', async () => { + const cap = makeCapturer() + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: 1 + }) + await cap.captureCommand('click', ['#el'], undefined, undefined, { + timestamp: 2 + }) + await Promise.all(cap.snapshotCaptures) + + vi.mocked(coreCapture).mockClear() + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + cap.cleanup() + }) }) diff --git a/packages/selenium-devtools-py/changes/page-changing-capture.md b/packages/selenium-devtools-py/changes/page-changing-capture.md index 02160ba61..b00a731a6 100644 --- a/packages/selenium-devtools-py/changes/page-changing-capture.md +++ b/packages/selenium-devtools-py/changes/page-changing-capture.md @@ -2,4 +2,4 @@ patch --- -Trace mode reads each page's element tree only after a command that changes the page, and after each test's first command, instead of after every command. Reads and waits keep their own rows and show the tree from the last page change. +Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. A test whose last page change is followed by reads also gets one more read of the page it ended on. Reads and waits keep their own rows and show the tree from the last capture before them. diff --git a/packages/selenium-devtools-py/scripts/gen_contract.py b/packages/selenium-devtools-py/scripts/gen_contract.py index 28a869371..6639a5eba 100644 --- a/packages/selenium-devtools-py/scripts/gen_contract.py +++ b/packages/selenium-devtools-py/scripts/gen_contract.py @@ -205,17 +205,18 @@ def _runner_cwd_env(runner_ts: str) -> str: def _action_commands(schema_path: Path) -> dict[str, list[str]]: """The runner-native command names the capture gate reads, evaluated from - shared's ACTION_MAP by ``wire-schema.ts`` rather than restated here. + shared's ``skipsCapture`` by ``wire-schema.ts`` rather than restated here. - Generated because a name that drifts is silent: a command missing from the - page-changing set is never captured, so its row shows the page before it. + Generated because a name that drifts is silent: a read missing from the + read-only set is captured on every poll, and a page change missing from it + is never captured, so its row shows the page before it. """ if not schema_path.exists(): raise SystemExit( f"{schema_path} is missing: run `npx tsx scripts/wire-schema.ts` in packages/shared" ) sets = json.loads(schema_path.read_text()).get("actionCommands") - if not sets or "pageChanging" not in sets or "mapped" not in sets: + if not sets or "readOnly" not in sets or "mapped" not in sets: raise SystemExit( "contract drift: `actionCommands` is missing from wire-schema.json; " "regenerate it with packages/shared/scripts/wire-schema.ts" @@ -430,7 +431,7 @@ def main() -> int: lines += [ "", f"DATA_SCOPES = frozenset({sorted(data_keys)!r})", - f"PAGE_CHANGING_COMMANDS = frozenset({action_commands['pageChanging']!r})", + f"READ_ONLY_COMMANDS = frozenset({action_commands['readOnly']!r})", f"MAPPED_COMMANDS = frozenset({action_commands['mapped']!r})", f"CONTROL_SCOPES = frozenset({sorted(control.values())!r})", "", diff --git a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py index ddf904e18..da98dd0e7 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py @@ -15,7 +15,7 @@ SCOPE_REPLACE_COMMAND = "replaceCommand" DATA_SCOPES = frozenset(['actionSnapshots', 'commands', 'config', 'consoleLogs', 'logs', 'metadata', 'mutations', 'networkRequests', 'screencast', 'screencastFrames', 'sources', 'suites']) -PAGE_CHANGING_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'goBack', 'goForward', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url']) +READ_ONLY_COMMANDS = frozenset(['$', '$$', 'elementScreenshot', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getHTML', 'getLocation', 'getLog', 'getPageSource', 'getProperty', 'getRect', 'getScreenOrientation', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowRect', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'screenshot', 'status', 'takeScreenshot', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) MAPPED_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'getAttribute', 'getCSSProperty', 'getCssValue', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getLocation', 'getPageSource', 'getProperty', 'getRect', 'getSize', 'getTagName', 'getText', 'getTitle', 'getUrl', 'getValue', 'goBack', 'goForward', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) CONTROL_SCOPES = frozenset(['clearCommands', 'clearExecutionData', 'clientConnected', 'clientDisconnected', 'config', 'replaceCommand', 'testStopped']) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index 3c0fda94c..f880c1a8c 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -30,7 +30,7 @@ read_value_locators, wait_tracking, ) -from ._contract import MAPPED_COMMANDS, PAGE_CHANGING_COMMANDS +from ._contract import MAPPED_COMMANDS, READ_ONLY_COMMANDS from .assert_tracer import ScriptAssertionTracer from .capturer import SessionCapturer from .cdp_screencast import start_push_screencast @@ -292,6 +292,12 @@ def _capture_source(capturer: SessionCapturer, call_src: Optional[str]) -> None: # Re-armed per test by reset_test_capture(). "awaiting_initial_frame": True, "initial_frame_retried": False, + # The test's command rows as (command, timestamp), and the timestamp of the + # last capture attempted — what the end-of-test capture is decided from. + "test_rows": [], + "test_started_ms": 0, + "last_captured_ts": None, + "final_frame_taken": False, } @@ -425,7 +431,13 @@ def resolved_output_dir() -> Optional[str]: return _state.get("output_dir") -_SUBMIT_SCRIPT_MARKER = "/* submitForm */" +# The atoms Selenium's own WebElement methods run as scripts, tagged with a +# leading comment; the command each one stands for. +_ATOM_SCRIPT_COMMANDS = { + "/* submitForm */": "submit", + "/* getAttribute */": "getElementAttribute", + "/* isDisplayed */": "isDisplayed", +} def _action_capture_enabled(driver: Any) -> bool: @@ -481,15 +493,17 @@ def _capture_action_snapshot( def _maybe_capture_action_snapshot( driver: Any, command: str, timestamp: int, shot: Optional[str] ) -> None: - """Capture on a page-changing command, and on a test's first mapped command - as its initial frame. Reads and waits inherit the capture before them.""" + """Capture on every command that is not a known read, and on a test's first + mapped command as its initial frame. Reads and waits inherit the capture + before them.""" initial = _state["awaiting_initial_frame"] and command in MAPPED_COMMANDS - if command not in PAGE_CHANGING_COMMANDS and not initial: + if command in READ_ONLY_COMMANDS and not initial: return if not _action_capture_enabled(driver): return if initial: _state["awaiting_initial_frame"] = False + _state["last_captured_ts"] = timestamp snapshot = _capture_action_snapshot(driver, command, timestamp, shot) if not initial or _state["initial_frame_retried"]: return @@ -499,16 +513,80 @@ def _maybe_capture_action_snapshot( def _capture_command(driver_command: str, params: Any) -> str: - """Selenium's `WebElement.submit()` reaches `execute` as a marked script.""" + """The command a script-run atom stands for: Selenium's `WebElement` methods + reach `execute` as marked scripts, not as the command they perform.""" script = params.get("script") if isinstance(params, dict) else None - if isinstance(script, str) and script.startswith(_SUBMIT_SCRIPT_MARKER): - return "submit" + if isinstance(script, str): + for marker, command in _ATOM_SCRIPT_COMMANDS.items(): + if script.startswith(marker): + return command return driver_command +def _final_frame_stamp( + rows: List[tuple], test_started_ms: int, last_captured_ts: Optional[int] +) -> Optional[tuple]: + """Mirrors core's `finalFrameStamp`: the (command, timestamp) the end-of-test + capture replaces, or None when no row follows the last capture.""" + in_test = [row for row in rows if row[1] >= test_started_ms] + if not in_test: + return None + if last_captured_ts is None or last_captured_ts < test_started_ms: + return in_test[-1] + if not any(ts > last_captured_ts for _, ts in in_test): + return None + at_or_before = [row for row in in_test if row[1] <= last_captured_ts] + owner = at_or_before[-1] if at_or_before else in_test[0] + return (owner[0], last_captured_ts) + + +def _capture_final_frame(driver: Any) -> bool: + if not _action_capture_enabled(driver): + return False + stamp = _final_frame_stamp( + _state["test_rows"], _state["test_started_ms"], _state["last_captured_ts"] + ) + if stamp is None: + return False + command, timestamp = stamp + snapshot = _capture_action_snapshot(driver, command, timestamp, _take_screenshot(driver)) + if snapshot is None: + return False + _state["action_snapshots"][:] = [ + kept + for kept in _state["action_snapshots"] + if kept["timestamp"] != timestamp or kept is snapshot + ] + _state["last_captured_ts"] = timestamp + return True + + +def capture_final_frame(driver: Any = None) -> None: + """Replace the test's last capture with the page the test ended on, when + reads followed it. + + The last page-changing command can return before its navigation commits, + so its capture shows the page it left; the reads after it saw the new one. + Once per test: the call report and the driver's quit both reach here. + """ + if _state["final_frame_taken"]: + return + for live in [driver] if driver is not None else list(_state["sessions"]): + try: + if _capture_final_frame(live): + _state["final_frame_taken"] = True + return + except Exception as exc: # noqa: BLE001 — a missing end frame, not a failed run + _log.debug("final frame capture failed: %s", exc) + + def reset_test_capture() -> None: - """Re-arm the initial frame for the test about to start.""" - _state.update(awaiting_initial_frame=True, initial_frame_retried=False) + """Re-arm the initial frame and start the row ledger for the test about to start.""" + _state.update( + awaiting_initial_frame=True, initial_frame_retried=False, + test_rows=[], test_started_ms=now_ms(), + last_captured_ts=None, final_frame_taken=False, + ) def _begin_screencast_run(entry: Optional[dict], shot: Optional[str] = None) -> None: @@ -1056,6 +1134,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN # Finalize the screencast BEFORE quit tears the session down — after # orig_execute the driver's WebSocket/session is gone. if driver_command == "quit": + capture_final_frame(self) _on_quit(capturer, self) elif driver_command == "newSession": _enable_bidi_capability(params) # request BiDi before the session opens @@ -1084,6 +1163,9 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, failed) + _state["test_rows"].append( + (_capture_command(driver_command, params), failed["timestamp"]) + ) raise # WebDriver.execute returns the full response dict; the useful payload @@ -1107,12 +1189,10 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, row) - _maybe_capture_action_snapshot( - self, - _capture_command(driver_command, params), - row.get("timestamp", start), - shot, - ) + command = _capture_command(driver_command, params) + stamp = row.get("timestamp", start) + _state["test_rows"].append((command, stamp)) + _maybe_capture_action_snapshot(self, command, stamp, shot) if driver_command in NAVIGATION_COMMANDS: _attach_performance(capturer, self, row, params) # No per-command line here: the Actions timeline lists every command as @@ -1142,6 +1222,8 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN trace=trace, filmstrip=filmstrip, filmstrip_frames=[], a11y=a11y, element_scripts=None, page_scripts=None, action_snapshots=[], awaiting_initial_frame=True, initial_frame_retried=False, + test_rows=[], test_started_ms=now_ms(), last_captured_ts=None, + final_frame_taken=False, ) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py index cccaa327b..b4191cca1 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py @@ -644,6 +644,8 @@ def pytest_runtest_logreport(report) -> None: # noqa: ANN001 return from . import instrumentation + if report.when == "call": + instrumentation.capture_final_frame() instrumentation.drain_native_network(capturer) file, line, name = report.location _registry.record( diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index 2104ac85b..598af9757 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -15,7 +15,7 @@ from selenium_devtools._contract import ( ELEMENT_SCRIPTS_PATH, MAPPED_COMMANDS, - PAGE_CHANGING_COMMANDS, + READ_ONLY_COMMANDS, RUNNER_ID, SCOPE_ACTION_SNAPSHOTS, ) @@ -300,13 +300,143 @@ class NativeDriver(self.Driver): self.assertTrue(instrumentation._state["awaiting_initial_frame"]) self.assertEqual(driver.scripts_run, []) + # Deny-list: a command the set does not name is captured, so only commands + # known not to change the page may be listed. def test_the_generated_sets_agree_with_the_gate(self): - self.assertIn("get", PAGE_CHANGING_COMMANDS) - self.assertIn("clickElement", PAGE_CHANGING_COMMANDS) - self.assertNotIn("getElementText", PAGE_CHANGING_COMMANDS) + self.assertIn("findElement", READ_ONLY_COMMANDS) + self.assertIn("getElementText", READ_ONLY_COMMANDS) + self.assertIn("isDisplayed", READ_ONLY_COMMANDS) + self.assertIn("getElementAttribute", READ_ONLY_COMMANDS) + self.assertTrue( + READ_ONLY_COMMANDS.isdisjoint({"get", "clickElement", "submit", "actions", "w3cExecuteScript"}) + ) self.assertIn("getElementText", MAPPED_COMMANDS) self.assertNotIn("findElement", MAPPED_COMMANDS) - self.assertLessEqual(PAGE_CHANGING_COMMANDS, MAPPED_COMMANDS) + + def test_a_script_the_user_runs_is_a_page_change_and_captured(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "w3cExecuteScript") + self._gate(driver, "actions") + self.assertEqual(self._captured(), ["clickElement", "w3cExecuteScript", "actions"]) + + def test_a_lookup_and_a_text_read_after_a_click_are_not_captured(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "findElement") + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["clickElement"]) + + def test_selenium_atom_scripts_are_read_as_the_reads_they_are(self): + is_displayed = {"script": "/* isDisplayed */return (function(){}).apply(null, arguments);", "args": []} + get_attribute = {"script": "/* getAttribute */return (function(){}).apply(null, arguments);", "args": []} + self.assertEqual(instrumentation._capture_command("w3cExecuteScript", is_displayed), "isDisplayed") + self.assertEqual(instrumentation._capture_command("w3cExecuteScript", get_attribute), "getElementAttribute") + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", is_displayed)) + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", get_attribute)) + self.assertEqual(self._captured(), ["clickElement"]) + self.assertEqual(len(driver.scripts_run), 2, "read the page for an atom read") + + +class TestTheEndOfATest(unittest.TestCase): + """The last page change can return before its navigation commits, so its + capture shows the page it left. The reads after it saw the new page, and + the end of the test is captured again to replace that stale capture.""" + + class Driver: + session_id = "s" + + def __init__(self): + self.tree = [{"selector": "#before"}] + self.scripts_run = 0 + + def execute_script(self, script, *args): + self.scripts_run += 1 + return self.tree + + @contextlib.contextmanager + def _ended(self): + with mock.patch.dict( + instrumentation._state, + { + "trace": True, + "a11y": True, + "element_scripts": SCRIPTS, + "action_snapshots": [], + "awaiting_initial_frame": False, + "initial_frame_retried": False, + "test_rows": [], + "test_started_ms": 1000, + "last_captured_ts": None, + "final_frame_taken": False, + }, + ): + yield + + def _step(self, driver, command, timestamp): + instrumentation._state["test_rows"].append((command, timestamp)) + instrumentation._maybe_capture_action_snapshot(driver, command, timestamp, None) + + def test_the_reads_after_a_click_replace_its_stale_capture(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + driver.tree = [{"selector": "#after"}] + self._step(driver, "getElementText", 1300) + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([s["command"] for s in snaps], ["clickElement"]) + self.assertEqual(snaps[0]["timestamp"], 1200) + self.assertEqual(snaps[0]["elements"][0]["selector"], "#after") + + def test_a_test_that_ends_on_a_capture_takes_no_final_one(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + reads = driver.scripts_run + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, reads, "re-read a page nothing followed") + self.assertEqual(len(instrumentation.action_snapshots()), 1) + + def test_a_test_with_no_capture_is_stamped_at_its_last_command(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "findElement", 1200) + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([(s["command"], s["timestamp"]) for s in snaps], [("findElement", 1200)]) + + def test_a_test_with_no_rows_takes_no_final_capture(self): + driver = self.Driver() + with self._ended(): + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, 0) + self.assertEqual(instrumentation.action_snapshots(), []) + + def test_the_final_capture_is_taken_once_per_test(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + self._step(driver, "getElementText", 1300) + instrumentation.capture_final_frame(driver) + reads = driver.scripts_run + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, reads, "the quit repeated the call report's capture") + + def test_a_row_before_the_test_started_does_not_stamp_it(self): + self.assertIsNone( + instrumentation._final_frame_stamp([("clickElement", 500)], 1000, None) + ) + + def test_the_stamp_follows_core_final_frame_stamp(self): + stamp = instrumentation._final_frame_stamp + rows = [("clickElement", 1200), ("getElementText", 1300)] + self.assertEqual(stamp(rows, 1000, None), ("getElementText", 1300)) + self.assertEqual(stamp(rows, 1000, 1200), ("clickElement", 1200)) + self.assertIsNone(stamp(rows[:1], 1000, 1200)) + self.assertIsNone(stamp([], 1000, None)) class TestStreamingTheSnapshots(unittest.TestCase): diff --git a/packages/selenium-devtools-py/tests/test_pytest_plugin.py b/packages/selenium-devtools-py/tests/test_pytest_plugin.py index 9bda8d391..795bdf2a5 100644 --- a/packages/selenium-devtools-py/tests/test_pytest_plugin.py +++ b/packages/selenium-devtools-py/tests/test_pytest_plugin.py @@ -565,3 +565,30 @@ def test_a_test_starting_re_arms_the_initial_frame(self): self.assertIs(instrumentation._state["awaiting_initial_frame"], True) self.assertIs(instrumentation._state["initial_frame_retried"], False) + + +class TestTheFinalFrameIsTakenAtTheCallReport(unittest.TestCase): + """The call report lands while the driver is still alive, before the + fixture teardown that quits it. Setup and teardown reports are not the + test's last page.""" + + def _log(self, when): + report = _Report(when, "passed") + report.nodeid = "tests/t.py::test_a" + report.location = ("tests/t.py", 1, "test_a") + with mock.patch.object(plugin, "_opted_in", return_value=True), mock.patch.object( + plugin.devtools, "get_capturer", return_value=object() + ), mock.patch.object(plugin, "_publish"), mock.patch.object( + plugin, "_registry" + ), mock.patch.object(instrumentation, "drain_native_network"), mock.patch.object( + instrumentation, "capture_final_frame" + ) as final: + plugin.pytest_runtest_logreport(report) + return final + + def test_the_call_report_takes_the_final_frame(self): + self._log("call").assert_called_once_with() + + def test_the_setup_and_teardown_reports_do_not(self): + self._log("setup").assert_not_called() + self._log("teardown").assert_not_called() diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index b8df0e357..81659c551 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -11,7 +11,7 @@ import { type CapturedPerformancePayload, type RetryTracker } from '@wdio/devtools-core' -import { changesPage } from '@wdio/devtools-shared' +import { skipsCapture } from '@wdio/devtools-shared' import { ELEMENT_LOCATOR_METHODS } from '../constants.js' import { getDriverOriginals, getElementOriginals } from '../driverPatcher.js' import { captureOrReplaceCommand } from './captureOrReplaceCommand.js' @@ -366,15 +366,16 @@ function queueActionSnapshot( if (ctx.options.mode !== 'trace' || error || !ctx.driver) { return } - // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + // Reads, waits and lookups inherit the earlier capture; only the test's first mapped command adds an initial frame. const initialFrame = capturer.awaitingInitialFrame && !!mapCommandToAction(cmd.command) - if (!changesPage(cmd.command) && !initialFrame) { + if (skipsCapture(cmd.command) && !initialFrame) { return } if (initialFrame) { capturer.awaitingInitialFrame = false } + capturer.lastCapturedTimestamp = timestamp const generation = capturer.testCaptureGeneration ctx.snapshotCaptures.push( captureActionSnapshot( diff --git a/packages/selenium-devtools/src/helpers/finalFrame.ts b/packages/selenium-devtools/src/helpers/finalFrame.ts new file mode 100644 index 000000000..f71192e34 --- /dev/null +++ b/packages/selenium-devtools/src/helpers/finalFrame.ts @@ -0,0 +1,47 @@ +import logger from '@wdio/logger' +import { + errorMessage, + finalFrameStamp, + replaceSnapshotAt +} from '@wdio/devtools-core' +import { captureActionSnapshot } from '../action-snapshot.js' +import type { SessionLifecycleCtx } from '../session-lifecycle.js' + +const log = logger('@wdio/selenium-devtools:finalFrame') + +/** Rows after the test's last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ +export async function captureFinalFrame( + ctx: SessionLifecycleCtx +): Promise { + const capturer = ctx.sessionCapturer + if ( + ctx.options.mode !== 'trace' || + ctx.finalized || + !ctx.driver || + !capturer + ) { + return + } + await Promise.all(ctx.snapshotCaptures) + const stamp = finalFrameStamp( + capturer.commandsLog, + capturer.testStartWallTime, + capturer.lastCapturedTimestamp + ) + if (!stamp) { + return + } + try { + const snap = await captureActionSnapshot( + ctx.driver, + stamp.command, + stamp.timestamp, + capturer.isNativeAppSession + ) + if (snap) { + replaceSnapshotAt(ctx.actionSnapshots, snap) + } + } catch (err) { + log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) + } +} diff --git a/packages/selenium-devtools/src/index.ts b/packages/selenium-devtools/src/index.ts index e607676be..07f389d94 100644 --- a/packages/selenium-devtools/src/index.ts +++ b/packages/selenium-devtools/src/index.ts @@ -25,6 +25,7 @@ import { recordTraceBoundary, flushCurrentTestTrace } from './session-lifecycle.js' +import { captureFinalFrame } from './helpers/finalFrame.js' import type { SpecRange, TraceArtifact } from '@wdio/devtools-core' import { startTest as tmStartTest, @@ -480,6 +481,7 @@ class SeleniumDevToolsPlugin { async endTest(state: TestStats['state'] = 'passed') { const ended = this.#testManager?.getCurrentTest() ?? null tmEndTest(this.#getInternals(), state) + await captureFinalFrame(this.#getInternals()) const flushed = flushCurrentTestTrace(this.#getInternals()) await this.#emitTestArtifacts(ended, state === 'failed', flushed) } @@ -493,6 +495,7 @@ class SeleniumDevToolsPlugin { async endScenario(state: TestStats['state'] = 'passed') { const ended = this.#testManager?.getCurrentTest() ?? null tmEndScenario(this.#getInternals(), state) + await captureFinalFrame(this.#getInternals()) const flushed = flushCurrentTestTrace(this.#getInternals()) await this.#emitTestArtifacts(ended, state === 'failed', flushed) } diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index c659888b9..c90e3bc16 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -59,11 +59,15 @@ export class SessionCapturer extends SessionCapturerBase { awaitingInitialFrame = true initialFrameRetried = false testCaptureGeneration = 0 + /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ + lastCapturedTimestamp: number | undefined = undefined + testStartWallTime = 0 resetTestCapture(): void { this.awaitingInitialFrame = true this.initialFrameRetried = false this.testCaptureGeneration++ + this.testStartWallTime = Date.now() } #clientConnected = false diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index ea677689c..d346f29aa 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -11,6 +11,8 @@ import { type OnCommandCtx } from '../src/helpers/commandPostActions.js' import { captureActionSnapshot } from '../src/action-snapshot.js' +import { captureFinalFrame } from '../src/helpers/finalFrame.js' +import type { SessionLifecycleCtx } from '../src/session-lifecycle.js' import { getDriverOriginals } from '../src/driverPatcher.js' import { RetryTracker, @@ -109,7 +111,7 @@ describe('selenium action-snapshot timestamp binding', () => { ) }) - it('does not capture for a non-mapped command', async () => { + it('does not capture for a read-only lookup', async () => { const { ctx, snapshotCaptures, actionSnapshots } = makeCtx(fakeDriver()) await handleOnCommand( ctx, @@ -165,6 +167,11 @@ describe('selenium action-snapshot capture gate', () => { expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1]) }) + it('captures an unmapped command that can change the page', async () => { + const { actionSnapshots } = await run(['executeScript', 'actions']) + expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + }) + it('captures a leading read as the initial frame, once per test', async () => { const bag = makeCtx(fakeDriver()) await run(['getText', 'isDisplayed'], bag) @@ -258,6 +265,47 @@ describe('selenium action-snapshot capture gate', () => { expect(bag.capturer.initialFrameRetried).toBe(false) expect(bag.capturer.awaitingInitialFrame).toBe(true) }) + + describe('end-of-test frame', () => { + const endOfTest = (bag: ReturnType) => + captureFinalFrame(bag.ctx as unknown as SessionLifecycleCtx) + + it('replaces the last capture with the end state when reads follow it', async () => { + const bag = await run(['click', 'getText']) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + vi.mocked(coreCapture).mockClear() + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'FINAL', + elements: [{ selector: '#final' }] + } as unknown as ActionSnapshot) + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(bag.actionSnapshots).toHaveLength(1) + expect(bag.actionSnapshots[0]!.screenshot).toBe('FINAL') + expect(bag.actionSnapshots[0]!.elements).toEqual([{ selector: '#final' }]) + }) + + it('takes no end-of-test frame when the test ends on a capturing command', async () => { + const bag = await run(['getText', 'click']) + vi.mocked(coreCapture).mockClear() + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + }) + + it('takes no end-of-test frame for a test with no rows', async () => { + const bag = makeCtx(fakeDriver()) + vi.mocked(coreCapture).mockClear() + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + }) + }) }) describe('selenium action-snapshot locator dialect', () => { diff --git a/packages/shared/scripts/wire-schema.ts b/packages/shared/scripts/wire-schema.ts index 305e9cb3b..cdc6ffc7c 100644 --- a/packages/shared/scripts/wire-schema.ts +++ b/packages/shared/scripts/wire-schema.ts @@ -4,7 +4,11 @@ import { realpathSync, writeFileSync } from 'node:fs' import path from 'node:path' import { fileURLToPath } from 'node:url' import ts from 'typescript' -import { ACTION_MAP, changesPage } from '../src/trace-actions.js' +import { + ACTION_MAP, + READ_ONLY_COMMANDS, + skipsCapture +} from '../src/trace-actions.js' export type JsonSchema = { [key: string]: unknown } @@ -14,7 +18,10 @@ export interface WireSchema { scopes: Record $defs: Record // Runner-native command names, evaluated from ACTION_MAP rather than typed out. - actionCommands: { pageChanging: string[]; mapped: string[] } + actionCommands: { + mapped: string[] + readOnly: string[] + } } const HERE = path.dirname(fileURLToPath(import.meta.url)) @@ -250,9 +257,10 @@ export function buildWireSchema(): WireSchema { // Every ACTION_MAP key is mapped by mapCommandToAction (its fallback only adds assert.*, which no WebDriver hook sees). function actionCommandSets(): WireSchema['actionCommands'] { const commands = Object.keys(ACTION_MAP).sort() + const known = [...new Set([...commands, ...READ_ONLY_COMMANDS])].sort() return { - pageChanging: commands.filter((c) => changesPage(c)), - mapped: commands + mapped: commands, + readOnly: known.filter((c) => skipsCapture(c)) } } diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index 58e77dffa..91e46176e 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -277,3 +277,50 @@ export function changesPage(command: string): boolean { } return METHOD_EFFECT[ACTION_MAP[command].method] === 'changes' } + +/** Driver lookups and reads that never change the page and have no ACTION_MAP + * entry; the ACTION_MAP reads are covered by METHOD_EFFECT in `skipsCapture`. */ +export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ + 'findElement', + 'findElements', + 'findChildElement', + 'findChildElements', + '$', + '$$', + 'findElementFromShadowRoot', + 'findElementsFromShadowRoot', + 'getElementShadowRoot', + 'getActiveElement', + 'getWindowHandle', + 'getWindowHandles', + 'getCurrentContextHandle', + 'getContextHandles', + 'getCookies', + 'getAllCookies', + 'getCookie', + 'getWindowRect', + 'getLog', + 'getAvailableLogTypes', + 'getAlertText', + 'getTimeouts', + 'getScreenOrientation', + 'status', + 'takeScreenshot', + 'screenshot', + 'elementScreenshot', + 'getHTML', + 'getComputedRole', + 'getComputedLabel' +]) + +/** Whether a command is known not to change the page, so its capture can be + * skipped. Deny-list by design: an unlisted command still captures. */ +export function skipsCapture(command: string): boolean { + if (READ_ONLY_COMMANDS.has(command) || mapAssertCommand(command) !== null) { + return true + } + if (!Object.prototype.hasOwnProperty.call(ACTION_MAP, command)) { + return false + } + return METHOD_EFFECT[ACTION_MAP[command].method] === 'reads' +} diff --git a/packages/shared/tests/change-page.test.ts b/packages/shared/tests/change-page.test.ts index 611b7dde9..f5dae4727 100644 --- a/packages/shared/tests/change-page.test.ts +++ b/packages/shared/tests/change-page.test.ts @@ -1,5 +1,10 @@ import { describe, it, expect } from 'vitest' -import { ACTION_MAP, METHOD_EFFECT, changesPage } from '@wdio/devtools-shared' +import { + ACTION_MAP, + METHOD_EFFECT, + changesPage, + skipsCapture +} from '@wdio/devtools-shared' describe('changesPage', () => { it.each([ @@ -35,3 +40,29 @@ describe('changesPage', () => { } }) }) + +describe('skipsCapture', () => { + it.each([ + 'getText', + 'waitForDisplayed', + 'findElement', + 'getCurrentUrl', + 'assert.equal', + 'expect.toHaveText', + 'getElementText' + ])('is true for %s', (command) => { + expect(skipsCapture(command)).toBe(true) + }) + + it.each([ + 'click', + 'executeScript', + 'actions', + 'submitForm', + 'acceptAlert', + 'w3cExecuteScript', + 'someUnknownCommand' + ])('is false for %s', (command) => { + expect(skipsCapture(command)).toBe(false) + }) +}) diff --git a/packages/shared/tests/wire-schema.test.ts b/packages/shared/tests/wire-schema.test.ts index fa0952c97..80b061246 100644 --- a/packages/shared/tests/wire-schema.test.ts +++ b/packages/shared/tests/wire-schema.test.ts @@ -39,6 +39,17 @@ describe('wire schema', () => { }) }) + it('lists the read-only commands the capture gate skips', () => { + expect(schema.actionCommands.readOnly).toEqual( + expect.arrayContaining([ + 'findElement', + 'getElementText', + 'waitForDisplayed' + ]) + ) + expect(schema.actionCommands.readOnly).not.toContain('click') + }) + it('describes a Date as the ISO string it serializes to', () => { const props = (schema.$defs.TestStats.properties ?? {}) as Record< string, diff --git a/packages/shared/wire-schema.json b/packages/shared/wire-schema.json index 8564ce392..5dc4d5a82 100644 --- a/packages/shared/wire-schema.json +++ b/packages/shared/wire-schema.json @@ -1435,39 +1435,6 @@ } }, "actionCommands": { - "pageChanging": [ - "addValue", - "back", - "clear", - "clearElement", - "click", - "clickElement", - "doubleClick", - "dragAndDrop", - "execute", - "executeAsync", - "forward", - "get", - "goBack", - "goForward", - "keys", - "moveTo", - "navigateTo", - "newWindow", - "refresh", - "scrollIntoView", - "selectByVisibleText", - "sendKeys", - "sendKeysToElement", - "setValue", - "submit", - "switchToFrame", - "switchToParentFrame", - "switchToWindow", - "to", - "touchAction", - "url" - ], "mapped": [ "addValue", "back", @@ -1539,6 +1506,77 @@ "waitForEnabled", "waitForExist", "waitUntil" + ], + "readOnly": [ + "$", + "$$", + "elementScreenshot", + "findChildElement", + "findChildElements", + "findElement", + "findElementFromShadowRoot", + "findElements", + "findElementsFromShadowRoot", + "getActiveElement", + "getAlertText", + "getAllCookies", + "getAttribute", + "getAvailableLogTypes", + "getCSSProperty", + "getComputedLabel", + "getComputedRole", + "getContextHandles", + "getCookie", + "getCookies", + "getCssValue", + "getCurrentContextHandle", + "getCurrentUrl", + "getElementAttribute", + "getElementProperty", + "getElementRect", + "getElementShadowRoot", + "getElementTagName", + "getElementText", + "getElementValueOfCssProperty", + "getHTML", + "getLocation", + "getLog", + "getPageSource", + "getProperty", + "getRect", + "getScreenOrientation", + "getSize", + "getTagName", + "getText", + "getTimeouts", + "getTitle", + "getUrl", + "getValue", + "getWindowHandle", + "getWindowHandles", + "getWindowRect", + "isClickable", + "isDisplayed", + "isElementEnabled", + "isElementSelected", + "isEnabled", + "isExisting", + "isFocused", + "isSelected", + "screenshot", + "status", + "takeScreenshot", + "wait", + "waitForClickable", + "waitForDisplayed", + "waitForElementLocated", + "waitForElementNotPresent", + "waitForElementNotVisible", + "waitForElementPresent", + "waitForElementVisible", + "waitForEnabled", + "waitForExist", + "waitUntil" ] } } diff --git a/packages/trace/src/trace-snapshots.ts b/packages/trace/src/trace-snapshots.ts index 9633ea80d..026939fb2 100644 --- a/packages/trace/src/trace-snapshots.ts +++ b/packages/trace/src/trace-snapshots.ts @@ -70,6 +70,19 @@ export function upsertRichestSnapshot( mergeSnapshotInto(snapshots[idx]!, snap) } +/** Replace whatever holds the snapshot's timestamp, unlike upsertRichestSnapshot, for a capture that is strictly later state and so supersedes its fields. */ +export function replaceSnapshotAt( + snapshots: ActionSnapshot[], + snap: ActionSnapshot +): void { + for (let i = snapshots.length - 1; i >= 0; i--) { + if (snapshots[i]!.timestamp === snap.timestamp) { + snapshots.splice(i, 1) + } + } + snapshots.push(snap) +} + export function buildSnapshotResources( rawSnapshots: ActionSnapshot[], pageId: string diff --git a/packages/trace/tests/trace-snapshots.test.ts b/packages/trace/tests/trace-snapshots.test.ts index 3339d7f4d..e706cc698 100644 --- a/packages/trace/tests/trace-snapshots.test.ts +++ b/packages/trace/tests/trace-snapshots.test.ts @@ -3,6 +3,7 @@ import os from 'node:os' import path from 'node:path' import { describe, it, expect } from 'vitest' import { + replaceSnapshotAt, upsertRichestSnapshot, writeTraceZip, type TraceCapturer @@ -188,3 +189,28 @@ describe('snapshot binding contract (capture → export)', () => { await fs.rm(outputDir, { recursive: true, force: true }) }) }) + +describe('replaceSnapshotAt', () => { + it('replaces the record at its timestamp whole, elements included', () => { + const list = [ + snap(50, REAL), + snap(100, REAL, { elements: [{ selector: '#old' }] }) + ] + replaceSnapshotAt(list, snap(100, BLANK, { snapshotText: 'tree' })) + expect(list.filter((s) => s.timestamp === 100)).toEqual([ + { + timestamp: 100, + command: 'click', + screenshot: BLANK, + snapshotText: 'tree' + } + ]) + expect(list).toHaveLength(2) + }) + + it('appends when no record holds the timestamp', () => { + const list = [snap(50, REAL)] + replaceSnapshotAt(list, snap(100, BLANK)) + expect(list.map((s) => s.timestamp)).toEqual([50, 100]) + }) +}) From f51a4e8ef58bf3f9b7cedb2f2d5400d666aac09c Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 17:17:44 +0100 Subject: [PATCH 08/14] fix(trace): skip the reads the adapters log beyond ACTION_MAP, and keep what a partial end capture lacks The deny-list missed Nightwatch's own reads (isVisible, isPresent, title, source, getCssProperty, element/elements and more), selenium-webdriver's getDomAttribute/getId/getCapabilities/getSession, and selenium-python's w3cGet*/getElementAria* names, so polling them took a full capture again. An end-of-test capture whose screenshot timed out replaced the last good one whole; it now supersedes field-wise. Python's end capture is taken through the driver that took the test's last capture, so a second driver neither lends its page nor, by quitting, uses up the owner's end capture. Co-Authored-By: Claude Opus 5.5 --- packages/nightwatch-devtools/src/session.ts | 4 +- .../src/selenium_devtools/_contract.py | 2 +- .../src/selenium_devtools/instrumentation.py | 43 +++++++++++++------ .../tests/test_a11y_elements.py | 43 +++++++++++++++++++ .../src/helpers/finalFrame.ts | 4 +- packages/shared/src/trace-actions.ts | 35 ++++++++++++++- packages/shared/tests/change-page.test.ts | 7 ++- packages/shared/wire-schema.json | 35 ++++++++++++++- packages/trace/src/trace-snapshots.ts | 14 ++++-- packages/trace/tests/trace-snapshots.test.ts | 29 ++++++++++--- 10 files changed, 187 insertions(+), 29 deletions(-) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 61fba27ec..570a7b6d9 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -11,7 +11,7 @@ import { mapCommandToAction, NativeNetworkCapture, pollUntilReady, - replaceSnapshotAt, + supersedeSnapshotAt, serializeError, upsertRichestSnapshot, type CapturedPerformancePayload, @@ -653,7 +653,7 @@ export class SessionCapturer extends SessionCapturerBase { this.isNativeAppSession ) if (snap) { - replaceSnapshotAt(this.actionSnapshots, snap) + supersedeSnapshotAt(this.actionSnapshots, snap) } } catch (err) { log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py index da98dd0e7..6040bded3 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py @@ -15,7 +15,7 @@ SCOPE_REPLACE_COMMAND = "replaceCommand" DATA_SCOPES = frozenset(['actionSnapshots', 'commands', 'config', 'consoleLogs', 'logs', 'metadata', 'mutations', 'networkRequests', 'screencast', 'screencastFrames', 'sources', 'suites']) -READ_ONLY_COMMANDS = frozenset(['$', '$$', 'elementScreenshot', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getHTML', 'getLocation', 'getLog', 'getPageSource', 'getProperty', 'getRect', 'getScreenOrientation', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowRect', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'screenshot', 'status', 'takeScreenshot', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) +READ_ONLY_COMMANDS = frozenset(['$', '$$', 'element', 'elementScreenshot', 'elements', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getAccessibleName', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAriaRole', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getCapabilities', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCssProperty', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getDomAttribute', 'getElementAriaLabel', 'getElementAriaRole', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementSize', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getFirstElementChild', 'getHTML', 'getId', 'getLastElementChild', 'getLocation', 'getLog', 'getLogTypes', 'getNetworkConnection', 'getNextSibling', 'getPageSource', 'getPreviousSibling', 'getProperty', 'getRect', 'getScreenOrientation', 'getSession', 'getShadowRoot', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowPosition', 'getWindowRect', 'getWindowSize', 'hasDescendants', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isLogAvailable', 'isPresent', 'isSelected', 'isVisible', 'screenshot', 'source', 'status', 'takeScreenshot', 'title', 'w3cGetActiveElement', 'w3cGetAlertText', 'w3cGetCurrentWindowHandle', 'w3cGetWindowHandles', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil', 'windowHandle', 'windowHandles']) MAPPED_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'getAttribute', 'getCSSProperty', 'getCssValue', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getLocation', 'getPageSource', 'getProperty', 'getRect', 'getSize', 'getTagName', 'getText', 'getTitle', 'getUrl', 'getValue', 'goBack', 'goForward', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) CONTROL_SCOPES = frozenset(['clearCommands', 'clearExecutionData', 'clientConnected', 'clientDisconnected', 'config', 'replaceCommand', 'testStopped']) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index f880c1a8c..3d1dd4189 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -297,6 +297,7 @@ def _capture_source(capturer: SessionCapturer, call_src: Optional[str]) -> None: "test_rows": [], "test_started_ms": 0, "last_captured_ts": None, + "final_frame_driver": None, "final_frame_taken": False, } @@ -497,6 +498,7 @@ def _maybe_capture_action_snapshot( mapped command as its initial frame. Reads and waits inherit the capture before them.""" initial = _state["awaiting_initial_frame"] and command in MAPPED_COMMANDS + _note_final_frame_driver(driver) if command in READ_ONLY_COMMANDS and not initial: return if not _action_capture_enabled(driver): @@ -504,6 +506,7 @@ def _maybe_capture_action_snapshot( if initial: _state["awaiting_initial_frame"] = False _state["last_captured_ts"] = timestamp + _state["final_frame_driver"] = weakref.ref(driver) snapshot = _capture_action_snapshot(driver, command, timestamp, shot) if not initial or _state["initial_frame_retried"]: return @@ -540,6 +543,12 @@ def _final_frame_stamp( return (owner[0], last_captured_ts) +def _note_final_frame_driver(driver: Any) -> None: + """Until a capture is taken, the test's stamp belongs to the driver that ran its latest row.""" + if _state["last_captured_ts"] is None: + _state["final_frame_driver"] = weakref.ref(driver) + + def _capture_final_frame(driver: Any) -> bool: if not _action_capture_enabled(driver): return False @@ -552,11 +561,14 @@ def _capture_final_frame(driver: Any) -> bool: snapshot = _capture_action_snapshot(driver, command, timestamp, _take_screenshot(driver)) if snapshot is None: return False + superseded: dict = {} + for kept in _state["action_snapshots"]: + if kept["timestamp"] == timestamp and kept is not snapshot: + superseded.update(kept) _state["action_snapshots"][:] = [ - kept - for kept in _state["action_snapshots"] - if kept["timestamp"] != timestamp or kept is snapshot + kept for kept in _state["action_snapshots"] if kept["timestamp"] != timestamp ] + _state["action_snapshots"].append({**superseded, **snapshot}) _state["last_captured_ts"] = timestamp return True @@ -567,17 +579,21 @@ def capture_final_frame(driver: Any = None) -> None: The last page-changing command can return before its navigation commits, so its capture shows the page it left; the reads after it saw the new one. - Once per test: the call report and the driver's quit both reach here. + Once per test: the call report and the driver's quit both reach here. Only + the driver that owns the stamp acts, so another session quitting mid-test + neither captures from nor consumes the test's one final frame. """ if _state["final_frame_taken"]: return - for live in [driver] if driver is not None else list(_state["sessions"]): - try: - if _capture_final_frame(live): - _state["final_frame_taken"] = True - return - except Exception as exc: # noqa: BLE001 — a missing end frame, not a failed run - _log.debug("final frame capture failed: %s", exc) + owner_ref = _state["final_frame_driver"] + owner = owner_ref() if owner_ref is not None else None + if owner is None or (driver is not None and driver is not owner): + return + try: + if _capture_final_frame(owner): + _state["final_frame_taken"] = True + except Exception as exc: # noqa: BLE001 — a missing end frame, not a failed run + _log.debug("final frame capture failed: %s", exc) def reset_test_capture() -> None: @@ -585,7 +601,7 @@ def reset_test_capture() -> None: _state.update( awaiting_initial_frame=True, initial_frame_retried=False, test_rows=[], test_started_ms=now_ms(), - last_captured_ts=None, final_frame_taken=False, + last_captured_ts=None, final_frame_driver=None, final_frame_taken=False, ) @@ -1163,6 +1179,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, failed) + _note_final_frame_driver(self) _state["test_rows"].append( (_capture_command(driver_command, params), failed["timestamp"]) ) @@ -1223,7 +1240,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN a11y=a11y, element_scripts=None, page_scripts=None, action_snapshots=[], awaiting_initial_frame=True, initial_frame_retried=False, test_rows=[], test_started_ms=now_ms(), last_captured_ts=None, - final_frame_taken=False, + final_frame_driver=None, final_frame_taken=False, ) diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index 598af9757..542fe6c87 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -425,6 +425,49 @@ def test_the_final_capture_is_taken_once_per_test(self): instrumentation.capture_final_frame(driver) self.assertEqual(driver.scripts_run, reads, "the quit repeated the call report's capture") + def test_a_partial_final_capture_keeps_the_earlier_screenshot(self): + driver = self.Driver() + with self._ended(): + instrumentation._state["test_rows"].append(("clickElement", 1200)) + instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") + driver.tree = [{"selector": "#after"}] + self._step(driver, "getElementText", 1300) + with mock.patch.object(instrumentation, "_take_screenshot", return_value=None): + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual(snaps[0]["screenshot"], "SHOT") + self.assertEqual(snaps[0]["elements"][0]["selector"], "#after") + + def test_the_final_frame_comes_from_the_driver_that_captured_last(self): + owner, other = self.Driver(), self.Driver() + with self._ended(): + self._step(owner, "clickElement", 1200) + before = owner.scripts_run + self._step(other, "getElementText", 1300) + instrumentation.capture_final_frame() + self.assertEqual(owner.scripts_run, before + 2, "the owner was not re-read") + self.assertEqual(other.scripts_run, 0, "a read on another driver took the frame") + + def test_another_drivers_quit_leaves_the_final_frame_to_the_owner(self): + owner, other = self.Driver(), self.Driver() + with self._ended(): + self._step(owner, "clickElement", 1200) + self._step(other, "getElementText", 1300) + instrumentation.capture_final_frame(other) + self.assertFalse(instrumentation._state["final_frame_taken"]) + before = owner.scripts_run + instrumentation.capture_final_frame(owner) + self.assertEqual(owner.scripts_run, before + 2) + self.assertTrue(instrumentation._state["final_frame_taken"]) + + def test_window_and_aria_reads_take_no_capture(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "w3cGetCurrentWindowHandle", 1200) + self._step(driver, "getElementAriaRole", 1300) + self.assertEqual(driver.scripts_run, 0) + self.assertEqual(instrumentation.action_snapshots(), []) + def test_a_row_before_the_test_started_does_not_stamp_it(self): self.assertIsNone( instrumentation._final_frame_stamp([("clickElement", 500)], 1000, None) diff --git a/packages/selenium-devtools/src/helpers/finalFrame.ts b/packages/selenium-devtools/src/helpers/finalFrame.ts index f71192e34..ba2f701dc 100644 --- a/packages/selenium-devtools/src/helpers/finalFrame.ts +++ b/packages/selenium-devtools/src/helpers/finalFrame.ts @@ -2,7 +2,7 @@ import logger from '@wdio/logger' import { errorMessage, finalFrameStamp, - replaceSnapshotAt + supersedeSnapshotAt } from '@wdio/devtools-core' import { captureActionSnapshot } from '../action-snapshot.js' import type { SessionLifecycleCtx } from '../session-lifecycle.js' @@ -39,7 +39,7 @@ export async function captureFinalFrame( capturer.isNativeAppSession ) if (snap) { - replaceSnapshotAt(ctx.actionSnapshots, snap) + supersedeSnapshotAt(ctx.actionSnapshots, snap) } } catch (err) { log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index 91e46176e..6700af013 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -310,7 +310,40 @@ export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ 'elementScreenshot', 'getHTML', 'getComputedRole', - 'getComputedLabel' + 'getComputedLabel', + 'getElementAriaLabel', + 'getElementAriaRole', + 'getShadowRoot', + 'w3cGetActiveElement', + 'w3cGetAlertText', + 'w3cGetCurrentWindowHandle', + 'w3cGetWindowHandles', + 'getNetworkConnection', + 'isVisible', + 'isPresent', + 'title', + 'source', + 'getCssProperty', + 'getElementSize', + 'getAriaRole', + 'getAccessibleName', + 'getFirstElementChild', + 'getLastElementChild', + 'getNextSibling', + 'getPreviousSibling', + 'hasDescendants', + 'element', + 'elements', + 'windowHandle', + 'windowHandles', + 'getWindowSize', + 'getWindowPosition', + 'getLogTypes', + 'isLogAvailable', + 'getDomAttribute', + 'getId', + 'getCapabilities', + 'getSession' ]) /** Whether a command is known not to change the page, so its capture can be diff --git a/packages/shared/tests/change-page.test.ts b/packages/shared/tests/change-page.test.ts index f5dae4727..e8f16d29e 100644 --- a/packages/shared/tests/change-page.test.ts +++ b/packages/shared/tests/change-page.test.ts @@ -49,7 +49,12 @@ describe('skipsCapture', () => { 'getCurrentUrl', 'assert.equal', 'expect.toHaveText', - 'getElementText' + 'getElementText', + 'w3cGetCurrentWindowHandle', + 'getElementAriaRole', + 'isVisible', + 'title', + 'getDomAttribute' ])('is true for %s', (command) => { expect(skipsCapture(command)).toBe(true) }) diff --git a/packages/shared/wire-schema.json b/packages/shared/wire-schema.json index 5dc4d5a82..9b03aa678 100644 --- a/packages/shared/wire-schema.json +++ b/packages/shared/wire-schema.json @@ -1510,41 +1510,60 @@ "readOnly": [ "$", "$$", + "element", "elementScreenshot", + "elements", "findChildElement", "findChildElements", "findElement", "findElementFromShadowRoot", "findElements", "findElementsFromShadowRoot", + "getAccessibleName", "getActiveElement", "getAlertText", "getAllCookies", + "getAriaRole", "getAttribute", "getAvailableLogTypes", "getCSSProperty", + "getCapabilities", "getComputedLabel", "getComputedRole", "getContextHandles", "getCookie", "getCookies", + "getCssProperty", "getCssValue", "getCurrentContextHandle", "getCurrentUrl", + "getDomAttribute", + "getElementAriaLabel", + "getElementAriaRole", "getElementAttribute", "getElementProperty", "getElementRect", "getElementShadowRoot", + "getElementSize", "getElementTagName", "getElementText", "getElementValueOfCssProperty", + "getFirstElementChild", "getHTML", + "getId", + "getLastElementChild", "getLocation", "getLog", + "getLogTypes", + "getNetworkConnection", + "getNextSibling", "getPageSource", + "getPreviousSibling", "getProperty", "getRect", "getScreenOrientation", + "getSession", + "getShadowRoot", "getSize", "getTagName", "getText", @@ -1554,7 +1573,10 @@ "getValue", "getWindowHandle", "getWindowHandles", + "getWindowPosition", "getWindowRect", + "getWindowSize", + "hasDescendants", "isClickable", "isDisplayed", "isElementEnabled", @@ -1562,10 +1584,19 @@ "isEnabled", "isExisting", "isFocused", + "isLogAvailable", + "isPresent", "isSelected", + "isVisible", "screenshot", + "source", "status", "takeScreenshot", + "title", + "w3cGetActiveElement", + "w3cGetAlertText", + "w3cGetCurrentWindowHandle", + "w3cGetWindowHandles", "wait", "waitForClickable", "waitForDisplayed", @@ -1576,7 +1607,9 @@ "waitForElementVisible", "waitForEnabled", "waitForExist", - "waitUntil" + "waitUntil", + "windowHandle", + "windowHandles" ] } } diff --git a/packages/trace/src/trace-snapshots.ts b/packages/trace/src/trace-snapshots.ts index 026939fb2..4ba8804db 100644 --- a/packages/trace/src/trace-snapshots.ts +++ b/packages/trace/src/trace-snapshots.ts @@ -70,17 +70,25 @@ export function upsertRichestSnapshot( mergeSnapshotInto(snapshots[idx]!, snap) } -/** Replace whatever holds the snapshot's timestamp, unlike upsertRichestSnapshot, for a capture that is strictly later state and so supersedes its fields. */ -export function replaceSnapshotAt( +function definedFields(snap: ActionSnapshot): Partial { + return Object.fromEntries( + Object.entries(snap).filter(([, value]) => value !== undefined) + ) +} + +/** Supersede the record holding the snapshot's timestamp field by field: a later capture's defined fields win, and one it lacks (a screenshot that timed out) keeps the earlier value. */ +export function supersedeSnapshotAt( snapshots: ActionSnapshot[], snap: ActionSnapshot ): void { + let merged = snap for (let i = snapshots.length - 1; i >= 0; i--) { if (snapshots[i]!.timestamp === snap.timestamp) { + merged = { ...snapshots[i]!, ...definedFields(merged) } snapshots.splice(i, 1) } } - snapshots.push(snap) + snapshots.push(merged) } export function buildSnapshotResources( diff --git a/packages/trace/tests/trace-snapshots.test.ts b/packages/trace/tests/trace-snapshots.test.ts index e706cc698..c52eeefe5 100644 --- a/packages/trace/tests/trace-snapshots.test.ts +++ b/packages/trace/tests/trace-snapshots.test.ts @@ -3,7 +3,7 @@ import os from 'node:os' import path from 'node:path' import { describe, it, expect } from 'vitest' import { - replaceSnapshotAt, + supersedeSnapshotAt, upsertRichestSnapshot, writeTraceZip, type TraceCapturer @@ -190,27 +190,46 @@ describe('snapshot binding contract (capture → export)', () => { }) }) -describe('replaceSnapshotAt', () => { - it('replaces the record at its timestamp whole, elements included', () => { +describe('supersedeSnapshotAt', () => { + it('takes the later capture’s defined fields and keeps the rest', () => { const list = [ snap(50, REAL), snap(100, REAL, { elements: [{ selector: '#old' }] }) ] - replaceSnapshotAt(list, snap(100, BLANK, { snapshotText: 'tree' })) + supersedeSnapshotAt(list, snap(100, BLANK, { snapshotText: 'tree' })) expect(list.filter((s) => s.timestamp === 100)).toEqual([ { timestamp: 100, command: 'click', screenshot: BLANK, + elements: [{ selector: '#old' }], snapshotText: 'tree' } ]) expect(list).toHaveLength(2) }) + it('keeps the earlier screenshot when a partial final capture has none', () => { + const list = [snap(100, REAL, { elements: [{ selector: '#old' }] })] + supersedeSnapshotAt(list, { + timestamp: 100, + command: 'click', + screenshot: undefined, + elements: [{ selector: '#new' }] + }) + expect(list).toEqual([ + { + timestamp: 100, + command: 'click', + screenshot: REAL, + elements: [{ selector: '#new' }] + } + ]) + }) + it('appends when no record holds the timestamp', () => { const list = [snap(50, REAL)] - replaceSnapshotAt(list, snap(100, BLANK)) + supersedeSnapshotAt(list, snap(100, BLANK)) expect(list.map((s) => s.timestamp)).toEqual([50, 100]) }) }) From 6c447fbd282f64b7ff1a4fd809ee697b07a80ca9 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 17:41:50 +0100 Subject: [PATCH 09/14] fix(trace): capture after actions().perform(), and never let an empty end capture supersede selenium-webdriver sends actions().perform() through driver.execute, which the patcher deliberately leaves unwrapped, so a navigating Actions click took no capture and the reads after it showed the old page. perform() is now wrapped to take a capture on completion without adding a row, since main never listed one. A test that ends its own session leaves the end-of-test capture with no url and no screenshot; its empty elements and placeholder text still superseded the last capture's. core capturedPageState() gates the supersede. Co-Authored-By: Claude Opus 5.5 --- packages/core/src/final-frame.ts | 7 ++ packages/core/tests/final-frame.test.ts | 32 +++++++- packages/nightwatch-devtools/src/session.ts | 3 +- .../tests/actionSnapshot.test.ts | 32 ++++++++ .../selenium-devtools/src/driverPatcher.ts | 2 + .../src/helpers/actions-patcher.ts | 70 +++++++++++++++++ .../src/helpers/commandPostActions.ts | 19 +++-- .../src/helpers/finalFrame.ts | 3 +- packages/selenium-devtools/src/index.ts | 10 ++- packages/selenium-devtools/src/types.ts | 2 + .../tests/action-snapshot.test.ts | 40 ++++++++++ .../tests/driverPatcher.test.ts | 75 ++++++++++++++++++- 12 files changed, 285 insertions(+), 10 deletions(-) create mode 100644 packages/selenium-devtools/src/helpers/actions-patcher.ts diff --git a/packages/core/src/final-frame.ts b/packages/core/src/final-frame.ts index ed037d457..a1073784c 100644 --- a/packages/core/src/final-frame.ts +++ b/packages/core/src/final-frame.ts @@ -1,3 +1,10 @@ +import type { ActionSnapshot } from '@wdio/devtools-shared' + +/** A capture that read neither a url nor a screenshot saw no page, so it must not replace one that did. */ +export function capturedPageState(snap: ActionSnapshot): boolean { + return Boolean(snap.url || snap.screenshot) +} + export interface FinalFrameCommand { command: string timestamp: number diff --git a/packages/core/tests/final-frame.test.ts b/packages/core/tests/final-frame.test.ts index fde36d525..5e14edbd0 100644 --- a/packages/core/tests/final-frame.test.ts +++ b/packages/core/tests/final-frame.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest' -import { finalFrameStamp } from '@wdio/devtools-core' +import { capturedPageState, finalFrameStamp } from '@wdio/devtools-core' const commands = [ { command: 'url', timestamp: 90 }, @@ -46,3 +46,33 @@ describe('finalFrameStamp', () => { }) }) }) + +describe('capturedPageState', () => { + it('holds for a capture that read a url or a screenshot', () => { + expect( + capturedPageState({ timestamp: 1, command: 'click', url: 'https://a/' }) + ).toBe(true) + expect( + capturedPageState({ timestamp: 1, command: 'click', screenshot: 'PNG' }) + ).toBe(true) + }) + + it('fails for a hollow capture from a session that had already ended', () => { + expect( + capturedPageState({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + }) + ).toBe(false) + expect( + capturedPageState({ + timestamp: 1, + command: 'click', + url: '', + screenshot: '' + }) + ).toBe(false) + }) +}) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 570a7b6d9..a1cd4e437 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -3,6 +3,7 @@ import { CAPTURE_PERFORMANCE_SCRIPT, SessionCapturerBase, applyPerformanceData, + capturedPageState, collectorDrainExpression, drainCollectorWithRecovery, errorMessage, @@ -652,7 +653,7 @@ export class SessionCapturer extends SessionCapturerBase { this.runner, this.isNativeAppSession ) - if (snap) { + if (snap && capturedPageState(snap)) { supersedeSnapshotAt(this.actionSnapshots, snap) } } catch (err) { diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index a1c7c794f..0e3ece450 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -248,6 +248,38 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) + it('keeps the earlier capture when the end-of-test frame saw no page', async () => { + type Snap = Awaited> + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'GOOD', + elements: [{ selector: '#good' }] + } as Snap) + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + const before = { ...cap.actionSnapshots[0]! } + expect(before.screenshot).toBe('GOOD') + + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + } as Snap) + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots).toEqual([before]) + cap.cleanup() + }) + it('takes no end-of-test frame when the test ends on a capturing command', async () => { const cap = makeCapturer() await cap.captureCommand('getText', ['#el'], undefined, undefined, { diff --git a/packages/selenium-devtools/src/driverPatcher.ts b/packages/selenium-devtools/src/driverPatcher.ts index 778baba3a..cf89c3a1a 100644 --- a/packages/selenium-devtools/src/driverPatcher.ts +++ b/packages/selenium-devtools/src/driverPatcher.ts @@ -12,6 +12,7 @@ import { TRACKED_ELEMENT_METHODS, NAVIGATION_COMMANDS } from './constants.js' +import { patchActions } from './helpers/actions-patcher.js' import { rememberElementLocator, selectorForElement @@ -524,6 +525,7 @@ export function patchSelenium(hooks: DriverPatcherHooks): boolean { ) log.info(`Wrapped ${wrappedDriver.length} WebDriver method(s)`) + patchActions(WebDriver, hooks) patchDriverQuit(WebDriver.prototype as Patchable, hooks) if (WebElement) { patchWebElement(WebElement, hooks) diff --git a/packages/selenium-devtools/src/helpers/actions-patcher.ts b/packages/selenium-devtools/src/helpers/actions-patcher.ts new file mode 100644 index 000000000..51d5283ff --- /dev/null +++ b/packages/selenium-devtools/src/helpers/actions-patcher.ts @@ -0,0 +1,70 @@ +import logger from '@wdio/logger' +import { errorMessage } from '@wdio/devtools-core' +import { PATCHED_SYMBOL } from '../constants.js' +import type { + DriverPatcherHooks, + Patchable, + SeleniumDriverLike +} from '../types.js' + +const log = logger('@wdio/selenium-devtools:actionsPatcher') + +/** The driver each action sequence was built on, read when its perform() settles. */ +const actionsOwners = new WeakMap() + +function patchActionsPerform( + actionsProto: Patchable, + hooks: DriverPatcherHooks +): void { + if (actionsProto[PATCHED_SYMBOL]) { + return + } + actionsProto[PATCHED_SYMBOL] = true + + const originalPerform = actionsProto.perform as ( + this: unknown + ) => Promise + actionsProto.perform = async function patchedPerform(this: unknown) { + await originalPerform.call(this) + const driver = actionsOwners.get(this as object) + if (!driver || !hooks.afterActions) { + return + } + try { + hooks.afterActions(driver) + } catch (err) { + log.warn(`afterActions hook threw: ${errorMessage(err)}`) + } + } +} + +/** perform() reaches the driver through execute(), so no wrapped method sees the page change it makes. */ +export function patchActions( + WebDriver: { prototype: object }, + hooks: DriverPatcherHooks +): void { + const driverProto = WebDriver.prototype as Patchable + const originalActions = driverProto.actions + if ( + typeof originalActions !== 'function' || + PATCHED_SYMBOL in originalActions + ) { + return + } + const original = originalActions as ( + this: unknown, + options?: unknown + ) => object + const patchedActions = function patchedActions( + this: SeleniumDriverLike, + options?: unknown + ) { + const actions = original.call(this, options) + actionsOwners.set(actions, this) + patchActionsPerform(Object.getPrototypeOf(actions) as Patchable, hooks) + return actions + } + driverProto.actions = Object.assign(patchedActions, { + [PATCHED_SYMBOL]: true + }) +} diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index 81659c551..044a5c721 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -348,10 +348,19 @@ export async function handleOnCommand( } maybeDrainAfterDomCommand(ctx, capturer, cmd) maybeDrainAfterLiveCommand(ctx, capturer, cmd) - queueActionSnapshot(ctx, capturer, cmd, entry.timestamp, error) + queueActionSnapshot(ctx, capturer, cmd.command, entry.timestamp, error) void capturer.nativeNetwork.forward({ throttle: true }) } +/** A driver-level action sequence's page change, stamped at completion for later reads to inherit. */ +export function handleActionsPerformed(ctx: OnCommandCtx): void { + const capturer = ctx.sessionCapturer + if (!capturer) { + return + } + queueActionSnapshot(ctx, capturer, 'performActions', Date.now(), undefined) +} + /** Fire-and-forget post-action snapshot, drained at finalize. Stamped with the * logged command's timestamp, not the capture's own: the capture resolves after * the command completes, and export claims snapshots by exact equality with the @@ -359,7 +368,7 @@ export async function handleOnCommand( function queueActionSnapshot( ctx: OnCommandCtx, capturer: SessionCapturer, - cmd: CapturedCommand, + command: string, timestamp: number, error: unknown ): void { @@ -368,8 +377,8 @@ function queueActionSnapshot( } // Reads, waits and lookups inherit the earlier capture; only the test's first mapped command adds an initial frame. const initialFrame = - capturer.awaitingInitialFrame && !!mapCommandToAction(cmd.command) - if (skipsCapture(cmd.command) && !initialFrame) { + capturer.awaitingInitialFrame && !!mapCommandToAction(command) + if (skipsCapture(command) && !initialFrame) { return } if (initialFrame) { @@ -380,7 +389,7 @@ function queueActionSnapshot( ctx.snapshotCaptures.push( captureActionSnapshot( ctx.driver, - cmd.command, + command, timestamp, capturer.isNativeAppSession ).then((snap) => { diff --git a/packages/selenium-devtools/src/helpers/finalFrame.ts b/packages/selenium-devtools/src/helpers/finalFrame.ts index ba2f701dc..db785dd64 100644 --- a/packages/selenium-devtools/src/helpers/finalFrame.ts +++ b/packages/selenium-devtools/src/helpers/finalFrame.ts @@ -1,5 +1,6 @@ import logger from '@wdio/logger' import { + capturedPageState, errorMessage, finalFrameStamp, supersedeSnapshotAt @@ -38,7 +39,7 @@ export async function captureFinalFrame( stamp.timestamp, capturer.isNativeAppSession ) - if (snap) { + if (snap && capturedPageState(snap)) { supersedeSnapshotAt(ctx.actionSnapshots, snap) } } catch (err) { diff --git a/packages/selenium-devtools/src/index.ts b/packages/selenium-devtools/src/index.ts index 07f389d94..1fc5f6009 100644 --- a/packages/selenium-devtools/src/index.ts +++ b/packages/selenium-devtools/src/index.ts @@ -6,7 +6,10 @@ import logger from '@wdio/logger' import { startDetachedBackend } from './helpers/detachedBackend.js' import { openDashboard } from './helpers/dashboardLauncher.js' import { getDriverOriginals } from './driverPatcher.js' -import { handleOnCommand } from './helpers/commandPostActions.js' +import { + handleActionsPerformed, + handleOnCommand +} from './helpers/commandPostActions.js' import { logConfigSummary } from './helpers/configSummary.js' import { registerProcessHooks } from './helpers/processHooks.js' import { patchSelenium } from './driverPatcher.js' @@ -551,6 +554,10 @@ class SeleniumDevToolsPlugin { return handled } + onActionsPerformed(driver: SeleniumDriverLike): void { + handleActionsPerformed({ ...this.#getInternals(), driver }) + } + /** Fallback path only: under the preload the destination instrumented itself. */ async instrumentNavigationTarget() { const capturer = this.#sessionCapturer @@ -690,6 +697,7 @@ const patched = patchSelenium({ onCommand: (cmd) => plugin.onCommand(cmd), afterNavigation: () => plugin.instrumentNavigationTarget(), beforeInput: () => plugin.focusForInput(), + afterActions: (driver) => plugin.onActionsPerformed(driver), // Per-scenario cleanup ONLY here (finalizes that driver's screencast, // clears per-driver state). The session-wide teardown — set-finalized, // session summary, capturer cleanup, interactive shutdown — lives in diff --git a/packages/selenium-devtools/src/types.ts b/packages/selenium-devtools/src/types.ts index 0dac4297a..2a9057d3a 100644 --- a/packages/selenium-devtools/src/types.ts +++ b/packages/selenium-devtools/src/types.ts @@ -159,6 +159,8 @@ export interface DriverPatcherHooks { afterNavigation?: (command: string) => Promise // Awaited before an element click, sendKeys, clear or submit is sent. beforeInput?: () => Promise + // Fired once an action sequence's perform() succeeds: it moves the page with no row of its own. + afterActions?: (driver: SeleniumDriverLike) => void } // Unwrapped WebDriver methods used by the plugin's own internal calls diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index d346f29aa..a4c32d353 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -12,6 +12,7 @@ import { } from '../src/helpers/commandPostActions.js' import { captureActionSnapshot } from '../src/action-snapshot.js' import { captureFinalFrame } from '../src/helpers/finalFrame.js' +import { handleActionsPerformed } from '../src/helpers/commandPostActions.js' import type { SessionLifecycleCtx } from '../src/session-lifecycle.js' import { getDriverOriginals } from '../src/driverPatcher.js' import { @@ -305,6 +306,30 @@ describe('selenium action-snapshot capture gate', () => { expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() }) + + it('keeps the earlier capture when the end state saw no page', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'GOOD', + elements: [{ selector: '#good' }] + } as unknown as ActionSnapshot) + const bag = await run(['click', 'getText']) + const before = { ...bag.actionSnapshots[0]! } + expect(before.screenshot).toBe('GOOD') + + vi.mocked(coreCapture).mockClear() + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + } as unknown as ActionSnapshot) + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(bag.actionSnapshots).toEqual([before]) + }) }) }) @@ -387,3 +412,18 @@ describe('the per-action snapshot on a native session', () => { } }) }) + +describe('driver-level action sequences', () => { + it('captures the page a perform left without adding a timeline row', async () => { + const bag = makeCtx(fakeDriver()) + vi.mocked(coreCapture).mockClear() + + handleActionsPerformed(bag.ctx) + await Promise.all(bag.snapshotCaptures) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(bag.capturer.commandsLog).toHaveLength(0) + expect(bag.actionSnapshots).toHaveLength(1) + expect(bag.actionSnapshots[0]!.command).toBe('performActions') + }) +}) diff --git a/packages/selenium-devtools/tests/driverPatcher.test.ts b/packages/selenium-devtools/tests/driverPatcher.test.ts index 3ad7b38a5..dec8c276e 100644 --- a/packages/selenium-devtools/tests/driverPatcher.test.ts +++ b/packages/selenium-devtools/tests/driverPatcher.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect } from 'vitest' +import { describe, it, expect, vi } from 'vitest' import { isWebElementLike, safeSerialize, @@ -6,6 +6,8 @@ import { getDriverOriginals, getElementOriginals } from '../src/driverPatcher.js' +import { patchActions } from '../src/helpers/actions-patcher.js' +import type { DriverPatcherHooks } from '../src/types.js' describe('isWebElementLike', () => { it('detects an object with getId + click as a WebElement', () => { @@ -122,3 +124,74 @@ describe('getDriverOriginals / getElementOriginals', () => { expect(e).not.toBeNull() }) }) + +describe('patchActions', () => { + // A fresh class per test: patching marks the prototype, so a shared one would + // keep the first test's hooks. + function fakeDriverClass() { + class FakeActions { + constructor( + private readonly executor_: unknown, + private readonly fails = false + ) {} + async perform() { + if (this.fails) { + throw new Error('stale element') + } + return this.executor_ + } + } + return class FakeDriver { + actions(options?: { fails?: boolean }) { + return new FakeActions(this, options?.fails) + } + } + } + + function hooksWith( + afterActions = vi.fn(), + onCommand = vi.fn() + ): DriverPatcherHooks { + return { + onDriverCreated: () => {}, + onCommand, + afterActions + } as unknown as DriverPatcherHooks + } + + it('reports the owning driver once a sequence performs, with no command row', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + const onCommand = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions, onCommand)) + const driver = new FakeDriver() + + await driver.actions().perform() + + expect(afterActions).toHaveBeenCalledTimes(1) + expect(afterActions.mock.calls[0]![0]).toBe(driver) + expect(onCommand).not.toHaveBeenCalled() + }) + + it('reports nothing when a sequence fails to perform', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions)) + + await expect( + new FakeDriver().actions({ fails: true }).perform() + ).rejects.toThrow('stale element') + expect(afterActions).not.toHaveBeenCalled() + }) + + it('wraps perform once however often it is patched', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions)) + patchActions(FakeDriver, hooksWith(afterActions)) + + await new FakeDriver().actions().perform() + + expect(afterActions).toHaveBeenCalledTimes(1) + }) +}) From d8df1b675fc04d91a2dd30a00165527b35f1898e Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 18:43:28 +0100 Subject: [PATCH 10/14] fix(nightwatch-devtools): re-capture a command whose navigation commits after it returns Nightwatch's click resolves before its navigation commits, so its per-action capture records the page it is leaving. main hid that by capturing again on every read; with reads skipped, the click and the reads after it showed the old page (iOS Safari, 2 of 2 runs). When anchorAfterNavigation sees the document replaced, the command that triggered it is captured once more and the result supersedes its record. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 2 +- .../src/helpers/browserProxy.ts | 9 +- .../src/navigation-recapture.ts | 55 +++++++ packages/nightwatch-devtools/src/session.ts | 81 +++++++---- .../tests/navigationRecapture.test.ts | 135 ++++++++++++++++++ 5 files changed, 253 insertions(+), 29 deletions(-) create mode 100644 packages/nightwatch-devtools/src/navigation-recapture.ts create mode 100644 packages/nightwatch-devtools/tests/navigationRecapture.test.ts diff --git a/CLAUDE.md b/CLAUDE.md index 0334a01ea..0946b182c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -89,7 +89,7 @@ The test for a new helper: *would the backend ever need this to build a zip?* If ### One DOM snapshot per action, taken before it -In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. +In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose navigation commits after it returns. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and replacing its record whole; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. diff --git a/packages/nightwatch-devtools/src/helpers/browserProxy.ts b/packages/nightwatch-devtools/src/helpers/browserProxy.ts index 6736f3f7d..320d052ad 100644 --- a/packages/nightwatch-devtools/src/helpers/browserProxy.ts +++ b/packages/nightwatch-devtools/src/helpers/browserProxy.ts @@ -280,7 +280,7 @@ export class BrowserProxy { log.info(`[command] ${methodName}`) this.attachScreenshot(browser, lastCommand, methodName) } - this.anchorDestination(browser, methodName) + this.anchorDestination(browser, methodName, lastCommand?.timestamp) // A DOM-mutating command already drains it through `captureTrace`. if (!this.isDomMutating(methodName)) { void this.sessionCapturer.nativeNetwork.forward({ throttle: true }) @@ -409,7 +409,8 @@ export class BrowserProxy { */ private anchorDestination( browser: NightwatchBrowser, - methodName: string + methodName: string, + timestamp: number | undefined ): void { if (!this.isDomMutating(methodName)) { return @@ -417,7 +418,9 @@ export class BrowserProxy { // Tracked so finalize awaits it — an untracked anchor drain can still be in // flight when the trace is written, which loses the document entirely. this.sessionCapturer.snapshotCaptures.push( - this.sessionCapturer.anchorAfterNavigation(browser).catch(() => {}) + this.sessionCapturer + .anchorAfterNavigation(browser, timestamp) + .catch(() => {}) ) } diff --git a/packages/nightwatch-devtools/src/navigation-recapture.ts b/packages/nightwatch-devtools/src/navigation-recapture.ts new file mode 100644 index 000000000..29566bcf0 --- /dev/null +++ b/packages/nightwatch-devtools/src/navigation-recapture.ts @@ -0,0 +1,55 @@ +import logger from '@wdio/logger' +import { + capturedPageState, + errorMessage, + supersedeSnapshotAt +} from '@wdio/devtools-core' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import { captureActionSnapshot } from './action-snapshot.js' +import type { NightwatchBrowser, TestRunnerId } from './types.js' + +const log = logger('@wdio/nightwatch-devtools:navigationRecapture') + +/** A per-action capture a command queued, kept so a navigation seen after the command returns can supersede it once it has settled. */ +export interface QueuedPageCapture { + command: string + timestamp: number + settled: Promise +} + +export interface NavigationRecapture { + browser: NightwatchBrowser + runner: TestRunnerId + native: boolean + snapshots: ActionSnapshot[] + queued: QueuedPageCapture +} + +/** + * A click resolves before its navigation commits, so the capture it queued can + * read the page it left. The replacement is taken after that capture settles, + * because a late upsert would otherwise merge the stale screenshot back over it. + */ +export async function recaptureAfterNavigation({ + browser, + runner, + native, + snapshots, + queued +}: NavigationRecapture): Promise { + await queued.settled + try { + const snap = await captureActionSnapshot( + browser, + queued.command, + queued.timestamp, + runner, + native + ) + if (snap && capturedPageState(snap)) { + supersedeSnapshotAt(snapshots, snap) + } + } catch (err) { + log.debug(`Post-navigation capture failed: ${errorMessage(err)}`) + } +} diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index a1cd4e437..9ccd88be4 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -24,6 +24,10 @@ import { skipsCapture } from '@wdio/devtools-shared' import { captureActionSnapshot } from './action-snapshot.js' +import { + recaptureAfterNavigation, + type QueuedPageCapture +} from './navigation-recapture.js' import { nightwatchRunnerId } from './helpers/utils.js' import { NAVIGATION_COMMANDS } from './constants.js' import { @@ -53,6 +57,8 @@ const log = logger('@wdio/nightwatch-devtools:SessionCapturer') export class SessionCapturer extends SessionCapturerBase { #browser: NightwatchBrowser | undefined + /** The most recent page-changing capture; read synchronously when a navigation is detected for its command. */ + #lastPageCapture: QueuedPageCapture | undefined // True once BiDi inspectors are attached — the per-command perf-log network // capture path skips when set, so we don't double-emit network requests. @@ -211,29 +217,31 @@ export class SessionCapturer extends SessionCapturerBase { } this.lastCapturedTimestamp = timestamp const generation = this.testCaptureGeneration - this.snapshotCaptures.push( - captureActionSnapshot( - this.#browser, - command, - timestamp, - this.runner, - this.isNativeAppSession - ).then((snap) => { - if (snap) { - upsertRichestSnapshot(this.actionSnapshots, snap) - } - // A capture resolving after resetTestCapture() belongs to the previous test. - if ( - initialFrame && - !snap?.screenshot && - !this.initialFrameRetried && - generation === this.testCaptureGeneration - ) { - this.initialFrameRetried = true - this.awaitingInitialFrame = true - } - }) - ) + const settled = captureActionSnapshot( + this.#browser, + command, + timestamp, + this.runner, + this.isNativeAppSession + ).then((snap) => { + if (snap) { + upsertRichestSnapshot(this.actionSnapshots, snap) + } + // A capture resolving after resetTestCapture() belongs to the previous test. + if ( + initialFrame && + !snap?.screenshot && + !this.initialFrameRetried && + generation === this.testCaptureGeneration + ) { + this.initialFrameRetried = true + this.awaitingInitialFrame = true + } + }) + this.snapshotCaptures.push(settled) + if (pageChanging) { + this.#lastPageCapture = { command, timestamp, settled } + } } async #capturePerformanceData( @@ -446,12 +454,20 @@ export class SessionCapturer extends SessionCapturerBase { * of a run anchors unconditionally rather than guessing — `captureCurrentDom` * is idempotent per document, so an already-anchored page costs one drain. */ - async anchorAfterNavigation(browser: NightwatchBrowser): Promise { + async anchorAfterNavigation( + browser: NightwatchBrowser, + timestamp?: number + ): Promise { // Polls the page for its own document identity, so there is nothing to // poll and nothing to anchor without one. if (this.isNativeAppSession) { return } + // Read before the first await: the capture is whichever one the triggering command queued. + const queued = + timestamp !== undefined && this.#lastPageCapture?.timestamp === timestamp + ? this.#lastPageCapture + : undefined const before = this.lastDocumentOrigin const replaced = await pollUntilReady( async () => { @@ -467,6 +483,18 @@ export class SessionCapturer extends SessionCapturerBase { if (!replaced) { return } + // Only a change from an origin this capturer already saw is a navigation; the + // first observation has no baseline to have changed from. + const recapture = + queued && before !== undefined + ? recaptureAfterNavigation({ + browser, + runner: this.runner, + native: this.isNativeAppSession, + snapshots: this.actionSnapshots, + queued + }) + : Promise.resolve() // captureTrace's own recovery injects into the fresh document and re-drains, // and it gates that on the url being worth instrumenting — so a torn-down // session falls out here rather than logging an injection failure. @@ -484,7 +512,10 @@ export class SessionCapturer extends SessionCapturerBase { // the one that caused it. The guard exists for the case where that is a guess. // A wait sent running is newest from its start, but it has not ended, so it // did not cause the navigation. - await this.captureTrace(browser, true, this.#newestEndedTimestamp()) + await Promise.all([ + this.captureTrace(browser, true, this.#newestEndedTimestamp()), + recapture + ]) } /** diff --git a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts new file mode 100644 index 000000000..278cc8364 --- /dev/null +++ b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts @@ -0,0 +1,135 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' +import { captureActionSnapshot } from '../src/action-snapshot.js' +import { SessionCapturer } from '../src/session.js' +import { webdriverExecute } from '../src/helpers/webdriverHttp.js' +import type * as WebdriverHttp from '../src/helpers/webdriverHttp.js' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import type { NightwatchBrowser } from '../src/types.js' + +vi.mock('../src/action-snapshot.js', () => ({ + captureActionSnapshot: vi.fn() +})) + +vi.mock('../src/helpers/webdriverHttp.js', async (importOriginal) => ({ + ...(await importOriginal()), + webdriverExecute: vi.fn() +})) + +const browser = { + sessionId: 'sess-recapture', + capabilities: {}, + options: { webdriver: { host: 'localhost', port: 4444 } } +} as unknown as NightwatchBrowser + +/** `performance.timeOrigin` reads, one per poll; the last origin repeats. */ +function originsRead(origins: number[]) { + let read = 0 + vi.mocked(webdriverExecute).mockImplementation(async () => { + return origins[Math.min(read++, origins.length - 1)] as never + }) +} + +function shot(fields: Partial): ActionSnapshot { + return { command: 'click', timestamp: 100, ...fields } as ActionSnapshot +} + +/** A click on /login whose capture read the page before the submit committed. */ +async function clickThenAnchor(cap: SessionCapturer) { + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 100 + }) + await cap.anchorAfterNavigation(browser, 100) + await Promise.all(cap.snapshotCaptures) +} + +describe('Nightwatch post-navigation re-capture', () => { + let cap: SessionCapturer + + beforeEach(() => { + vi.mocked(captureActionSnapshot).mockReset() + vi.spyOn(SessionCapturer.prototype, 'captureTrace').mockResolvedValue() + cap = new SessionCapturer({}, browser) + cap.traceMode = 'trace' + }) + + afterEach(() => { + cap.cleanup() + vi.restoreAllMocks() + }) + + it('supersedes the click capture with one taken once navigation is seen', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/secure', screenshot: 'NEW' }) + ) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(vi.mocked(captureActionSnapshot).mock.calls[1]?.[2]).toBe(100) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + timestamp: 100, + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + }) + + it('takes no extra capture when the document is not replaced', async () => { + originsRead([1000]) + vi.mocked(captureActionSnapshot).mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD' + }) + }) + + it('leaves the record when the re-capture holds no page state', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ + url: 'http://app.test/login', + screenshot: 'OLD', + snapshotText: 'old' + }) + ) + .mockResolvedValueOnce(shot({ snapshotText: 'blank' })) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD', + snapshotText: 'old' + }) + }) + + it('takes no re-capture for a command that queued none', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot).mockResolvedValue(shot({})) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('getText', ['#flash'], 'x', undefined, { + timestamp: 100 + }) + const queued = vi.mocked(captureActionSnapshot).mock.calls.length + await cap.anchorAfterNavigation(browser, 100) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(queued) + }) +}) From 3a48d61be46483c9fa95f45e282ff9d164133ab0 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 18:57:52 +0100 Subject: [PATCH 11/14] fix(nightwatch-devtools): credit a late navigation to the command that caused it The re-capture was bound to the command whose poll saw the document replaced. A fill's poll outlasts it, so when the next command (the submit click) navigated during that poll, the fill took the destination page and its row showed /secure before the login had happened. The target is now the latest captured command at the moment the replacement is seen, and overlapping polls re-capture it once. Co-Authored-By: Claude Opus 5.5 --- packages/nightwatch-devtools/src/session.ts | 40 ++++++---- .../tests/navigationRecapture.test.ts | 77 +++++++++++++++++++ 2 files changed, 101 insertions(+), 16 deletions(-) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 9ccd88be4..a928920ee 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -59,6 +59,8 @@ export class SessionCapturer extends SessionCapturerBase { #browser: NightwatchBrowser | undefined /** The most recent page-changing capture; read synchronously when a navigation is detected for its command. */ #lastPageCapture: QueuedPageCapture | undefined + /** Capture timestamps already re-captured or re-capturing; overlapping polls see the same replacement. */ + #lastRecapturedAt?: number // True once BiDi inspectors are attached — the per-command perf-log network // capture path skips when set, so we don't double-emit network requests. @@ -463,11 +465,6 @@ export class SessionCapturer extends SessionCapturerBase { if (this.isNativeAppSession) { return } - // Read before the first await: the capture is whichever one the triggering command queued. - const queued = - timestamp !== undefined && this.#lastPageCapture?.timestamp === timestamp - ? this.#lastPageCapture - : undefined const before = this.lastDocumentOrigin const replaced = await pollUntilReady( async () => { @@ -484,17 +481,28 @@ export class SessionCapturer extends SessionCapturerBase { return } // Only a change from an origin this capturer already saw is a navigation; the - // first observation has no baseline to have changed from. - const recapture = - queued && before !== undefined - ? recaptureAfterNavigation({ - browser, - runner: this.runner, - native: this.isNativeAppSession, - snapshots: this.actionSnapshots, - queued - }) - : Promise.resolve() + // first observation has no baseline to have changed from. The newest page + // capture is read here, after the poll: the click that caused the navigation + // may have queued its capture while this poll was running, and the fill that + // started the poll must never be re-stamped with the destination. + const queued = this.#lastPageCapture + let recapture: Promise = Promise.resolve() + if ( + timestamp !== undefined && + queued && + before !== undefined && + queued.timestamp >= timestamp && + queued.timestamp !== this.#lastRecapturedAt + ) { + this.#lastRecapturedAt = queued.timestamp + recapture = recaptureAfterNavigation({ + browser, + runner: this.runner, + native: this.isNativeAppSession, + snapshots: this.actionSnapshots, + queued + }) + } // captureTrace's own recovery injects into the fresh document and re-drains, // and it gates that on the url being worth instrumenting — so a torn-down // session falls out here rather than logging an injection failure. diff --git a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts index 278cc8364..7a37ae578 100644 --- a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts +++ b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts @@ -132,4 +132,81 @@ describe('Nightwatch post-navigation re-capture', () => { expect(captureActionSnapshot).toHaveBeenCalledTimes(queued) }) + + it('attributes a navigation seen during a fill poll to the click that caused it', async () => { + let pageUrl = 'http://app.test/login' + let reads = 0 + vi.mocked(captureActionSnapshot).mockImplementation( + async (_browser, command, timestamp) => + shot({ command, timestamp, url: pageUrl, screenshot: pageUrl }) + ) + vi.mocked(webdriverExecute).mockImplementation(async () => { + reads++ + if (reads === 2) { + void cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 200 + }) + return 1000 as never + } + if (reads === 3) { + pageUrl = 'http://app.test/secure' + return 2000 as never + } + return 1000 as never + }) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('setValue', ['#username'], 'tomsmith', undefined, { + timestamp: 100 + }) + await cap.anchorAfterNavigation(browser, 100) + await Promise.all(cap.snapshotCaptures) + + expect( + vi + .mocked(captureActionSnapshot) + .mock.calls.map((call) => [call[1], call[2]]) + ).toEqual([ + ['setValue', 100], + ['click', 200], + ['click', 200] + ]) + expect(cap.actionSnapshots.find((s) => s.timestamp === 100)).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'http://app.test/login' + }) + expect(cap.actionSnapshots.find((s) => s.timestamp === 200)).toMatchObject({ + url: 'http://app.test/secure', + screenshot: 'http://app.test/secure' + }) + }) + + it('re-captures once when overlapping polls detect the same replacement', async () => { + let pageUrl = 'http://app.test/login' + vi.mocked(captureActionSnapshot).mockImplementation( + async (_browser, command, timestamp) => + shot({ command, timestamp, url: pageUrl, screenshot: pageUrl }) + ) + vi.mocked(webdriverExecute).mockImplementation( + async () => (pageUrl === 'http://app.test/login' ? 1000 : 2000) as never + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 200 + }) + pageUrl = 'http://app.test/secure' + await Promise.all([ + cap.anchorAfterNavigation(browser, 150), + cap.anchorAfterNavigation(browser, 180) + ]) + await Promise.all(cap.snapshotCaptures) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + timestamp: 200, + url: 'http://app.test/secure' + }) + }) }) From ca44a94ef95f3e707391c6ff313df0efcee3d002 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 19:22:06 +0100 Subject: [PATCH 12/14] fix(nightwatch-devtools): re-capture only a command that could have read the page it left The re-capture fired for any command whose poll saw the document replaced, including url(), which resolves after load and had already captured its destination. Its late re-read then landed after the next command navigated again and gave the url() row that later page. A command is now re-captured only when its capture was queued before the destination document's performance.timeOrigin, and the re-capture is dropped if a newer page capture was queued while it ran. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 2 +- .../src/navigation-recapture.ts | 7 +- packages/nightwatch-devtools/src/session.ts | 7 +- .../tests/navigationRecapture.test.ts | 67 +++++++++++++++++++ 4 files changed, 79 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 0946b182c..b9abadcb8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -89,7 +89,7 @@ The test for a new helper: *would the backend ever need this to build a zip?* If ### One DOM snapshot per action, taken before it -In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose navigation commits after it returns. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. +In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose capture predates the destination document's `performance.timeOrigin` (a navigation that committed after it returned), and discards that re-capture if a newer page capture was queued meanwhile. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and replacing its record whole; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. diff --git a/packages/nightwatch-devtools/src/navigation-recapture.ts b/packages/nightwatch-devtools/src/navigation-recapture.ts index 29566bcf0..226f95c27 100644 --- a/packages/nightwatch-devtools/src/navigation-recapture.ts +++ b/packages/nightwatch-devtools/src/navigation-recapture.ts @@ -23,6 +23,8 @@ export interface NavigationRecapture { native: boolean snapshots: ActionSnapshot[] queued: QueuedPageCapture + /** False once a newer page capture has been queued; its own capture then reads the newer state. */ + isLatest: () => boolean } /** @@ -35,7 +37,8 @@ export async function recaptureAfterNavigation({ runner, native, snapshots, - queued + queued, + isLatest }: NavigationRecapture): Promise { await queued.settled try { @@ -46,7 +49,7 @@ export async function recaptureAfterNavigation({ runner, native ) - if (snap && capturedPageState(snap)) { + if (snap && capturedPageState(snap) && isLatest()) { supersedeSnapshotAt(snapshots, snap) } } catch (err) { diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index a928920ee..d4dc563b4 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -466,6 +466,7 @@ export class SessionCapturer extends SessionCapturerBase { return } const before = this.lastDocumentOrigin + let born: number | undefined const replaced = await pollUntilReady( async () => { const now = await this.documentOrigin(browser) @@ -473,6 +474,7 @@ export class SessionCapturer extends SessionCapturerBase { return false } this.lastDocumentOrigin = now + born = now return before === undefined || now !== before }, { attempts: 5, intervalMs: 150 } @@ -491,7 +493,9 @@ export class SessionCapturer extends SessionCapturerBase { timestamp !== undefined && queued && before !== undefined && + born !== undefined && queued.timestamp >= timestamp && + queued.timestamp < born && queued.timestamp !== this.#lastRecapturedAt ) { this.#lastRecapturedAt = queued.timestamp @@ -500,7 +504,8 @@ export class SessionCapturer extends SessionCapturerBase { runner: this.runner, native: this.isNativeAppSession, snapshots: this.actionSnapshots, - queued + queued, + isLatest: () => this.#lastPageCapture === queued }) } // captureTrace's own recovery injects into the fresh document and re-drains, diff --git a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts index 7a37ae578..ca9c18ef6 100644 --- a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts +++ b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts @@ -80,6 +80,31 @@ describe('Nightwatch post-navigation re-capture', () => { }) }) + it('takes no re-capture when the queued capture already read the destination', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot).mockResolvedValueOnce( + shot({ + command: 'click', + timestamp: 2500, + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#next'], null, undefined, { + timestamp: 2500 + }) + await cap.anchorAfterNavigation(browser, 2500) + await Promise.all(cap.snapshotCaptures) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + }) + it('takes no extra capture when the document is not replaced', async () => { originsRead([1000]) vi.mocked(captureActionSnapshot).mockResolvedValueOnce( @@ -181,6 +206,48 @@ describe('Nightwatch post-navigation re-capture', () => { }) }) + it('discards a re-capture superseded by a page capture queued while it was in flight', async () => { + originsRead([1000, 2000]) + let finishRecapture!: (snap: ActionSnapshot) => void + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + .mockImplementationOnce( + () => new Promise((resolve) => (finishRecapture = resolve)) + ) + .mockResolvedValueOnce( + shot({ + command: 'click', + timestamp: 150, + url: 'http://app.test/secure', + screenshot: 'NEXT' + }) + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 100 + }) + const anchored = cap.anchorAfterNavigation(browser, 100) + await vi.waitFor(() => + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + ) + await cap.captureCommand('click', ['#next'], null, undefined, { + timestamp: 150 + }) + finishRecapture( + shot({ url: 'http://app.test/secure', screenshot: 'STALE' }) + ) + await anchored + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.find((s) => s.timestamp === 100)).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD' + }) + }) + it('re-captures once when overlapping polls detect the same replacement', async () => { let pageUrl = 'http://app.test/login' vi.mocked(captureActionSnapshot).mockImplementation( From 1eed2617a231afad04f61c57adaf923a1374ca07 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Fri, 30 Oct 2026 19:48:59 +0100 Subject: [PATCH 13/14] fix(trace): only a capture with a screenshot supersedes, and a failed perform() still captures A url-only end or post-navigation capture (screenshot probe timed out) was merged over the earlier record, leaving the new page's url on the old page's image. capturedPageState now requires a screenshot, and Python's end capture follows the same rule and keeps a screenshot-only capture as the JS path does. An actions() sequence that moved the page and then rejected now still takes its capture. Nightwatch awaits its post-navigation re-capture even when the drain beside it fails, and logs that failure. Docs and changesets now describe the capture as keyed on the last command that took one. Co-Authored-By: Claude Opus 5.5 --- .changeset/capture-only-page-changes.md | 2 +- CLAUDE.md | 2 +- packages/core/src/final-frame.ts | 4 ++-- packages/core/tests/final-frame.test.ts | 11 ++++++---- .../src/navigation-recapture.ts | 2 +- packages/nightwatch-devtools/src/session.ts | 6 +++-- .../changes/page-changing-capture.md | 2 +- .../src/selenium_devtools/instrumentation.py | 11 ++++++++-- .../tests/test_a11y_elements.py | 22 ++++++++++++++++--- .../src/helpers/actions-patcher.ts | 18 ++++++++------- .../tests/driverPatcher.test.ts | 12 +++++----- 11 files changed, 62 insertions(+), 30 deletions(-) diff --git a/.changeset/capture-only-page-changes.md b/.changeset/capture-only-page-changes.md index e33551f8b..dd5da0cf4 100644 --- a/.changeset/capture-only-page-changes.md +++ b/.changeset/capture-only-page-changes.md @@ -4,6 +4,6 @@ '@wdio/nightwatch-devtools': patch --- -Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for reads, waits and lookups such as `getText`, `isDisplayed`, `waitForDisplayed` or `findElement`. Those rows still appear in the trace and show the page as of the last command that changed it. Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). +Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for reads, waits and lookups such as `getText`, `isDisplayed`, `waitForDisplayed` or `findElement`. Those rows still appear in the trace and show the page as of the last command that took a capture (any command except reads, waits, lookups and asserts). Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). Selenium and Nightwatch now capture every other command, including ones that can navigate but are not in the action map (`executeScript`, `actions`, Nightwatch `submitForm`), and take one more capture at a test's end when reads follow its last capture, so a click that returns before its navigation commits no longer leaves the reads showing the page it left. The WDIO service keeps its page-changing gate. diff --git a/CLAUDE.md b/CLAUDE.md index b9abadcb8..23ee832a1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,7 +91,7 @@ The test for a new helper: *would the backend ever need this to build a zip?* If In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose capture predates the destination document's `performance.timeOrigin` (a navigation that committed after it returned), and discards that re-capture if a newer page capture was queued meanwhile. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. -Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and replacing its record whole; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. +Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and superseding that record field by field, only when it has a screenshot; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. ### Adapters are thin and isolated diff --git a/packages/core/src/final-frame.ts b/packages/core/src/final-frame.ts index a1073784c..d3fdb48a2 100644 --- a/packages/core/src/final-frame.ts +++ b/packages/core/src/final-frame.ts @@ -1,8 +1,8 @@ import type { ActionSnapshot } from '@wdio/devtools-shared' -/** A capture that read neither a url nor a screenshot saw no page, so it must not replace one that did. */ +/** Frames are screenshot-backed: a url-only capture merged over an earlier record would pair the new url with the old image and DOM. */ export function capturedPageState(snap: ActionSnapshot): boolean { - return Boolean(snap.url || snap.screenshot) + return Boolean(snap.screenshot) } export interface FinalFrameCommand { diff --git a/packages/core/tests/final-frame.test.ts b/packages/core/tests/final-frame.test.ts index 5e14edbd0..ef71b3ce0 100644 --- a/packages/core/tests/final-frame.test.ts +++ b/packages/core/tests/final-frame.test.ts @@ -48,15 +48,18 @@ describe('finalFrameStamp', () => { }) describe('capturedPageState', () => { - it('holds for a capture that read a url or a screenshot', () => { - expect( - capturedPageState({ timestamp: 1, command: 'click', url: 'https://a/' }) - ).toBe(true) + it('holds for a capture that read a screenshot', () => { expect( capturedPageState({ timestamp: 1, command: 'click', screenshot: 'PNG' }) ).toBe(true) }) + it('fails for a url-only capture, which has no image to pair the url with', () => { + expect( + capturedPageState({ timestamp: 1, command: 'click', url: 'https://a/' }) + ).toBe(false) + }) + it('fails for a hollow capture from a session that had already ended', () => { expect( capturedPageState({ diff --git a/packages/nightwatch-devtools/src/navigation-recapture.ts b/packages/nightwatch-devtools/src/navigation-recapture.ts index 226f95c27..89d130fd5 100644 --- a/packages/nightwatch-devtools/src/navigation-recapture.ts +++ b/packages/nightwatch-devtools/src/navigation-recapture.ts @@ -40,8 +40,8 @@ export async function recaptureAfterNavigation({ queued, isLatest }: NavigationRecapture): Promise { - await queued.settled try { + await queued.settled const snap = await captureActionSnapshot( browser, queued.command, diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index d4dc563b4..5e8ac94ad 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -525,10 +525,13 @@ export class SessionCapturer extends SessionCapturerBase { // the one that caused it. The guard exists for the case where that is a guess. // A wait sent running is newest from its start, but it has not ended, so it // did not cause the navigation. - await Promise.all([ + const [drain] = await Promise.allSettled([ this.captureTrace(browser, true, this.#newestEndedTimestamp()), recapture ]) + if (drain.status === 'rejected') { + log.debug(`Post-navigation drain failed: ${errorMessage(drain.reason)}`) + } } /** @@ -677,7 +680,6 @@ export class SessionCapturer extends SessionCapturerBase { await this.captureNetworkFromPerformanceLogs(browser) } - /** At a test or scenario end, before its slice is written: the proxy drain there is never throttled. */ /** Rows after the last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ async captureFinalFrame(): Promise { await Promise.all(this.snapshotCaptures) diff --git a/packages/selenium-devtools-py/changes/page-changing-capture.md b/packages/selenium-devtools-py/changes/page-changing-capture.md index b00a731a6..53e41f67b 100644 --- a/packages/selenium-devtools-py/changes/page-changing-capture.md +++ b/packages/selenium-devtools-py/changes/page-changing-capture.md @@ -2,4 +2,4 @@ patch --- -Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. A test whose last page change is followed by reads also gets one more read of the page it ended on. Reads and waits keep their own rows and show the tree from the last capture before them. +Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. When reads follow a test's last capture, the test's end takes one more capture, stamped at that last capture and superseding it; the capture adds no command row. Reads and waits keep their own rows and show the tree from the last capture before them. diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index 3d1dd4189..483ef32ce 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -558,9 +558,16 @@ def _capture_final_frame(driver: Any) -> bool: if stamp is None: return False command, timestamp = stamp - snapshot = _capture_action_snapshot(driver, command, timestamp, _take_screenshot(driver)) - if snapshot is None: + # Frames are screenshot-backed: panes merged over an earlier image would pair two documents. + # Frames are screenshot-backed: panes merged over an earlier image would pair two documents. + shot = _take_screenshot(driver) + if not shot: return False + snapshot = _capture_action_snapshot(driver, command, timestamp, shot) or { + "timestamp": timestamp, + "command": command, + "screenshot": shot, + } superseded: dict = {} for kept in _state["action_snapshots"]: if kept["timestamp"] == timestamp and kept is not snapshot: diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index 542fe6c87..f755e149c 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -352,6 +352,9 @@ def __init__(self): self.tree = [{"selector": "#before"}] self.scripts_run = 0 + def get_screenshot_as_base64(self): + return "END" + def execute_script(self, script, *args): self.scripts_run += 1 return self.tree @@ -425,18 +428,31 @@ def test_the_final_capture_is_taken_once_per_test(self): instrumentation.capture_final_frame(driver) self.assertEqual(driver.scripts_run, reads, "the quit repeated the call report's capture") - def test_a_partial_final_capture_keeps_the_earlier_screenshot(self): + def test_a_final_capture_without_a_screenshot_leaves_the_record_unchanged(self): driver = self.Driver() with self._ended(): instrumentation._state["test_rows"].append(("clickElement", 1200)) instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") + before = dict(instrumentation.action_snapshots()[0]) driver.tree = [{"selector": "#after"}] self._step(driver, "getElementText", 1300) with mock.patch.object(instrumentation, "_take_screenshot", return_value=None): instrumentation.capture_final_frame(driver) snaps = instrumentation.action_snapshots() - self.assertEqual(snaps[0]["screenshot"], "SHOT") - self.assertEqual(snaps[0]["elements"][0]["selector"], "#after") + self.assertEqual(snaps, [before]) + + def test_a_screenshot_only_final_capture_supersedes_the_screenshot(self): + driver = self.Driver() + with self._ended(): + instrumentation._state["test_rows"].append(("clickElement", 1200)) + instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") + driver.tree = [] + self._step(driver, "getElementText", 1300) + with mock.patch.object(instrumentation, "_take_screenshot", return_value="END"): + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([(s["timestamp"], s["screenshot"]) for s in snaps], [(1200, "END")]) + self.assertEqual(snaps[0]["elements"][0]["selector"], "#before") def test_the_final_frame_comes_from_the_driver_that_captured_last(self): owner, other = self.Driver(), self.Driver() diff --git a/packages/selenium-devtools/src/helpers/actions-patcher.ts b/packages/selenium-devtools/src/helpers/actions-patcher.ts index 51d5283ff..d9e93562e 100644 --- a/packages/selenium-devtools/src/helpers/actions-patcher.ts +++ b/packages/selenium-devtools/src/helpers/actions-patcher.ts @@ -25,15 +25,17 @@ function patchActionsPerform( this: unknown ) => Promise actionsProto.perform = async function patchedPerform(this: unknown) { - await originalPerform.call(this) - const driver = actionsOwners.get(this as object) - if (!driver || !hooks.afterActions) { - return - } try { - hooks.afterActions(driver) - } catch (err) { - log.warn(`afterActions hook threw: ${errorMessage(err)}`) + await originalPerform.call(this) + } finally { + const driver = actionsOwners.get(this as object) + if (driver && hooks.afterActions) { + try { + hooks.afterActions(driver) + } catch (err) { + log.warn(`afterActions hook threw: ${errorMessage(err)}`) + } + } } } } diff --git a/packages/selenium-devtools/tests/driverPatcher.test.ts b/packages/selenium-devtools/tests/driverPatcher.test.ts index dec8c276e..7fa59c3ce 100644 --- a/packages/selenium-devtools/tests/driverPatcher.test.ts +++ b/packages/selenium-devtools/tests/driverPatcher.test.ts @@ -173,15 +173,17 @@ describe('patchActions', () => { expect(onCommand).not.toHaveBeenCalled() }) - it('reports nothing when a sequence fails to perform', async () => { + it('still reports a sequence that moved the page before it failed, and rethrows its error', async () => { const FakeDriver = fakeDriverClass() const afterActions = vi.fn() patchActions(FakeDriver, hooksWith(afterActions)) + const driver = new FakeDriver() - await expect( - new FakeDriver().actions({ fails: true }).perform() - ).rejects.toThrow('stale element') - expect(afterActions).not.toHaveBeenCalled() + await expect(driver.actions({ fails: true }).perform()).rejects.toThrow( + 'stale element' + ) + expect(afterActions).toHaveBeenCalledTimes(1) + expect(afterActions.mock.calls[0]![0]).toBe(driver) }) it('wraps perform once however often it is patched', async () => { From a83e63096e00b6e79af46e131094ec2d6c72db98 Mon Sep 17 00:00:00 2001 From: Vince Graics Date: Mon, 2 Nov 2026 11:10:40 +0100 Subject: [PATCH 14/14] refactor(trace): one capture gate, per-test capture state in core Addresses the review on #442. - The service gates on skipsCapture like every other adapter; changesPage is gone, since for ACTION_MAP commands the two always agreed. - Selenium and Nightwatch shared the per-test frame state, first-frame retry and end-of-test frame verbatim; that now lives in core's TestCapture. Selenium's end frame reads ctx.driver after the pending captures settle, as before the move. - Python keeps the same state in PerTestCapture rather than seven _state keys, and finalFrameStamp has a mirror-vectors section run by both sides. - getAllWindowHandles, printPage, getCredentials and getDownloadableFiles join READ_ONLY_COMMANDS; wire-schema and the Python contract regenerated. - CLAUDE.md describes the single gate; the afterActions comment says why a failed perform() still captures. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 6 +- packages/core/src/session-capturer.ts | 8 ++ packages/core/src/test-capture.ts | 102 ++++++++++++++++ packages/core/tests/test-capture.test.ts | 100 ++++++++++++++++ packages/nightwatch-devtools/src/session.ts | 109 ++++++------------ .../tests/actionSnapshot.test.ts | 4 +- .../changes/page-changing-capture.md | 2 +- .../src/selenium_devtools/_contract.py | 2 +- .../src/selenium_devtools/instrumentation.py | 54 ++++----- .../src/selenium_devtools/per_test_capture.py | 29 +++++ .../tests/test_a11y_elements.py | 23 ++-- .../tests/test_mirror_vectors.py | 14 +++ .../tests/test_pytest_plugin.py | 7 +- .../src/helpers/commandPostActions.ts | 48 +++----- .../src/helpers/finalFrame.ts | 41 +++---- packages/selenium-devtools/src/session.ts | 15 --- packages/selenium-devtools/src/types.ts | 2 +- .../tests/action-snapshot.test.ts | 4 +- .../tests/mirror-vectors.test.ts | 25 +++- packages/service/src/index.ts | 11 +- packages/shared/mirror-vectors.json | 52 +++++++++ packages/shared/src/trace-actions.ts | 13 +-- ...nge-page.test.ts => skips-capture.test.ts} | 61 ++++------ packages/shared/wire-schema.json | 4 + 24 files changed, 473 insertions(+), 263 deletions(-) create mode 100644 packages/core/src/test-capture.ts create mode 100644 packages/core/tests/test-capture.test.ts create mode 100644 packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py rename packages/shared/tests/{change-page.test.ts => skips-capture.test.ts} (66%) diff --git a/CLAUDE.md b/CLAUDE.md index 23ee832a1..754e0adbe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,7 +91,7 @@ The test for a new helper: *would the backend ever need this to build a zip?* If In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose capture predates the destination document's `performance.timeOrigin` (a navigation that committed after it returned), and discards that re-capture if a newer page capture was queued meanwhile. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. -Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and superseding that record field by field, only when it has a screenshot; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO keeps its page-changing gate (`changesPage`) and its next-command capture, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. +Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and superseding that record field by field, only when it has a screenshot; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO applies the same gate, but only to commands it renders as action rows, and captures before the next command rather than after its own, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. ### Adapters are thin and isolated @@ -389,7 +389,7 @@ Documented divergences from the conventions above. They exist today as debt to b - **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2. - A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch. - **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop. -- **Only a command that changes the page is captured** (`shared/trace-actions.ts` `changesPage`, read off the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit. +- **Only a command that can change the page is captured** (`shared/trace-actions.ts` `skipsCapture`, a deny-list: reads, waits, lookups and assertions are skipped, anything unlisted captures; the ACTION_MAP reads come from the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit. - The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands. ### File-size (raw line counts; soft cap is 500 logic lines) @@ -400,7 +400,7 @@ Most entries below don't trigger the `max-lines` lint rule after `skipBlankLines - `packages/nightwatch-devtools/src/index.ts` (783 raw / 676 logic). Cucumber/test/run-lifecycle, session-init, event-hub and now the screencast seam (`plugin-screencast.ts`, 105 raw / 60 logic) are extracted; the remainder is the `PluginInternals` accessor bag plus per-method delegators plus the factory. The bag is deliberately declarative — accept as-is. - `packages/selenium-devtools/src/index.ts` (~644 raw, down from ~758 — the dead `scriptInjected` accessor pair and setter are gone). Session/test-lifecycle **and** the per-test-artifact seam are now extracted: the sink cache + input snapshot + produce/attach flow live in `selenium-devtools/src/test-artifacts.ts` as `SeleniumTestArtifacts` (mirrors Nightwatch's twin — a typed input bag threading the Allure sink + flushed-trace promise), and the plugin keeps only a thin bag-building delegator. Remainder is the `PluginInternals` accessor bag plus onCommand/onDriverCreated wiring. Still over the 500 **raw** soft cap (under the logic-line cap after `skipBlankLines`/`skipComments`); the accessor bag / command wiring is the next extraction candidate if it grows. - `packages/nightwatch-devtools/src/session.ts` (519 raw, under the logic-line cap after `skipBlankLines`/`skipComments`). `captureNetworkFromPerformanceLogs` + `drainCollector` are tightly coupled to NightwatchBrowser state. Coverage at 78% after recent backfill; further extraction would need rewriting the browser-coupling. -- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (975 raw). Every seam here reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` itself should split it per concern first; `action_snapshot.py` is then a clean lift. +- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (1297 raw). The per-test frame state is split out (`per_test_capture.py` `PerTestCapture`, one `_state["test"]` entry); every other seam here still reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` should split another concern out the same way; `action_snapshot.py` is then a clean lift. ### Test coverage gaps (worst-risk-first) diff --git a/packages/core/src/session-capturer.ts b/packages/core/src/session-capturer.ts index 685021657..7c0457d02 100644 --- a/packages/core/src/session-capturer.ts +++ b/packages/core/src/session-capturer.ts @@ -24,6 +24,7 @@ import { } from '@wdio/devtools-shared' import { resolveRunId } from './run-id.js' import { reattributeDomAnchors } from '@wdio/devtools-trace/trace-mutations' +import { TestCapture } from './test-capture.js' import { CONSOLE_METHODS, LOG_SOURCES, @@ -139,6 +140,13 @@ export abstract class SessionCapturerBase { return sessionHasDocument(this.metadata?.capabilities, this.currentContext) } + /** Per-test capture state for adapters that capture after each command. */ + readonly testCapture = new TestCapture() + + resetTestCapture(): void { + this.testCapture.reset() + } + // ── Construction ──────────────────────────────────────────────────────── constructor(opts: SessionCapturerOptions = {}) { const { hostname, port, reconnect } = opts diff --git a/packages/core/src/test-capture.ts b/packages/core/src/test-capture.ts new file mode 100644 index 000000000..d4f2a05aa --- /dev/null +++ b/packages/core/src/test-capture.ts @@ -0,0 +1,102 @@ +import type { ActionSnapshot, CommandLog } from '@wdio/devtools-shared' +import { mapCommandToAction } from '@wdio/devtools-shared' +import { + supersedeSnapshotAt, + upsertRichestSnapshot +} from '@wdio/devtools-trace/trace-snapshots' +import { + capturedPageState, + finalFrameStamp, + type FinalFrameCommand +} from './final-frame.js' + +/** + * Per-test state of the capture an adapter takes after each command. The + * adapter decides whether a command may capture and supplies the capture call; + * this owns the initial-frame retry, the generation guard and the end frame. + */ +export class TestCapture { + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + initialFrameRetried = false + generation = 0 + /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ + lastCapturedTimestamp: number | undefined = undefined + startWallTime = 0 + + reset(): void { + this.awaitingInitialFrame = true + this.initialFrameRetried = false + this.generation++ + this.startWallTime = Date.now() + } + + /** + * `changesPage` is the adapter's call on whether the command can move the + * page. Reads, waits and lookups skip the capture and inherit the earlier + * one, except the test's first mapped command, which always takes one. + */ + queue( + command: string, + timestamp: number, + changesPage: boolean, + snapshots: ActionSnapshot[], + pending: Promise[], + capture: () => Promise + ): Promise | undefined { + const initialFrame = + this.awaitingInitialFrame && !!mapCommandToAction(command) + if (!changesPage && !initialFrame) { + return undefined + } + if (initialFrame) { + this.awaitingInitialFrame = false + } + this.lastCapturedTimestamp = timestamp + const generation = this.generation + const settled = capture().then((snap) => { + if (snap) { + upsertRichestSnapshot(snapshots, snap) + } + // A capture resolving after reset() belongs to the previous test. + if ( + initialFrame && + !snap?.screenshot && + !this.initialFrameRetried && + generation === this.generation + ) { + this.initialFrameRetried = true + this.awaitingInitialFrame = true + } + }) + pending.push(settled) + return settled + } + + /** Stamped at the capture that last saw the page; supersedes its record only with a screenshot. */ + async captureEndFrame( + commands: ReadonlyArray, + pending: Promise[], + snapshots: ActionSnapshot[], + capture: (stamp: FinalFrameCommand) => Promise, + onError: (err: unknown) => void + ): Promise { + await Promise.all(pending) + const stamp = finalFrameStamp( + commands, + this.startWallTime, + this.lastCapturedTimestamp + ) + if (!stamp) { + return + } + try { + const snap = await capture(stamp) + if (snap && capturedPageState(snap)) { + supersedeSnapshotAt(snapshots, snap) + } + } catch (err) { + onError(err) + } + } +} diff --git a/packages/core/tests/test-capture.test.ts b/packages/core/tests/test-capture.test.ts new file mode 100644 index 000000000..99e56eecb --- /dev/null +++ b/packages/core/tests/test-capture.test.ts @@ -0,0 +1,100 @@ +import { describe, it, expect } from 'vitest' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import { SessionCapturerBase } from '../src/session-capturer.js' + +class TestCapturer extends SessionCapturerBase { + readonly snapshots: ActionSnapshot[] = [] + readonly pending: Promise[] = [] + + queue( + command: string, + timestamp: number, + changesPage: boolean, + capture: () => Promise + ) { + return this.testCapture.queue( + command, + timestamp, + changesPage, + this.snapshots, + this.pending, + capture + ) + } +} + +const withScreenshot = (timestamp: number): Promise => + Promise.resolve({ timestamp, command: 'x', screenshot: 'png' }) +const withoutScreenshot = (timestamp: number): Promise => + Promise.resolve({ timestamp, command: 'x' }) + +function deferred() { + let resolve!: (value: T) => void + const promise = new Promise((r) => (resolve = r)) + return { promise, resolve } +} + +describe('TestCapture', () => { + it('clears the initial frame only on a capture that holds a screenshot', async () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withScreenshot(1)) + await Promise.all(capturer.pending) + + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + expect(capturer.snapshots.map((s) => s.timestamp)).toEqual([1]) + }) + + it('retries the initial frame once when its capture has no screenshot', async () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withoutScreenshot(1)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(true) + + capturer.queue('getText', 2, false, () => withoutScreenshot(2)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + + capturer.queue('getText', 3, false, () => withoutScreenshot(3)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + }) + + it('returns no capture for a read that is not the initial frame', () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withScreenshot(1)) + const skipped = capturer.queue('getText', 2, false, () => withScreenshot(2)) + + expect(skipped).toBeUndefined() + expect(capturer.testCapture.lastCapturedTimestamp).toBe(1) + }) + + it('ignores a capture that resolves after resetTestCapture()', async () => { + const capturer = new TestCapturer() + const stale = deferred() + capturer.queue('click', 1, true, () => stale.promise) + capturer.resetTestCapture() + + stale.resolve({ timestamp: 1, command: 'click' }) + await Promise.all(capturer.pending) + + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + }) + + it('resetTestCapture re-arms the initial frame and restamps the test start', () => { + const capturer = new TestCapturer() + capturer.queue('click', 1, true, () => withScreenshot(1)) + const generation = capturer.testCapture.generation + const before = Date.now() + + capturer.resetTestCapture() + + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + expect(capturer.testCapture.generation).toBe(generation + 1) + expect(capturer.testCapture.startWallTime).toBeGreaterThanOrEqual(before) + expect(capturer.testCapture.startWallTime).toBeLessThanOrEqual(Date.now()) + }) +}) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 5e8ac94ad..0b7d585ab 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -3,18 +3,13 @@ import { CAPTURE_PERFORMANCE_SCRIPT, SessionCapturerBase, applyPerformanceData, - capturedPageState, collectorDrainExpression, drainCollectorWithRecovery, errorMessage, - finalFrameStamp, loadInjectableScript, - mapCommandToAction, NativeNetworkCapture, pollUntilReady, - supersedeSnapshotAt, serializeError, - upsertRichestSnapshot, type CapturedPerformancePayload, type SessionCapturerOptions } from '@wdio/devtools-core' @@ -102,21 +97,6 @@ export class SessionCapturer extends SessionCapturerBase { log[level](message) ) - /** True until the current test's first mapped command takes its capture. */ - awaitingInitialFrame = true - initialFrameRetried = false - testCaptureGeneration = 0 - /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ - lastCapturedTimestamp: number | undefined = undefined - testStartWallTime = 0 - - resetTestCapture(): void { - this.awaitingInitialFrame = true - this.initialFrameRetried = false - this.testCaptureGeneration++ - this.testStartWallTime = Date.now() - } - constructor( devtoolsOptions: SessionCapturerOptions = {}, browser?: NightwatchBrowser @@ -204,44 +184,27 @@ export class SessionCapturer extends SessionCapturerBase { if (this.traceMode !== 'trace' || !this.#browser) { return } - // Reads, waits and lookups inherit the earlier capture; only the test's first mapped command adds an initial frame. - const initialFrame = - this.awaitingInitialFrame && !!mapCommandToAction(command) // `url()` without a string argument is a getter; it does not navigate. const pageChanging = !skipsCapture(command) && !(command === 'url' && typeof args[0] !== 'string') - if (!pageChanging && !initialFrame) { - return - } - if (initialFrame) { - this.awaitingInitialFrame = false - } - this.lastCapturedTimestamp = timestamp - const generation = this.testCaptureGeneration - const settled = captureActionSnapshot( - this.#browser, + const browser = this.#browser + const settled = this.testCapture.queue( command, timestamp, - this.runner, - this.isNativeAppSession - ).then((snap) => { - if (snap) { - upsertRichestSnapshot(this.actionSnapshots, snap) - } - // A capture resolving after resetTestCapture() belongs to the previous test. - if ( - initialFrame && - !snap?.screenshot && - !this.initialFrameRetried && - generation === this.testCaptureGeneration - ) { - this.initialFrameRetried = true - this.awaitingInitialFrame = true - } - }) - this.snapshotCaptures.push(settled) - if (pageChanging) { + pageChanging, + this.actionSnapshots, + this.snapshotCaptures, + () => + captureActionSnapshot( + browser, + command, + timestamp, + this.runner, + this.isNativeAppSession + ) + ) + if (settled && pageChanging) { this.#lastPageCapture = { command, timestamp, settled } } } @@ -682,31 +645,31 @@ export class SessionCapturer extends SessionCapturerBase { /** Rows after the last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ async captureFinalFrame(): Promise { - await Promise.all(this.snapshotCaptures) - const stamp = finalFrameStamp( - this.commandsLog, - this.testStartWallTime, - this.lastCapturedTimestamp - ) - if (this.traceMode !== 'trace' || !this.#browser || !stamp) { + if (this.traceMode !== 'trace') { return } - try { - const snap = await captureActionSnapshot( - this.#browser, - stamp.command, - stamp.timestamp, - this.runner, - this.isNativeAppSession - ) - if (snap && capturedPageState(snap)) { - supersedeSnapshotAt(this.actionSnapshots, snap) - } - } catch (err) { - log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) - } + await this.testCapture.captureEndFrame( + this.commandsLog, + this.snapshotCaptures, + this.actionSnapshots, + async (stamp) => { + if (!this.#browser) { + return null + } + return captureActionSnapshot( + this.#browser, + stamp.command, + stamp.timestamp, + this.runner, + this.isNativeAppSession + ) + }, + (err) => + log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) + ) } + /** At a test or scenario end, before its slice is written: the proxy drain there is never throttled. */ async captureTraceAtTestEnd(browser: NightwatchBrowser) { if (this.isNativeAppSession) { await this.nativeNetwork.forward() diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index 0e3ece450..f53752681 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -196,8 +196,8 @@ describe('nightwatch action-snapshot timestamp binding', () => { resolveFirst({ timestamp: 1, command: 'getText' } as Snap) await Promise.all(cap.snapshotCaptures) - expect(cap.initialFrameRetried).toBe(false) - expect(cap.awaitingInitialFrame).toBe(true) + expect(cap.testCapture.initialFrameRetried).toBe(false) + expect(cap.testCapture.awaitingInitialFrame).toBe(true) cap.cleanup() }) diff --git a/packages/selenium-devtools-py/changes/page-changing-capture.md b/packages/selenium-devtools-py/changes/page-changing-capture.md index 53e41f67b..d0c6caabf 100644 --- a/packages/selenium-devtools-py/changes/page-changing-capture.md +++ b/packages/selenium-devtools-py/changes/page-changing-capture.md @@ -2,4 +2,4 @@ patch --- -Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. When reads follow a test's last capture, the test's end takes one more capture, stamped at that last capture and superseding it; the capture adds no command row. Reads and waits keep their own rows and show the tree from the last capture before them. +Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. When reads follow a test's last capture, the test's end takes one more capture, stamped at that last capture and superseding it; the capture adds no command row. Reads and waits keep their own rows and show the tree from the last capture before them. Every command that returns still takes its screenshot, so a read saves only its element-tree read. diff --git a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py index 6040bded3..02a07429c 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py @@ -15,7 +15,7 @@ SCOPE_REPLACE_COMMAND = "replaceCommand" DATA_SCOPES = frozenset(['actionSnapshots', 'commands', 'config', 'consoleLogs', 'logs', 'metadata', 'mutations', 'networkRequests', 'screencast', 'screencastFrames', 'sources', 'suites']) -READ_ONLY_COMMANDS = frozenset(['$', '$$', 'element', 'elementScreenshot', 'elements', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getAccessibleName', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAriaRole', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getCapabilities', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCssProperty', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getDomAttribute', 'getElementAriaLabel', 'getElementAriaRole', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementSize', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getFirstElementChild', 'getHTML', 'getId', 'getLastElementChild', 'getLocation', 'getLog', 'getLogTypes', 'getNetworkConnection', 'getNextSibling', 'getPageSource', 'getPreviousSibling', 'getProperty', 'getRect', 'getScreenOrientation', 'getSession', 'getShadowRoot', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowPosition', 'getWindowRect', 'getWindowSize', 'hasDescendants', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isLogAvailable', 'isPresent', 'isSelected', 'isVisible', 'screenshot', 'source', 'status', 'takeScreenshot', 'title', 'w3cGetActiveElement', 'w3cGetAlertText', 'w3cGetCurrentWindowHandle', 'w3cGetWindowHandles', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil', 'windowHandle', 'windowHandles']) +READ_ONLY_COMMANDS = frozenset(['$', '$$', 'element', 'elementScreenshot', 'elements', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getAccessibleName', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAllWindowHandles', 'getAriaRole', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getCapabilities', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCredentials', 'getCssProperty', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getDomAttribute', 'getDownloadableFiles', 'getElementAriaLabel', 'getElementAriaRole', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementSize', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getFirstElementChild', 'getHTML', 'getId', 'getLastElementChild', 'getLocation', 'getLog', 'getLogTypes', 'getNetworkConnection', 'getNextSibling', 'getPageSource', 'getPreviousSibling', 'getProperty', 'getRect', 'getScreenOrientation', 'getSession', 'getShadowRoot', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowPosition', 'getWindowRect', 'getWindowSize', 'hasDescendants', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isLogAvailable', 'isPresent', 'isSelected', 'isVisible', 'printPage', 'screenshot', 'source', 'status', 'takeScreenshot', 'title', 'w3cGetActiveElement', 'w3cGetAlertText', 'w3cGetCurrentWindowHandle', 'w3cGetWindowHandles', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil', 'windowHandle', 'windowHandles']) MAPPED_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'getAttribute', 'getCSSProperty', 'getCssValue', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getLocation', 'getPageSource', 'getProperty', 'getRect', 'getSize', 'getTagName', 'getText', 'getTitle', 'getUrl', 'getValue', 'goBack', 'goForward', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) CONTROL_SCOPES = frozenset(['clearCommands', 'clearExecutionData', 'clientConnected', 'clientDisconnected', 'config', 'replaceCommand', 'testStopped']) diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index 483ef32ce..bf0ab6560 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -38,6 +38,7 @@ from .device import driver_is_native_app, restore_browser_focus from .element_scripts import reset_cache as reset_element_scripts_cache from .page_scripts import reset_cache as reset_page_scripts_cache +from .per_test_capture import PerTestCapture from .constants import ( BIDI_CAPABILITY, DEFAULT_TEST_TITLE, @@ -290,15 +291,7 @@ def _capture_source(capturer: SessionCapturer, call_src: Optional[str]) -> None: # driver, and the filmstrip is the whole run. "filmstrip_frames": [], # Re-armed per test by reset_test_capture(). - "awaiting_initial_frame": True, - "initial_frame_retried": False, - # The test's command rows as (command, timestamp), and the timestamp of the - # last capture attempted — what the end-of-test capture is decided from. - "test_rows": [], - "test_started_ms": 0, - "last_captured_ts": None, - "final_frame_driver": None, - "final_frame_taken": False, + "test": PerTestCapture(), } @@ -497,22 +490,22 @@ def _maybe_capture_action_snapshot( """Capture on every command that is not a known read, and on a test's first mapped command as its initial frame. Reads and waits inherit the capture before them.""" - initial = _state["awaiting_initial_frame"] and command in MAPPED_COMMANDS + initial = _state["test"].awaiting_initial_frame and command in MAPPED_COMMANDS _note_final_frame_driver(driver) if command in READ_ONLY_COMMANDS and not initial: return if not _action_capture_enabled(driver): return if initial: - _state["awaiting_initial_frame"] = False - _state["last_captured_ts"] = timestamp - _state["final_frame_driver"] = weakref.ref(driver) + _state["test"].awaiting_initial_frame = False + _state["test"].last_captured_ts = timestamp + _state["test"].final_frame_driver = weakref.ref(driver) snapshot = _capture_action_snapshot(driver, command, timestamp, shot) - if not initial or _state["initial_frame_retried"]: + if not initial or _state["test"].initial_frame_retried: return if snapshot is None or "screenshot" not in snapshot: - _state["initial_frame_retried"] = True - _state["awaiting_initial_frame"] = True + _state["test"].initial_frame_retried = True + _state["test"].awaiting_initial_frame = True def _capture_command(driver_command: str, params: Any) -> str: @@ -545,21 +538,20 @@ def _final_frame_stamp( def _note_final_frame_driver(driver: Any) -> None: """Until a capture is taken, the test's stamp belongs to the driver that ran its latest row.""" - if _state["last_captured_ts"] is None: - _state["final_frame_driver"] = weakref.ref(driver) + if _state["test"].last_captured_ts is None: + _state["test"].final_frame_driver = weakref.ref(driver) def _capture_final_frame(driver: Any) -> bool: if not _action_capture_enabled(driver): return False stamp = _final_frame_stamp( - _state["test_rows"], _state["test_started_ms"], _state["last_captured_ts"] + _state["test"].rows, _state["test"].started_ms, _state["test"].last_captured_ts ) if stamp is None: return False command, timestamp = stamp # Frames are screenshot-backed: panes merged over an earlier image would pair two documents. - # Frames are screenshot-backed: panes merged over an earlier image would pair two documents. shot = _take_screenshot(driver) if not shot: return False @@ -576,7 +568,7 @@ def _capture_final_frame(driver: Any) -> bool: kept for kept in _state["action_snapshots"] if kept["timestamp"] != timestamp ] _state["action_snapshots"].append({**superseded, **snapshot}) - _state["last_captured_ts"] = timestamp + _state["test"].last_captured_ts = timestamp return True @@ -590,26 +582,22 @@ def capture_final_frame(driver: Any = None) -> None: the driver that owns the stamp acts, so another session quitting mid-test neither captures from nor consumes the test's one final frame. """ - if _state["final_frame_taken"]: + if _state["test"].final_frame_taken: return - owner_ref = _state["final_frame_driver"] + owner_ref = _state["test"].final_frame_driver owner = owner_ref() if owner_ref is not None else None if owner is None or (driver is not None and driver is not owner): return try: if _capture_final_frame(owner): - _state["final_frame_taken"] = True + _state["test"].final_frame_taken = True except Exception as exc: # noqa: BLE001 — a missing end frame, not a failed run _log.debug("final frame capture failed: %s", exc) def reset_test_capture() -> None: """Re-arm the initial frame and start the row ledger for the test about to start.""" - _state.update( - awaiting_initial_frame=True, initial_frame_retried=False, - test_rows=[], test_started_ms=now_ms(), - last_captured_ts=None, final_frame_driver=None, final_frame_taken=False, - ) + _state["test"].reset(now_ms()) def _begin_screencast_run(entry: Optional[dict], shot: Optional[str] = None) -> None: @@ -1187,7 +1175,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN ) wait_tracking.note_poll_row(poll, failed) _note_final_frame_driver(self) - _state["test_rows"].append( + _state["test"].rows.append( (_capture_command(driver_command, params), failed["timestamp"]) ) raise @@ -1215,7 +1203,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN wait_tracking.note_poll_row(poll, row) command = _capture_command(driver_command, params) stamp = row.get("timestamp", start) - _state["test_rows"].append((command, stamp)) + _state["test"].rows.append((command, stamp)) _maybe_capture_action_snapshot(self, command, stamp, shot) if driver_command in NAVIGATION_COMMANDS: _attach_performance(capturer, self, row, params) @@ -1245,10 +1233,8 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN sessions=weakref.WeakKeyDictionary(), output_dir=None, default_suite=None, trace=trace, filmstrip=filmstrip, filmstrip_frames=[], a11y=a11y, element_scripts=None, page_scripts=None, action_snapshots=[], - awaiting_initial_frame=True, initial_frame_retried=False, - test_rows=[], test_started_ms=now_ms(), last_captured_ts=None, - final_frame_driver=None, final_frame_taken=False, ) + _state["test"].reset(now_ms()) def _executor_host(driver: Any) -> Optional[str]: diff --git a/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py b/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py new file mode 100644 index 000000000..76b7ecc3a --- /dev/null +++ b/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py @@ -0,0 +1,29 @@ +"""The state one test's capture is decided from: its initial frame and its end frame.""" + +from __future__ import annotations + +import weakref +from dataclasses import dataclass, field +from typing import List, Optional + + +@dataclass +class PerTestCapture: + awaiting_initial_frame: bool = True + initial_frame_retried: bool = False + # The test's command rows as (command, timestamp). + rows: List[tuple] = field(default_factory=list) + started_ms: int = 0 + # The timestamp of the last capture attempted. + last_captured_ts: Optional[int] = None + final_frame_driver: Optional[weakref.ref] = None + final_frame_taken: bool = False + + def reset(self, started_ms: int) -> None: + self.awaiting_initial_frame = True + self.initial_frame_retried = False + self.rows = [] + self.started_ms = started_ms + self.last_captured_ts = None + self.final_frame_driver = None + self.final_frame_taken = False diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index f755e149c..ecad43f3c 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -12,6 +12,7 @@ from unittest import mock from selenium_devtools import element_scripts, instrumentation, trace_export +from selenium_devtools.per_test_capture import PerTestCapture from selenium_devtools._contract import ( ELEMENT_SCRIPTS_PATH, MAPPED_COMMANDS, @@ -229,8 +230,7 @@ def _armed(self, driver=None): "a11y": True, "element_scripts": SCRIPTS, "action_snapshots": [], - "awaiting_initial_frame": True, - "initial_frame_retried": False, + "test": PerTestCapture(), }, ): yield driver or self.Driver(elements=[{"selector": "#go"}]) @@ -297,7 +297,7 @@ class NativeDriver(self.Driver): with self._armed(NativeDriver(elements=[{"selector": "#go"}])) as driver: self._gate(driver, "getElementText") - self.assertTrue(instrumentation._state["awaiting_initial_frame"]) + self.assertTrue(instrumentation._state["test"].awaiting_initial_frame) self.assertEqual(driver.scripts_run, []) # Deny-list: a command the set does not name is captured, so only commands @@ -368,18 +368,13 @@ def _ended(self): "a11y": True, "element_scripts": SCRIPTS, "action_snapshots": [], - "awaiting_initial_frame": False, - "initial_frame_retried": False, - "test_rows": [], - "test_started_ms": 1000, - "last_captured_ts": None, - "final_frame_taken": False, + "test": PerTestCapture(awaiting_initial_frame=False, started_ms=1000), }, ): yield def _step(self, driver, command, timestamp): - instrumentation._state["test_rows"].append((command, timestamp)) + instrumentation._state["test"].rows.append((command, timestamp)) instrumentation._maybe_capture_action_snapshot(driver, command, timestamp, None) def test_the_reads_after_a_click_replace_its_stale_capture(self): @@ -431,7 +426,7 @@ def test_the_final_capture_is_taken_once_per_test(self): def test_a_final_capture_without_a_screenshot_leaves_the_record_unchanged(self): driver = self.Driver() with self._ended(): - instrumentation._state["test_rows"].append(("clickElement", 1200)) + instrumentation._state["test"].rows.append(("clickElement", 1200)) instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") before = dict(instrumentation.action_snapshots()[0]) driver.tree = [{"selector": "#after"}] @@ -444,7 +439,7 @@ def test_a_final_capture_without_a_screenshot_leaves_the_record_unchanged(self): def test_a_screenshot_only_final_capture_supersedes_the_screenshot(self): driver = self.Driver() with self._ended(): - instrumentation._state["test_rows"].append(("clickElement", 1200)) + instrumentation._state["test"].rows.append(("clickElement", 1200)) instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") driver.tree = [] self._step(driver, "getElementText", 1300) @@ -470,11 +465,11 @@ def test_another_drivers_quit_leaves_the_final_frame_to_the_owner(self): self._step(owner, "clickElement", 1200) self._step(other, "getElementText", 1300) instrumentation.capture_final_frame(other) - self.assertFalse(instrumentation._state["final_frame_taken"]) + self.assertFalse(instrumentation._state["test"].final_frame_taken) before = owner.scripts_run instrumentation.capture_final_frame(owner) self.assertEqual(owner.scripts_run, before + 2) - self.assertTrue(instrumentation._state["final_frame_taken"]) + self.assertTrue(instrumentation._state["test"].final_frame_taken) def test_window_and_aria_reads_take_no_capture(self): driver = self.Driver() diff --git a/packages/selenium-devtools-py/tests/test_mirror_vectors.py b/packages/selenium-devtools-py/tests/test_mirror_vectors.py index 5f069abd9..13644bc7d 100644 --- a/packages/selenium-devtools-py/tests/test_mirror_vectors.py +++ b/packages/selenium-devtools-py/tests/test_mirror_vectors.py @@ -21,6 +21,7 @@ _contract, capturer, device, + instrumentation, output_dir, performance, read_value_locators, @@ -50,6 +51,7 @@ def _vectors_path() -> Path: "nestedWaitIdentity", "performancePayload", "waitRowNaming", + "finalFrameStamp", } UUID4 = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$") @@ -348,5 +350,17 @@ def test_cases(self): self.assertEqual({"command": command, "selector": selector}, case["expect"]) +class TestFinalFrameStamp(unittest.TestCase): + def test_cases(self): + for case in _cases("finalFrameStamp"): + with self.subTest(case["name"]): + rows = [(row["command"], row["timestamp"]) for row in case["commands"]] + stamp = instrumentation._final_frame_stamp( + rows, case["testStartMs"], case["lastCapturedTimestamp"] + ) + expect = None if case["expect"] is None else (case["expect"]["command"], case["expect"]["timestamp"]) + self.assertEqual(stamp, expect) + + if __name__ == "__main__": unittest.main() diff --git a/packages/selenium-devtools-py/tests/test_pytest_plugin.py b/packages/selenium-devtools-py/tests/test_pytest_plugin.py index 795bdf2a5..40f17d133 100644 --- a/packages/selenium-devtools-py/tests/test_pytest_plugin.py +++ b/packages/selenium-devtools-py/tests/test_pytest_plugin.py @@ -12,6 +12,7 @@ from unittest import mock from selenium_devtools import assertions, instrumentation +from selenium_devtools.per_test_capture import PerTestCapture from selenium_devtools import pytest_plugin as plugin from selenium_devtools.pytest_plugin import _SuiteRegistry, reported_state from selenium_devtools.utils import iso @@ -556,14 +557,14 @@ class TestEachTestStartsWithItsOwnInitialFrame(unittest.TestCase): def test_a_test_starting_re_arms_the_initial_frame(self): with mock.patch.dict( instrumentation._state, - {"awaiting_initial_frame": False, "initial_frame_retried": True}, + {"test": PerTestCapture(awaiting_initial_frame=False, initial_frame_retried=True)}, ), mock.patch.object(plugin, "_opted_in", return_value=True), mock.patch.object( plugin.devtools, "get_capturer", return_value=None ): plugin.pytest_runtest_logstart("tests/t.py::test_a", ("tests/t.py", 1, "test_a")) - self.assertIs(instrumentation._state["awaiting_initial_frame"], True) - self.assertIs(instrumentation._state["initial_frame_retried"], False) + self.assertIs(instrumentation._state["test"].awaiting_initial_frame, True) + self.assertIs(instrumentation._state["test"].initial_frame_retried, False) diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index 044a5c721..2ea96ff87 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -5,9 +5,7 @@ import { errorMessage, isSessionGoneError, mapAssertCommand, - mapCommandToAction, toError, - upsertRichestSnapshot, type CapturedPerformancePayload, type RetryTracker } from '@wdio/devtools-core' @@ -375,37 +373,19 @@ function queueActionSnapshot( if (ctx.options.mode !== 'trace' || error || !ctx.driver) { return } - // Reads, waits and lookups inherit the earlier capture; only the test's first mapped command adds an initial frame. - const initialFrame = - capturer.awaitingInitialFrame && !!mapCommandToAction(command) - if (skipsCapture(command) && !initialFrame) { - return - } - if (initialFrame) { - capturer.awaitingInitialFrame = false - } - capturer.lastCapturedTimestamp = timestamp - const generation = capturer.testCaptureGeneration - ctx.snapshotCaptures.push( - captureActionSnapshot( - ctx.driver, - command, - timestamp, - capturer.isNativeAppSession - ).then((snap) => { - if (snap) { - upsertRichestSnapshot(ctx.actionSnapshots, snap) - } - // A capture resolving after resetTestCapture() belongs to the previous test. - if ( - initialFrame && - !snap?.screenshot && - !capturer.initialFrameRetried && - generation === capturer.testCaptureGeneration - ) { - capturer.initialFrameRetried = true - capturer.awaitingInitialFrame = true - } - }) + const driver = ctx.driver + capturer.testCapture.queue( + command, + timestamp, + !skipsCapture(command), + ctx.actionSnapshots, + ctx.snapshotCaptures, + () => + captureActionSnapshot( + driver, + command, + timestamp, + capturer.isNativeAppSession + ) ) } diff --git a/packages/selenium-devtools/src/helpers/finalFrame.ts b/packages/selenium-devtools/src/helpers/finalFrame.ts index db785dd64..a996e3950 100644 --- a/packages/selenium-devtools/src/helpers/finalFrame.ts +++ b/packages/selenium-devtools/src/helpers/finalFrame.ts @@ -1,10 +1,5 @@ import logger from '@wdio/logger' -import { - capturedPageState, - errorMessage, - finalFrameStamp, - supersedeSnapshotAt -} from '@wdio/devtools-core' +import { errorMessage } from '@wdio/devtools-core' import { captureActionSnapshot } from '../action-snapshot.js' import type { SessionLifecycleCtx } from '../session-lifecycle.js' @@ -23,26 +18,20 @@ export async function captureFinalFrame( ) { return } - await Promise.all(ctx.snapshotCaptures) - const stamp = finalFrameStamp( + await capturer.testCapture.captureEndFrame( capturer.commandsLog, - capturer.testStartWallTime, - capturer.lastCapturedTimestamp + ctx.snapshotCaptures, + ctx.actionSnapshots, + // ctx.driver is read after the pending captures settle: a driver swap meanwhile must be honoured. + (stamp) => + ctx.driver + ? captureActionSnapshot( + ctx.driver, + stamp.command, + stamp.timestamp, + capturer.isNativeAppSession + ) + : Promise.resolve(null), + (err) => log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) ) - if (!stamp) { - return - } - try { - const snap = await captureActionSnapshot( - ctx.driver, - stamp.command, - stamp.timestamp, - capturer.isNativeAppSession - ) - if (snap && capturedPageState(snap)) { - supersedeSnapshotAt(ctx.actionSnapshots, snap) - } - } catch (err) { - log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) - } } diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index c90e3bc16..f8e68003e 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -55,21 +55,6 @@ export class SessionCapturer extends SessionCapturerBase { // A screenshot that reaches a driver being quit is skipped, not reported as a failure. #endingDrivers = new WeakSet() - /** True until the current test's first mapped command takes its capture. */ - awaitingInitialFrame = true - initialFrameRetried = false - testCaptureGeneration = 0 - /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ - lastCapturedTimestamp: number | undefined = undefined - testStartWallTime = 0 - - resetTestCapture(): void { - this.awaitingInitialFrame = true - this.initialFrameRetried = false - this.testCaptureGeneration++ - this.testStartWallTime = Date.now() - } - #clientConnected = false #clientConnectedWaiters: Array<() => void> = [] #onClientDisconnected?: () => void diff --git a/packages/selenium-devtools/src/types.ts b/packages/selenium-devtools/src/types.ts index 2a9057d3a..e57912f85 100644 --- a/packages/selenium-devtools/src/types.ts +++ b/packages/selenium-devtools/src/types.ts @@ -159,7 +159,7 @@ export interface DriverPatcherHooks { afterNavigation?: (command: string) => Promise // Awaited before an element click, sendKeys, clear or submit is sent. beforeInput?: () => Promise - // Fired once an action sequence's perform() succeeds: it moves the page with no row of its own. + // Fired after perform() settles, failed too: a partly performed sequence may already have moved the page. afterActions?: (driver: SeleniumDriverLike) => void } diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index a4c32d353..af2ddfc2b 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -263,8 +263,8 @@ describe('selenium action-snapshot capture gate', () => { resolveFirst({ timestamp: 1, command: 'getText' } as Snap) await Promise.all(bag.snapshotCaptures) - expect(bag.capturer.initialFrameRetried).toBe(false) - expect(bag.capturer.awaitingInitialFrame).toBe(true) + expect(bag.capturer.testCapture.initialFrameRetried).toBe(false) + expect(bag.capturer.testCapture.awaitingInitialFrame).toBe(true) }) describe('end-of-test frame', () => { diff --git a/packages/selenium-devtools/tests/mirror-vectors.test.ts b/packages/selenium-devtools/tests/mirror-vectors.test.ts index f88cb1b81..abf78dfd9 100644 --- a/packages/selenium-devtools/tests/mirror-vectors.test.ts +++ b/packages/selenium-devtools/tests/mirror-vectors.test.ts @@ -17,6 +17,7 @@ import { } from '@wdio/devtools-shared' import { applyPerformanceData, + finalFrameStamp, resolveAdapterOutputDir, resolveRunId, SessionCapturerBase, @@ -75,7 +76,8 @@ const RUNNERS = [ 'waitPollKeys', 'nestedWaitIdentity', 'performancePayload', - 'waitRowNaming' + 'waitRowNaming', + 'finalFrameStamp' ] it('runs every section the vectors file declares', () => { @@ -407,3 +409,24 @@ describe('waitRowNaming', () => { }) } }) + +describe('finalFrameStamp', () => { + type Row = { command: string; timestamp: number } + for (const c of section< + Named & { + commands: Row[] + testStartMs: number + lastCapturedTimestamp: number | null + expect: Row | null + } + >('finalFrameStamp').cases) { + it(c.name, () => { + const stamp = finalFrameStamp( + c.commands, + c.testStartMs, + c.lastCapturedTimestamp ?? undefined + ) + expect(stamp).toEqual(c.expect) + }) + } +}) diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index 45034d20f..69b32fd4e 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -38,8 +38,8 @@ import { settleAfterLastAction } from './action-snapshot.js' import { - changesPage, isWaitCommand, + skipsCapture, type ActionSnapshot, type TestMetadataMap } from '@wdio/devtools-shared' @@ -591,7 +591,9 @@ export default class DevToolsHookService implements Services.ServiceInstance { // action's has no successor and this is the only capture of it. A session // that ran no action names it FINAL_SNAPSHOT_COMMAND, which the per-test // screenshot reads as "post-teardown frame, do not use". - const lastChange = this.#lastCommandWhere(changesPage) + const lastChange = this.#lastCommandWhere( + (command) => !skipsCapture(command) + ) const lastAction = lastChange && lastChange.timestamp >= this.#currentTestStartWallTime ? lastChange @@ -798,7 +800,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#sessionCapturer.currentContext ) && this.#isActionCommand(command) && - (changesPage(command) || this.#awaitingInitialCapture) + (!skipsCapture(command) || this.#awaitingInitialCapture) ) { // Stamped at the previous page-changing command's end so this capture IS that command's // result — except across a test boundary, where that slot already holds @@ -808,7 +810,8 @@ export default class DevToolsHookService implements Services.ServiceInstance { // capture of a test stamps now instead, the same rule the session's first // capture uses to become the initial frame. const previousEnd = - this.#lastCommandWhere(changesPage)?.timestamp ?? Date.now() + this.#lastCommandWhere((cmd) => !skipsCapture(cmd))?.timestamp ?? + Date.now() const snap = await captureActionSnapshot( this.#browser, command, diff --git a/packages/shared/mirror-vectors.json b/packages/shared/mirror-vectors.json index 991fa4372..f4255e80f 100644 --- a/packages/shared/mirror-vectors.json +++ b/packages/shared/mirror-vectors.json @@ -603,6 +603,58 @@ "expect": { "command": "wait", "selector": null } } ] + }, + "finalFrameStamp": { + "mirrors": { + "ts": "core/final-frame.ts finalFrameStamp", + "py": "instrumentation.py _final_frame_stamp" + }, + "notes": "commands are the test's rows in completion order. expect is the stamp the end-of-test capture takes, or null when no row follows the last capture. A stamp at the last capture names the last row at or before it, and a stamp at a row names that row.", + "cases": [ + { + "name": "no capture yet stamps the test's last row", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": null, + "expect": { "command": "getText", "timestamp": 1200 } + }, + { + "name": "a capture from before the test started does not count", + "commands": [ + { "command": "click", "timestamp": 500 }, + { "command": "getText", "timestamp": 1200 }, + { "command": "click", "timestamp": 1300 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 900, + "expect": { "command": "click", "timestamp": 1300 } + }, + { + "name": "no row after the capture takes none", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 1200, + "expect": null + }, + { + "name": "rows after the capture stamp it, owned by the last row at or before it", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 }, + { "command": "click", "timestamp": 1300 }, + { "command": "getText", "timestamp": 1400 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 1250, + "expect": { "command": "getText", "timestamp": 1250 } + } + ] } } } diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index 6700af013..a60ce0ed1 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -269,15 +269,6 @@ export function isInputDispatchingCommand(command: string): boolean { return action !== undefined && POINTABLE_METHODS.has(action.method) } -/** Whether a command can alter what the page shows; reads and waits don't - * warrant their own capture (cost) and inherit the latest earlier one via `claimAfter`. */ -export function changesPage(command: string): boolean { - if (!Object.prototype.hasOwnProperty.call(ACTION_MAP, command)) { - return false - } - return METHOD_EFFECT[ACTION_MAP[command].method] === 'changes' -} - /** Driver lookups and reads that never change the page and have no ACTION_MAP * entry; the ACTION_MAP reads are covered by METHOD_EFFECT in `skipsCapture`. */ export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ @@ -293,11 +284,14 @@ export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ 'getActiveElement', 'getWindowHandle', 'getWindowHandles', + 'getAllWindowHandles', 'getCurrentContextHandle', 'getContextHandles', 'getCookies', 'getAllCookies', 'getCookie', + 'getCredentials', + 'getDownloadableFiles', 'getWindowRect', 'getLog', 'getAvailableLogTypes', @@ -308,6 +302,7 @@ export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ 'takeScreenshot', 'screenshot', 'elementScreenshot', + 'printPage', 'getHTML', 'getComputedRole', 'getComputedLabel', diff --git a/packages/shared/tests/change-page.test.ts b/packages/shared/tests/skips-capture.test.ts similarity index 66% rename from packages/shared/tests/change-page.test.ts rename to packages/shared/tests/skips-capture.test.ts index e8f16d29e..69a48e120 100644 --- a/packages/shared/tests/change-page.test.ts +++ b/packages/shared/tests/skips-capture.test.ts @@ -1,51 +1,19 @@ import { describe, it, expect } from 'vitest' -import { - ACTION_MAP, - METHOD_EFFECT, - changesPage, - skipsCapture -} from '@wdio/devtools-shared' - -describe('changesPage', () => { - it.each([ - 'click', - 'setValue', - 'url', - 'execute', - 'keys', - 'sendKeys', - 'switchToWindow' - ])('is true for %s', (command) => { - expect(changesPage(command)).toBe(true) - }) +import { ACTION_MAP, METHOD_EFFECT, skipsCapture } from '@wdio/devtools-shared' +describe('skipsCapture', () => { it.each([ 'getText', 'isDisplayed', 'waitForDisplayed', + 'waitForElementVisible', 'waitUntil', 'getTitle', - 'waitForElementVisible', - 'expect.toHaveText', - 'assert.equal', - 'findElement', - 'constructor' - ])('is false for %s', (command) => { - expect(changesPage(command)).toBe(false) - }) - - it('classifies the method of every ACTION_MAP entry', () => { - for (const { method } of Object.values(ACTION_MAP)) { - expect(Object.keys(METHOD_EFFECT)).toContain(method) - } - }) -}) - -describe('skipsCapture', () => { - it.each([ - 'getText', - 'waitForDisplayed', 'findElement', + 'getAllWindowHandles', + 'printPage', + 'getCredentials', + 'getDownloadableFiles', 'getCurrentUrl', 'assert.equal', 'expect.toHaveText', @@ -61,13 +29,26 @@ describe('skipsCapture', () => { it.each([ 'click', + 'setValue', + 'url', + 'execute', + 'keys', + 'sendKeys', + 'switchToWindow', 'executeScript', 'actions', 'submitForm', 'acceptAlert', 'w3cExecuteScript', - 'someUnknownCommand' + 'someUnknownCommand', + 'constructor' ])('is false for %s', (command) => { expect(skipsCapture(command)).toBe(false) }) + + it('classifies the method of every ACTION_MAP entry', () => { + for (const { method } of Object.values(ACTION_MAP)) { + expect(Object.keys(METHOD_EFFECT)).toContain(method) + } + }) }) diff --git a/packages/shared/wire-schema.json b/packages/shared/wire-schema.json index 9b03aa678..14d50cadb 100644 --- a/packages/shared/wire-schema.json +++ b/packages/shared/wire-schema.json @@ -1523,6 +1523,7 @@ "getActiveElement", "getAlertText", "getAllCookies", + "getAllWindowHandles", "getAriaRole", "getAttribute", "getAvailableLogTypes", @@ -1533,11 +1534,13 @@ "getContextHandles", "getCookie", "getCookies", + "getCredentials", "getCssProperty", "getCssValue", "getCurrentContextHandle", "getCurrentUrl", "getDomAttribute", + "getDownloadableFiles", "getElementAriaLabel", "getElementAriaRole", "getElementAttribute", @@ -1588,6 +1591,7 @@ "isPresent", "isSelected", "isVisible", + "printPage", "screenshot", "source", "status",