diff --git a/.changeset/capture-only-page-changes.md b/.changeset/capture-only-page-changes.md new file mode 100644 index 000000000..dd5da0cf4 --- /dev/null +++ b/.changeset/capture-only-page-changes.md @@ -0,0 +1,9 @@ +--- +'@wdio/devtools-service': patch +'@wdio/selenium-devtools': patch +'@wdio/nightwatch-devtools': patch +--- + +Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for reads, waits and lookups such as `getText`, `isDisplayed`, `waitForDisplayed` or `findElement`. Those rows still appear in the trace and show the page as of the last command that took a capture (any command except reads, waits, lookups and asserts). Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). + +Selenium and Nightwatch now capture every other command, including ones that can navigate but are not in the action map (`executeScript`, `actions`, Nightwatch `submitForm`), and take one more capture at a test's end when reads follow its last capture, so a click that returns before its navigation commits no longer leaves the reads showing the page it left. The WDIO service keeps its page-changing gate. diff --git a/CLAUDE.md b/CLAUDE.md index ca8be4f93..c2df736bc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,7 +91,9 @@ The test for a new helper: *would the backend ever need this to build or read a ### One DOM snapshot per action, taken before it -The per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. +In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose capture predates the destination document's `performance.timeOrigin` (a navigation that committed after it returned), and discards that re-capture if a newer page capture was queued meanwhile. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag. + +Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and superseding that record field by field, only when it has a screenshot; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO applies the same gate, but only to commands it renders as action rows, and captures before the next command rather than after its own, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`. ### Adapters are thin and isolated @@ -398,7 +400,7 @@ Documented divergences from the conventions above. They exist today as debt to b - **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2. - A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch. - **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop. -- Residual: **a polling matcher fires one capture per poll** — each poll is a top-level mapped read — so "one capture per action" undercounts a real suite. Pre-existing in `d924a02`, unmeasured on a device, and the likeliest next lever. +- **Only a command that can change the page is captured** (`shared/trace-actions.ts` `skipsCapture`, a deny-list: reads, waits, lookups and assertions are skipped, anything unlisted captures; the ACTION_MAP reads come from the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit. - The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands. ### File-size (raw line counts; soft cap is 500 logic lines) @@ -409,7 +411,7 @@ Most entries below don't trigger the `max-lines` lint rule after `skipBlankLines - `packages/nightwatch-devtools/src/index.ts` (783 raw / 676 logic). Cucumber/test/run-lifecycle, session-init, event-hub and now the screencast seam (`plugin-screencast.ts`, 105 raw / 60 logic) are extracted; the remainder is the `PluginInternals` accessor bag plus per-method delegators plus the factory. The bag is deliberately declarative — accept as-is. - `packages/selenium-devtools/src/index.ts` (~644 raw, down from ~758 — the dead `scriptInjected` accessor pair and setter are gone). Session/test-lifecycle **and** the per-test-artifact seam are now extracted: the sink cache + input snapshot + produce/attach flow live in `selenium-devtools/src/test-artifacts.ts` as `SeleniumTestArtifacts` (mirrors Nightwatch's twin — a typed input bag threading the Allure sink + flushed-trace promise), and the plugin keeps only a thin bag-building delegator. Remainder is the `PluginInternals` accessor bag plus onCommand/onDriverCreated wiring. Still over the 500 **raw** soft cap (under the logic-line cap after `skipBlankLines`/`skipComments`); the accessor bag / command wiring is the next extraction candidate if it grows. - `packages/nightwatch-devtools/src/session.ts` (519 raw, under the logic-line cap after `skipBlankLines`/`skipComments`). `captureNetworkFromPerformanceLogs` + `drainCollector` are tightly coupled to NightwatchBrowser state. Coverage at 78% after recent backfill; further extraction would need rewriting the browser-coupling. -- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (975 raw). Every seam here reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` itself should split it per concern first; `action_snapshot.py` is then a clean lift. +- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (1297 raw). The per-test frame state is split out (`per_test_capture.py` `PerTestCapture`, one `_state["test"]` entry); every other seam here still reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` should split another concern out the same way; `action_snapshot.py` is then a clean lift. ### Test coverage gaps (worst-risk-first) diff --git a/docs/ADAPTERS.md b/docs/ADAPTERS.md index 05512988c..aa4d59b54 100644 --- a/docs/ADAPTERS.md +++ b/docs/ADAPTERS.md @@ -169,7 +169,7 @@ All times are epoch milliseconds except the suite/test `start`/`end` ISO strings ### Generating your types -`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with: +`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`, and the command names the Python capture gate reads under `actionCommands`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with: ```sh pnpm --filter @wdio/devtools-shared gen:wire-schema diff --git a/packages/core/src/final-frame.ts b/packages/core/src/final-frame.ts new file mode 100644 index 000000000..d3fdb48a2 --- /dev/null +++ b/packages/core/src/final-frame.ts @@ -0,0 +1,36 @@ +import type { ActionSnapshot } from '@wdio/devtools-shared' + +/** Frames are screenshot-backed: a url-only capture merged over an earlier record would pair the new url with the old image and DOM. */ +export function capturedPageState(snap: ActionSnapshot): boolean { + return Boolean(snap.screenshot) +} + +export interface FinalFrameCommand { + command: string + timestamp: number +} + +/** Rows after the last capture saw a state no capture holds; the end-of-test capture is stamped at that capture's timestamp so those rows resolve to it. */ +export function finalFrameStamp( + commands: ReadonlyArray, + testStartMs: number, + lastCapturedTimestamp: number | undefined +): FinalFrameCommand | null { + const inTest = commands.filter((c) => c.timestamp >= testStartMs) + const last = inTest[inTest.length - 1] + if (!last) { + return null + } + if ( + lastCapturedTimestamp === undefined || + lastCapturedTimestamp < testStartMs + ) { + return { timestamp: last.timestamp, command: last.command } + } + if (!inTest.some((c) => c.timestamp > lastCapturedTimestamp)) { + return null + } + const atOrBefore = inTest.filter((c) => c.timestamp <= lastCapturedTimestamp) + const owner = atOrBefore[atOrBefore.length - 1] ?? inTest[0]! + return { timestamp: lastCapturedTimestamp, command: owner.command } +} diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 7338de7ef..b6865ec45 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -68,6 +68,7 @@ export * from './request-type.js' export * from './stack.js' export * from './terminal-throttle.js' export * from './error.js' +export * from './final-frame.js' export * from './finalize-screencast.js' export * from './input-dispatch.js' export * from './output-dir.js' diff --git a/packages/core/src/session-capturer.ts b/packages/core/src/session-capturer.ts index 685021657..7c0457d02 100644 --- a/packages/core/src/session-capturer.ts +++ b/packages/core/src/session-capturer.ts @@ -24,6 +24,7 @@ import { } from '@wdio/devtools-shared' import { resolveRunId } from './run-id.js' import { reattributeDomAnchors } from '@wdio/devtools-trace/trace-mutations' +import { TestCapture } from './test-capture.js' import { CONSOLE_METHODS, LOG_SOURCES, @@ -139,6 +140,13 @@ export abstract class SessionCapturerBase { return sessionHasDocument(this.metadata?.capabilities, this.currentContext) } + /** Per-test capture state for adapters that capture after each command. */ + readonly testCapture = new TestCapture() + + resetTestCapture(): void { + this.testCapture.reset() + } + // ── Construction ──────────────────────────────────────────────────────── constructor(opts: SessionCapturerOptions = {}) { const { hostname, port, reconnect } = opts diff --git a/packages/core/src/test-capture.ts b/packages/core/src/test-capture.ts new file mode 100644 index 000000000..d4f2a05aa --- /dev/null +++ b/packages/core/src/test-capture.ts @@ -0,0 +1,102 @@ +import type { ActionSnapshot, CommandLog } from '@wdio/devtools-shared' +import { mapCommandToAction } from '@wdio/devtools-shared' +import { + supersedeSnapshotAt, + upsertRichestSnapshot +} from '@wdio/devtools-trace/trace-snapshots' +import { + capturedPageState, + finalFrameStamp, + type FinalFrameCommand +} from './final-frame.js' + +/** + * Per-test state of the capture an adapter takes after each command. The + * adapter decides whether a command may capture and supplies the capture call; + * this owns the initial-frame retry, the generation guard and the end frame. + */ +export class TestCapture { + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + initialFrameRetried = false + generation = 0 + /** Timestamp of the latest command that queued a capture; the end-of-test frame is stamped there. */ + lastCapturedTimestamp: number | undefined = undefined + startWallTime = 0 + + reset(): void { + this.awaitingInitialFrame = true + this.initialFrameRetried = false + this.generation++ + this.startWallTime = Date.now() + } + + /** + * `changesPage` is the adapter's call on whether the command can move the + * page. Reads, waits and lookups skip the capture and inherit the earlier + * one, except the test's first mapped command, which always takes one. + */ + queue( + command: string, + timestamp: number, + changesPage: boolean, + snapshots: ActionSnapshot[], + pending: Promise[], + capture: () => Promise + ): Promise | undefined { + const initialFrame = + this.awaitingInitialFrame && !!mapCommandToAction(command) + if (!changesPage && !initialFrame) { + return undefined + } + if (initialFrame) { + this.awaitingInitialFrame = false + } + this.lastCapturedTimestamp = timestamp + const generation = this.generation + const settled = capture().then((snap) => { + if (snap) { + upsertRichestSnapshot(snapshots, snap) + } + // A capture resolving after reset() belongs to the previous test. + if ( + initialFrame && + !snap?.screenshot && + !this.initialFrameRetried && + generation === this.generation + ) { + this.initialFrameRetried = true + this.awaitingInitialFrame = true + } + }) + pending.push(settled) + return settled + } + + /** Stamped at the capture that last saw the page; supersedes its record only with a screenshot. */ + async captureEndFrame( + commands: ReadonlyArray, + pending: Promise[], + snapshots: ActionSnapshot[], + capture: (stamp: FinalFrameCommand) => Promise, + onError: (err: unknown) => void + ): Promise { + await Promise.all(pending) + const stamp = finalFrameStamp( + commands, + this.startWallTime, + this.lastCapturedTimestamp + ) + if (!stamp) { + return + } + try { + const snap = await capture(stamp) + if (snap && capturedPageState(snap)) { + supersedeSnapshotAt(snapshots, snap) + } + } catch (err) { + onError(err) + } + } +} diff --git a/packages/core/tests/final-frame.test.ts b/packages/core/tests/final-frame.test.ts new file mode 100644 index 000000000..ef71b3ce0 --- /dev/null +++ b/packages/core/tests/final-frame.test.ts @@ -0,0 +1,81 @@ +import { describe, it, expect } from 'vitest' +import { capturedPageState, finalFrameStamp } from '@wdio/devtools-core' + +const commands = [ + { command: 'url', timestamp: 90 }, + { command: 'click', timestamp: 120 }, + { command: 'getText', timestamp: 130 }, + { command: 'isDisplayed', timestamp: 140 } +] + +describe('finalFrameStamp', () => { + it('returns null when the test logged no rows', () => { + expect(finalFrameStamp([], 100, undefined)).toBeNull() + }) + + it('returns null when every row predates the test', () => { + expect(finalFrameStamp(commands, 200, 150)).toBeNull() + }) + + it('stamps the last row when no capture was taken in this test', () => { + expect(finalFrameStamp(commands, 100, 50)).toEqual({ + timestamp: 140, + command: 'isDisplayed' + }) + expect(finalFrameStamp(commands, 100, undefined)).toEqual({ + timestamp: 140, + command: 'isDisplayed' + }) + }) + + it('stamps the last capture when reads followed it', () => { + expect(finalFrameStamp(commands, 100, 120)).toEqual({ + timestamp: 120, + command: 'click' + }) + }) + + it('returns null when the test ends on the capturing command', () => { + expect(finalFrameStamp(commands, 100, 140)).toBeNull() + }) + + it('names the last row at or before the capture when no row sits on it', () => { + expect(finalFrameStamp(commands, 100, 125)).toEqual({ + timestamp: 125, + command: 'click' + }) + }) +}) + +describe('capturedPageState', () => { + it('holds for a capture that read a screenshot', () => { + expect( + capturedPageState({ timestamp: 1, command: 'click', screenshot: 'PNG' }) + ).toBe(true) + }) + + it('fails for a url-only capture, which has no image to pair the url with', () => { + expect( + capturedPageState({ timestamp: 1, command: 'click', url: 'https://a/' }) + ).toBe(false) + }) + + it('fails for a hollow capture from a session that had already ended', () => { + expect( + capturedPageState({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + }) + ).toBe(false) + expect( + capturedPageState({ + timestamp: 1, + command: 'click', + url: '', + screenshot: '' + }) + ).toBe(false) + }) +}) diff --git a/packages/core/tests/test-capture.test.ts b/packages/core/tests/test-capture.test.ts new file mode 100644 index 000000000..99e56eecb --- /dev/null +++ b/packages/core/tests/test-capture.test.ts @@ -0,0 +1,100 @@ +import { describe, it, expect } from 'vitest' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import { SessionCapturerBase } from '../src/session-capturer.js' + +class TestCapturer extends SessionCapturerBase { + readonly snapshots: ActionSnapshot[] = [] + readonly pending: Promise[] = [] + + queue( + command: string, + timestamp: number, + changesPage: boolean, + capture: () => Promise + ) { + return this.testCapture.queue( + command, + timestamp, + changesPage, + this.snapshots, + this.pending, + capture + ) + } +} + +const withScreenshot = (timestamp: number): Promise => + Promise.resolve({ timestamp, command: 'x', screenshot: 'png' }) +const withoutScreenshot = (timestamp: number): Promise => + Promise.resolve({ timestamp, command: 'x' }) + +function deferred() { + let resolve!: (value: T) => void + const promise = new Promise((r) => (resolve = r)) + return { promise, resolve } +} + +describe('TestCapture', () => { + it('clears the initial frame only on a capture that holds a screenshot', async () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withScreenshot(1)) + await Promise.all(capturer.pending) + + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + expect(capturer.snapshots.map((s) => s.timestamp)).toEqual([1]) + }) + + it('retries the initial frame once when its capture has no screenshot', async () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withoutScreenshot(1)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(true) + + capturer.queue('getText', 2, false, () => withoutScreenshot(2)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + + capturer.queue('getText', 3, false, () => withoutScreenshot(3)) + await Promise.all(capturer.pending) + expect(capturer.testCapture.awaitingInitialFrame).toBe(false) + }) + + it('returns no capture for a read that is not the initial frame', () => { + const capturer = new TestCapturer() + capturer.queue('getText', 1, false, () => withScreenshot(1)) + const skipped = capturer.queue('getText', 2, false, () => withScreenshot(2)) + + expect(skipped).toBeUndefined() + expect(capturer.testCapture.lastCapturedTimestamp).toBe(1) + }) + + it('ignores a capture that resolves after resetTestCapture()', async () => { + const capturer = new TestCapturer() + const stale = deferred() + capturer.queue('click', 1, true, () => stale.promise) + capturer.resetTestCapture() + + stale.resolve({ timestamp: 1, command: 'click' }) + await Promise.all(capturer.pending) + + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + }) + + it('resetTestCapture re-arms the initial frame and restamps the test start', () => { + const capturer = new TestCapturer() + capturer.queue('click', 1, true, () => withScreenshot(1)) + const generation = capturer.testCapture.generation + const before = Date.now() + + capturer.resetTestCapture() + + expect(capturer.testCapture.awaitingInitialFrame).toBe(true) + expect(capturer.testCapture.initialFrameRetried).toBe(false) + expect(capturer.testCapture.generation).toBe(generation + 1) + expect(capturer.testCapture.startWallTime).toBeGreaterThanOrEqual(before) + expect(capturer.testCapture.startWallTime).toBeLessThanOrEqual(Date.now()) + }) +}) diff --git a/packages/nightwatch-devtools/src/cucumber-lifecycle.ts b/packages/nightwatch-devtools/src/cucumber-lifecycle.ts index 7a6491d60..3d696523a 100644 --- a/packages/nightwatch-devtools/src/cucumber-lifecycle.ts +++ b/packages/nightwatch-devtools/src/cucumber-lifecycle.ts @@ -283,6 +283,7 @@ export async function captureCucumberScenarioBeforeQuit( ) } await ctx.sessionCapturer.captureTraceAtTestEnd(browser) + await ctx.sessionCapturer.captureFinalFrame() flushTestSlice(ctx) await ctx.emitTestArtifacts( scenario?.uid, diff --git a/packages/nightwatch-devtools/src/helpers/browserProxy.ts b/packages/nightwatch-devtools/src/helpers/browserProxy.ts index 6736f3f7d..320d052ad 100644 --- a/packages/nightwatch-devtools/src/helpers/browserProxy.ts +++ b/packages/nightwatch-devtools/src/helpers/browserProxy.ts @@ -280,7 +280,7 @@ export class BrowserProxy { log.info(`[command] ${methodName}`) this.attachScreenshot(browser, lastCommand, methodName) } - this.anchorDestination(browser, methodName) + this.anchorDestination(browser, methodName, lastCommand?.timestamp) // A DOM-mutating command already drains it through `captureTrace`. if (!this.isDomMutating(methodName)) { void this.sessionCapturer.nativeNetwork.forward({ throttle: true }) @@ -409,7 +409,8 @@ export class BrowserProxy { */ private anchorDestination( browser: NightwatchBrowser, - methodName: string + methodName: string, + timestamp: number | undefined ): void { if (!this.isDomMutating(methodName)) { return @@ -417,7 +418,9 @@ export class BrowserProxy { // Tracked so finalize awaits it — an untracked anchor drain can still be in // flight when the trace is written, which loses the document entirely. this.sessionCapturer.snapshotCaptures.push( - this.sessionCapturer.anchorAfterNavigation(browser).catch(() => {}) + this.sessionCapturer + .anchorAfterNavigation(browser, timestamp) + .catch(() => {}) ) } diff --git a/packages/nightwatch-devtools/src/index.ts b/packages/nightwatch-devtools/src/index.ts index cc3fde46a..bf4d0a24b 100644 --- a/packages/nightwatch-devtools/src/index.ts +++ b/packages/nightwatch-devtools/src/index.ts @@ -572,6 +572,7 @@ class NightwatchDevToolsPlugin { ) } await this.sessionCapturer.captureTraceAtTestEnd(browser) + await this.sessionCapturer.captureFinalFrame() // Flush this test's slice before the next test overwrites its outcome. flushTestSlice(this.#getInternals()) const results = (browser.currentTest as NightwatchCurrentTest)?.results diff --git a/packages/nightwatch-devtools/src/navigation-recapture.ts b/packages/nightwatch-devtools/src/navigation-recapture.ts new file mode 100644 index 000000000..89d130fd5 --- /dev/null +++ b/packages/nightwatch-devtools/src/navigation-recapture.ts @@ -0,0 +1,58 @@ +import logger from '@wdio/logger' +import { + capturedPageState, + errorMessage, + supersedeSnapshotAt +} from '@wdio/devtools-core' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import { captureActionSnapshot } from './action-snapshot.js' +import type { NightwatchBrowser, TestRunnerId } from './types.js' + +const log = logger('@wdio/nightwatch-devtools:navigationRecapture') + +/** A per-action capture a command queued, kept so a navigation seen after the command returns can supersede it once it has settled. */ +export interface QueuedPageCapture { + command: string + timestamp: number + settled: Promise +} + +export interface NavigationRecapture { + browser: NightwatchBrowser + runner: TestRunnerId + native: boolean + snapshots: ActionSnapshot[] + queued: QueuedPageCapture + /** False once a newer page capture has been queued; its own capture then reads the newer state. */ + isLatest: () => boolean +} + +/** + * A click resolves before its navigation commits, so the capture it queued can + * read the page it left. The replacement is taken after that capture settles, + * because a late upsert would otherwise merge the stale screenshot back over it. + */ +export async function recaptureAfterNavigation({ + browser, + runner, + native, + snapshots, + queued, + isLatest +}: NavigationRecapture): Promise { + try { + await queued.settled + const snap = await captureActionSnapshot( + browser, + queued.command, + queued.timestamp, + runner, + native + ) + if (snap && capturedPageState(snap) && isLatest()) { + supersedeSnapshotAt(snapshots, snap) + } + } catch (err) { + log.debug(`Post-navigation capture failed: ${errorMessage(err)}`) + } +} diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index 9d2913d16..ebc1d471b 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -7,19 +7,22 @@ import { drainCollectorWithRecovery, errorMessage, loadInjectableScript, - mapCommandToAction, NativeNetworkCapture, pollUntilReady, serializeError, - upsertRichestSnapshot, type CapturedPerformancePayload, type SessionCapturerOptions } from '@wdio/devtools-core' import { COLLECTOR_INJECT_SCRIPT, - COLLECTOR_READY_SCRIPT + COLLECTOR_READY_SCRIPT, + skipsCapture } from '@wdio/devtools-shared' import { captureActionSnapshot } from './action-snapshot.js' +import { + recaptureAfterNavigation, + type QueuedPageCapture +} from './navigation-recapture.js' import { nightwatchRunnerId } from './helpers/utils.js' import { NAVIGATION_COMMANDS } from './constants.js' import { @@ -49,6 +52,10 @@ const log = logger('@wdio/nightwatch-devtools:SessionCapturer') export class SessionCapturer extends SessionCapturerBase { #browser: NightwatchBrowser | undefined + /** The most recent page-changing capture; read synchronously when a navigation is detected for its command. */ + #lastPageCapture: QueuedPageCapture | undefined + /** Capture timestamps already re-captured or re-capturing; overlapping polls see the same replacement. */ + #lastRecapturedAt?: number // True once BiDi inspectors are attached — the per-command perf-log network // capture path skips when set, so we don't double-emit network requests. @@ -161,6 +168,7 @@ export class SessionCapturer extends SessionCapturerBase { if (!error) { this.#queueActionSnapshot( command, + args, commandLogEntry.timestamp, meta.parentId ) @@ -175,30 +183,37 @@ export class SessionCapturer extends SessionCapturerBase { * with the command timestamp (FrameSnapshotIndex.claimAfter / elementsAt). */ #queueActionSnapshot( command: string, + args: unknown[], timestamp: number, parentId: number | undefined ): void { - if ( - this.traceMode !== 'trace' || - !this.#browser || - !mapCommandToAction(command) - ) { + if (this.traceMode !== 'trace' || !this.#browser) { return } - this.snapshotCaptures.push( - captureActionSnapshot( - this.#browser, - command, - timestamp, - this.runner, - this.isNativeAppSession, - parentId === undefined ? 'action' : 'wait-poll' - ).then((snap) => { - if (snap) { - upsertRichestSnapshot(this.actionSnapshots, snap) - } - }) + // `url()` without a string argument is a getter; it does not navigate. + const pageChanging = + !skipsCapture(command) && + !(command === 'url' && typeof args[0] !== 'string') + const browser = this.#browser + const settled = this.testCapture.queue( + command, + timestamp, + pageChanging, + this.actionSnapshots, + this.snapshotCaptures, + () => + captureActionSnapshot( + browser, + command, + timestamp, + this.runner, + this.isNativeAppSession, + parentId === undefined ? 'action' : 'wait-poll' + ) ) + if (settled && pageChanging) { + this.#lastPageCapture = { command, timestamp, settled } + } } async #capturePerformanceData( @@ -364,7 +379,12 @@ export class SessionCapturer extends SessionCapturerBase { entry.error = serializeError(error) entry.timestamp = timestamp if (!error) { - this.#queueActionSnapshot(entry.command, timestamp, entry.parentId) + this.#queueActionSnapshot( + entry.command, + entry.args, + timestamp, + entry.parentId + ) } this.sendReplaceCommand(oldTimestamp, entry) } @@ -411,13 +431,17 @@ export class SessionCapturer extends SessionCapturerBase { * of a run anchors unconditionally rather than guessing — `captureCurrentDom` * is idempotent per document, so an already-anchored page costs one drain. */ - async anchorAfterNavigation(browser: NightwatchBrowser): Promise { + async anchorAfterNavigation( + browser: NightwatchBrowser, + timestamp?: number + ): Promise { // Polls the page for its own document identity, so there is nothing to // poll and nothing to anchor without one. if (this.isNativeAppSession) { return } const before = this.lastDocumentOrigin + let born: number | undefined const replaced = await pollUntilReady( async () => { const now = await this.documentOrigin(browser) @@ -425,6 +449,7 @@ export class SessionCapturer extends SessionCapturerBase { return false } this.lastDocumentOrigin = now + born = now return before === undefined || now !== before }, { attempts: 5, intervalMs: 150 } @@ -432,6 +457,32 @@ export class SessionCapturer extends SessionCapturerBase { if (!replaced) { return } + // Only a change from an origin this capturer already saw is a navigation; the + // first observation has no baseline to have changed from. The newest page + // capture is read here, after the poll: the click that caused the navigation + // may have queued its capture while this poll was running, and the fill that + // started the poll must never be re-stamped with the destination. + const queued = this.#lastPageCapture + let recapture: Promise = Promise.resolve() + if ( + timestamp !== undefined && + queued && + before !== undefined && + born !== undefined && + queued.timestamp >= timestamp && + queued.timestamp < born && + queued.timestamp !== this.#lastRecapturedAt + ) { + this.#lastRecapturedAt = queued.timestamp + recapture = recaptureAfterNavigation({ + browser, + runner: this.runner, + native: this.isNativeAppSession, + snapshots: this.actionSnapshots, + queued, + isLatest: () => this.#lastPageCapture === queued + }) + } // captureTrace's own recovery injects into the fresh document and re-drains, // and it gates that on the url being worth instrumenting — so a torn-down // session falls out here rather than logging an injection failure. @@ -449,7 +500,13 @@ export class SessionCapturer extends SessionCapturerBase { // the one that caused it. The guard exists for the case where that is a guess. // A wait sent running is newest from its start, but it has not ended, so it // did not cause the navigation. - await this.captureTrace(browser, true, this.#newestEndedTimestamp()) + const [drain] = await Promise.allSettled([ + this.captureTrace(browser, true, this.#newestEndedTimestamp()), + recapture + ]) + if (drain.status === 'rejected') { + log.debug(`Post-navigation drain failed: ${errorMessage(drain.reason)}`) + } } /** @@ -598,6 +655,33 @@ export class SessionCapturer extends SessionCapturerBase { await this.captureNetworkFromPerformanceLogs(browser) } + /** Rows after the last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ + async captureFinalFrame(): Promise { + if (this.traceMode !== 'trace') { + return + } + await this.testCapture.captureEndFrame( + this.commandsLog, + this.snapshotCaptures, + this.actionSnapshots, + async (stamp) => { + if (!this.#browser) { + return null + } + return captureActionSnapshot( + this.#browser, + stamp.command, + stamp.timestamp, + this.runner, + this.isNativeAppSession, + 'final' + ) + }, + (err) => + log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) + ) + } + /** At a test or scenario end, before its slice is written: the proxy drain there is never throttled. */ async captureTraceAtTestEnd(browser: NightwatchBrowser) { if (this.isNativeAppSession) { diff --git a/packages/nightwatch-devtools/src/test-lifecycle.ts b/packages/nightwatch-devtools/src/test-lifecycle.ts index 4c1817043..508071e41 100644 --- a/packages/nightwatch-devtools/src/test-lifecycle.ts +++ b/packages/nightwatch-devtools/src/test-lifecycle.ts @@ -170,6 +170,7 @@ export function wrapBrowserOnce( ctx.isScriptInjected = true } ctx.browserProxy.resetTestTracking() + ctx.sessionCapturer.resetTestCapture() ctx.browserProxy.wrapBrowserCommands(browser) } diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index bcf033688..c43862a03 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -1,7 +1,17 @@ +import type * as DevtoolsCore from '@wdio/devtools-core' import { describe, it, expect, vi } from 'vitest' +import { captureActionSnapshot as coreCapture } from '@wdio/devtools-core' import { SessionCapturer } from '../src/session.js' import type { CommandLog, NightwatchBrowser } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + function makeMockBrowser(): NightwatchBrowser { return { url: vi.fn(async () => ({})), @@ -59,6 +69,138 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) + it('captures a click, but not a read or wait that follows it', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await at('waitForElementVisible', 3) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + expect(cap.commandsLog).toHaveLength(3) + cap.cleanup() + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + cap.resetTestCapture() + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 3]) + cap.cleanup() + }) + + it('attempts the initial frame at most twice when no capture has a screenshot', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValue({ timestamp: 1, command: 'getText' }) + const cap = makeCapturer() + for (let t = 1; t <= 4; t++) { + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: t + }) + } + await Promise.all(cap.snapshotCaptures) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + vi.mocked(coreCapture).mockReset() + cap.cleanup() + }) + + it('retries the initial frame when the first capture resolves null', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce(null) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + const cap = makeCapturer() + for (let t = 1; t <= 3; t++) { + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: t + }) + } + await Promise.all(cap.snapshotCaptures) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + cap.cleanup() + }) + + it('retries the initial frame when the first read captures no screenshot', async () => { + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await Promise.all(cap.snapshotCaptures) + await at('getText', 2) + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + expect(cap.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) + cap.cleanup() + }) + + it('takes no snapshot for a url() read, but one for a url() navigation', async () => { + vi.mocked(coreCapture).mockClear() + const cap = makeCapturer() + const at = (command: string, args: unknown[], timestamp: number) => + cap.captureCommand(command, args, undefined, undefined, { timestamp }) + await at('click', ['#btn'], 1) + await at('url', [() => undefined], 2) + await at('url', [], 3) + await at('url', ['http://example.test/next'], 4) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 4]) + cap.cleanup() + }) + + it('does not re-arm the initial frame from a capture that resolves after the next test starts', async () => { + type Snap = Awaited> + let resolveFirst!: (snap: Snap) => void + vi.mocked(coreCapture) + .mockClear() + .mockImplementationOnce( + () => + new Promise((resolve) => { + resolveFirst = resolve + }) + ) + const cap = makeCapturer() + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: 1 + }) + cap.resetTestCapture() + resolveFirst({ timestamp: 1, command: 'getText' } as Snap) + await Promise.all(cap.snapshotCaptures) + + expect(cap.testCapture.initialFrameRetried).toBe(false) + expect(cap.testCapture.awaitingInitialFrame).toBe(true) + cap.cleanup() + }) + it('does not capture outside trace mode', async () => { const cap = new SessionCapturer({}, makeMockBrowser()) await cap.captureCommand('click', [], undefined, undefined, { @@ -68,4 +210,94 @@ describe('nightwatch action-snapshot timestamp binding', () => { expect(cap.actionSnapshots).toHaveLength(0) cap.cleanup() }) + + it('captures an unmapped command that can change the page, but not a lookup', async () => { + const cap = makeCapturer() + const at = (command: string, args: unknown[], timestamp: number) => + cap.captureCommand(command, args, undefined, undefined, { timestamp }) + await at('submitForm', ['#form'], 1) + await at('findElement', ['#el'], 2) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + cap.cleanup() + }) + + it('takes an end-of-test frame that replaces the last capture when reads follow it', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + type Snap = Awaited> + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'FINAL' + } as Snap) + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(vi.mocked(coreCapture).mock.calls[0]?.[0]).toMatchObject({ + trigger: 'final' + }) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]!.screenshot).toBe('FINAL') + cap.cleanup() + }) + + it('keeps the earlier capture when the end-of-test frame saw no page', async () => { + type Snap = Awaited> + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'GOOD', + elements: [{ selector: '#good' }] + } as Snap) + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + const before = { ...cap.actionSnapshots[0]! } + expect(before.screenshot).toBe('GOOD') + + vi.mocked(coreCapture) + .mockClear() + .mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + } as Snap) + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots).toEqual([before]) + cap.cleanup() + }) + + it('takes no end-of-test frame when the test ends on a capturing command', async () => { + const cap = makeCapturer() + await cap.captureCommand('getText', ['#el'], undefined, undefined, { + timestamp: 1 + }) + await cap.captureCommand('click', ['#el'], undefined, undefined, { + timestamp: 2 + }) + await Promise.all(cap.snapshotCaptures) + + vi.mocked(coreCapture).mockClear() + await cap.captureFinalFrame() + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + cap.cleanup() + }) }) diff --git a/packages/nightwatch-devtools/tests/navigationRecapture.test.ts b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts new file mode 100644 index 000000000..ca9c18ef6 --- /dev/null +++ b/packages/nightwatch-devtools/tests/navigationRecapture.test.ts @@ -0,0 +1,279 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' +import { captureActionSnapshot } from '../src/action-snapshot.js' +import { SessionCapturer } from '../src/session.js' +import { webdriverExecute } from '../src/helpers/webdriverHttp.js' +import type * as WebdriverHttp from '../src/helpers/webdriverHttp.js' +import type { ActionSnapshot } from '@wdio/devtools-shared' +import type { NightwatchBrowser } from '../src/types.js' + +vi.mock('../src/action-snapshot.js', () => ({ + captureActionSnapshot: vi.fn() +})) + +vi.mock('../src/helpers/webdriverHttp.js', async (importOriginal) => ({ + ...(await importOriginal()), + webdriverExecute: vi.fn() +})) + +const browser = { + sessionId: 'sess-recapture', + capabilities: {}, + options: { webdriver: { host: 'localhost', port: 4444 } } +} as unknown as NightwatchBrowser + +/** `performance.timeOrigin` reads, one per poll; the last origin repeats. */ +function originsRead(origins: number[]) { + let read = 0 + vi.mocked(webdriverExecute).mockImplementation(async () => { + return origins[Math.min(read++, origins.length - 1)] as never + }) +} + +function shot(fields: Partial): ActionSnapshot { + return { command: 'click', timestamp: 100, ...fields } as ActionSnapshot +} + +/** A click on /login whose capture read the page before the submit committed. */ +async function clickThenAnchor(cap: SessionCapturer) { + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 100 + }) + await cap.anchorAfterNavigation(browser, 100) + await Promise.all(cap.snapshotCaptures) +} + +describe('Nightwatch post-navigation re-capture', () => { + let cap: SessionCapturer + + beforeEach(() => { + vi.mocked(captureActionSnapshot).mockReset() + vi.spyOn(SessionCapturer.prototype, 'captureTrace').mockResolvedValue() + cap = new SessionCapturer({}, browser) + cap.traceMode = 'trace' + }) + + afterEach(() => { + cap.cleanup() + vi.restoreAllMocks() + }) + + it('supersedes the click capture with one taken once navigation is seen', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/secure', screenshot: 'NEW' }) + ) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(vi.mocked(captureActionSnapshot).mock.calls[1]?.[2]).toBe(100) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + timestamp: 100, + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + }) + + it('takes no re-capture when the queued capture already read the destination', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot).mockResolvedValueOnce( + shot({ + command: 'click', + timestamp: 2500, + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#next'], null, undefined, { + timestamp: 2500 + }) + await cap.anchorAfterNavigation(browser, 2500) + await Promise.all(cap.snapshotCaptures) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/secure', + screenshot: 'NEW' + }) + }) + + it('takes no extra capture when the document is not replaced', async () => { + originsRead([1000]) + vi.mocked(captureActionSnapshot).mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD' + }) + }) + + it('leaves the record when the re-capture holds no page state', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ + url: 'http://app.test/login', + screenshot: 'OLD', + snapshotText: 'old' + }) + ) + .mockResolvedValueOnce(shot({ snapshotText: 'blank' })) + + await cap.anchorAfterNavigation(browser) + await clickThenAnchor(cap) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots[0]).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD', + snapshotText: 'old' + }) + }) + + it('takes no re-capture for a command that queued none', async () => { + originsRead([1000, 2000]) + vi.mocked(captureActionSnapshot).mockResolvedValue(shot({})) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('getText', ['#flash'], 'x', undefined, { + timestamp: 100 + }) + const queued = vi.mocked(captureActionSnapshot).mock.calls.length + await cap.anchorAfterNavigation(browser, 100) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(queued) + }) + + it('attributes a navigation seen during a fill poll to the click that caused it', async () => { + let pageUrl = 'http://app.test/login' + let reads = 0 + vi.mocked(captureActionSnapshot).mockImplementation( + async (_browser, command, timestamp) => + shot({ command, timestamp, url: pageUrl, screenshot: pageUrl }) + ) + vi.mocked(webdriverExecute).mockImplementation(async () => { + reads++ + if (reads === 2) { + void cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 200 + }) + return 1000 as never + } + if (reads === 3) { + pageUrl = 'http://app.test/secure' + return 2000 as never + } + return 1000 as never + }) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('setValue', ['#username'], 'tomsmith', undefined, { + timestamp: 100 + }) + await cap.anchorAfterNavigation(browser, 100) + await Promise.all(cap.snapshotCaptures) + + expect( + vi + .mocked(captureActionSnapshot) + .mock.calls.map((call) => [call[1], call[2]]) + ).toEqual([ + ['setValue', 100], + ['click', 200], + ['click', 200] + ]) + expect(cap.actionSnapshots.find((s) => s.timestamp === 100)).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'http://app.test/login' + }) + expect(cap.actionSnapshots.find((s) => s.timestamp === 200)).toMatchObject({ + url: 'http://app.test/secure', + screenshot: 'http://app.test/secure' + }) + }) + + it('discards a re-capture superseded by a page capture queued while it was in flight', async () => { + originsRead([1000, 2000]) + let finishRecapture!: (snap: ActionSnapshot) => void + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce( + shot({ url: 'http://app.test/login', screenshot: 'OLD' }) + ) + .mockImplementationOnce( + () => new Promise((resolve) => (finishRecapture = resolve)) + ) + .mockResolvedValueOnce( + shot({ + command: 'click', + timestamp: 150, + url: 'http://app.test/secure', + screenshot: 'NEXT' + }) + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 100 + }) + const anchored = cap.anchorAfterNavigation(browser, 100) + await vi.waitFor(() => + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + ) + await cap.captureCommand('click', ['#next'], null, undefined, { + timestamp: 150 + }) + finishRecapture( + shot({ url: 'http://app.test/secure', screenshot: 'STALE' }) + ) + await anchored + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.find((s) => s.timestamp === 100)).toMatchObject({ + url: 'http://app.test/login', + screenshot: 'OLD' + }) + }) + + it('re-captures once when overlapping polls detect the same replacement', async () => { + let pageUrl = 'http://app.test/login' + vi.mocked(captureActionSnapshot).mockImplementation( + async (_browser, command, timestamp) => + shot({ command, timestamp, url: pageUrl, screenshot: pageUrl }) + ) + vi.mocked(webdriverExecute).mockImplementation( + async () => (pageUrl === 'http://app.test/login' ? 1000 : 2000) as never + ) + + await cap.anchorAfterNavigation(browser) + await cap.captureCommand('click', ['#submit'], null, undefined, { + timestamp: 200 + }) + pageUrl = 'http://app.test/secure' + await Promise.all([ + cap.anchorAfterNavigation(browser, 150), + cap.anchorAfterNavigation(browser, 180) + ]) + await Promise.all(cap.snapshotCaptures) + + expect(captureActionSnapshot).toHaveBeenCalledTimes(2) + expect(cap.actionSnapshots).toHaveLength(1) + expect(cap.actionSnapshots[0]).toMatchObject({ + timestamp: 200, + url: 'http://app.test/secure' + }) + }) +}) diff --git a/packages/selenium-devtools-py/changes/page-changing-capture.md b/packages/selenium-devtools-py/changes/page-changing-capture.md new file mode 100644 index 000000000..d0c6caabf --- /dev/null +++ b/packages/selenium-devtools-py/changes/page-changing-capture.md @@ -0,0 +1,5 @@ +--- +patch +--- + +Trace mode reads each page's element tree after every command except known reads, waits and lookups, instead of only after a fixed list of page-changing commands, so a script run through the driver or an `actions` chain shows the page it changed. When reads follow a test's last capture, the test's end takes one more capture, stamped at that last capture and superseding it; the capture adds no command row. Reads and waits keep their own rows and show the tree from the last capture before them. Every command that returns still takes its screenshot, so a read saves only its element-tree read. diff --git a/packages/selenium-devtools-py/scripts/gen_contract.py b/packages/selenium-devtools-py/scripts/gen_contract.py index f3d6af633..6639a5eba 100644 --- a/packages/selenium-devtools-py/scripts/gen_contract.py +++ b/packages/selenium-devtools-py/scripts/gen_contract.py @@ -203,6 +203,27 @@ def _runner_cwd_env(runner_ts: str) -> str: return m.group(1) +def _action_commands(schema_path: Path) -> dict[str, list[str]]: + """The runner-native command names the capture gate reads, evaluated from + shared's ``skipsCapture`` by ``wire-schema.ts`` rather than restated here. + + Generated because a name that drifts is silent: a read missing from the + read-only set is captured on every poll, and a page change missing from it + is never captured, so its row shows the page before it. + """ + if not schema_path.exists(): + raise SystemExit( + f"{schema_path} is missing: run `npx tsx scripts/wire-schema.ts` in packages/shared" + ) + sets = json.loads(schema_path.read_text()).get("actionCommands") + if not sets or "readOnly" not in sets or "mapped" not in sets: + raise SystemExit( + "contract drift: `actionCommands` is missing from wire-schema.json; " + "regenerate it with packages/shared/scripts/wire-schema.ts" + ) + return sets + + def _test_runner_ids(types_ts: str) -> list[str]: m = re.search(r"export const TEST_RUNNER_IDS = \[(.*?)\] as const", types_ts, re.DOTALL) if not m: @@ -335,6 +356,7 @@ def main() -> int: rerun_slot = _rerun_slot(runner_ts) runner_cwd_env = _runner_cwd_env(runner_ts) reuse_env = _reuse_env(runner_ts) + action_commands = _action_commands(shared / "wire-schema.json") # Drift-guard. missing = [v for v in REQUIRED_DATA_SCOPES.values() if v not in data_keys] @@ -409,6 +431,8 @@ def main() -> int: lines += [ "", f"DATA_SCOPES = frozenset({sorted(data_keys)!r})", + f"READ_ONLY_COMMANDS = frozenset({action_commands['readOnly']!r})", + f"MAPPED_COMMANDS = frozenset({action_commands['mapped']!r})", f"CONTROL_SCOPES = frozenset({sorted(control.values())!r})", "", f'COLLECTOR_PATH = "{collector_path}"', diff --git a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py index 3023e6435..02a07429c 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/_contract.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/_contract.py @@ -15,6 +15,8 @@ SCOPE_REPLACE_COMMAND = "replaceCommand" DATA_SCOPES = frozenset(['actionSnapshots', 'commands', 'config', 'consoleLogs', 'logs', 'metadata', 'mutations', 'networkRequests', 'screencast', 'screencastFrames', 'sources', 'suites']) +READ_ONLY_COMMANDS = frozenset(['$', '$$', 'element', 'elementScreenshot', 'elements', 'findChildElement', 'findChildElements', 'findElement', 'findElementFromShadowRoot', 'findElements', 'findElementsFromShadowRoot', 'getAccessibleName', 'getActiveElement', 'getAlertText', 'getAllCookies', 'getAllWindowHandles', 'getAriaRole', 'getAttribute', 'getAvailableLogTypes', 'getCSSProperty', 'getCapabilities', 'getComputedLabel', 'getComputedRole', 'getContextHandles', 'getCookie', 'getCookies', 'getCredentials', 'getCssProperty', 'getCssValue', 'getCurrentContextHandle', 'getCurrentUrl', 'getDomAttribute', 'getDownloadableFiles', 'getElementAriaLabel', 'getElementAriaRole', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementShadowRoot', 'getElementSize', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getFirstElementChild', 'getHTML', 'getId', 'getLastElementChild', 'getLocation', 'getLog', 'getLogTypes', 'getNetworkConnection', 'getNextSibling', 'getPageSource', 'getPreviousSibling', 'getProperty', 'getRect', 'getScreenOrientation', 'getSession', 'getShadowRoot', 'getSize', 'getTagName', 'getText', 'getTimeouts', 'getTitle', 'getUrl', 'getValue', 'getWindowHandle', 'getWindowHandles', 'getWindowPosition', 'getWindowRect', 'getWindowSize', 'hasDescendants', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isLogAvailable', 'isPresent', 'isSelected', 'isVisible', 'printPage', 'screenshot', 'source', 'status', 'takeScreenshot', 'title', 'w3cGetActiveElement', 'w3cGetAlertText', 'w3cGetCurrentWindowHandle', 'w3cGetWindowHandles', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil', 'windowHandle', 'windowHandles']) +MAPPED_COMMANDS = frozenset(['addValue', 'back', 'clear', 'clearElement', 'click', 'clickElement', 'doubleClick', 'dragAndDrop', 'execute', 'executeAsync', 'forward', 'get', 'getAttribute', 'getCSSProperty', 'getCssValue', 'getCurrentUrl', 'getElementAttribute', 'getElementProperty', 'getElementRect', 'getElementTagName', 'getElementText', 'getElementValueOfCssProperty', 'getLocation', 'getPageSource', 'getProperty', 'getRect', 'getSize', 'getTagName', 'getText', 'getTitle', 'getUrl', 'getValue', 'goBack', 'goForward', 'isClickable', 'isDisplayed', 'isElementEnabled', 'isElementSelected', 'isEnabled', 'isExisting', 'isFocused', 'isSelected', 'keys', 'moveTo', 'navigateTo', 'newWindow', 'refresh', 'scrollIntoView', 'selectByVisibleText', 'sendKeys', 'sendKeysToElement', 'setValue', 'submit', 'switchToFrame', 'switchToParentFrame', 'switchToWindow', 'to', 'touchAction', 'url', 'wait', 'waitForClickable', 'waitForDisplayed', 'waitForElementLocated', 'waitForElementNotPresent', 'waitForElementNotVisible', 'waitForElementPresent', 'waitForElementVisible', 'waitForEnabled', 'waitForExist', 'waitUntil']) CONTROL_SCOPES = frozenset(['clearCommands', 'clearExecutionData', 'clientConnected', 'clientDisconnected', 'config', 'replaceCommand', 'testStopped']) COLLECTOR_PATH = "/api/collector" diff --git a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py index aa1fe69cd..6c19517b5 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py @@ -31,6 +31,7 @@ read_value_locators, wait_tracking, ) +from ._contract import MAPPED_COMMANDS, READ_ONLY_COMMANDS from .assert_tracer import ScriptAssertionTracer from .capturer import SessionCapturer from .cdp_screencast import start_push_screencast @@ -38,6 +39,7 @@ from .device import driver_is_native_app, restore_browser_focus from .element_scripts import reset_cache as reset_element_scripts_cache from .page_scripts import reset_cache as reset_page_scripts_cache +from .per_test_capture import PerTestCapture from .constants import ( BIDI_CAPABILITY, DEFAULT_TEST_TITLE, @@ -289,6 +291,8 @@ def _capture_source(capturer: SessionCapturer, call_src: Optional[str]) -> None: # is the only moment they are reachable. Run-scoped: a run may replace its # driver, and the filmstrip is the whole run. "filmstrip_frames": [], + # Re-armed per test by reset_test_capture(). + "test": PerTestCapture(), } @@ -422,6 +426,28 @@ def resolved_output_dir() -> Optional[str]: return _state.get("output_dir") +# The atoms Selenium's own WebElement methods run as scripts, tagged with a +# leading comment; the command each one stands for. +_ATOM_SCRIPT_COMMANDS = { + "/* submitForm */": "submit", + "/* getAttribute */": "getElementAttribute", + "/* isDisplayed */": "isDisplayed", +} + + +def _action_capture_enabled(driver: Any) -> bool: + if not _state["trace"] or not _state["a11y"] or not _state.get("element_scripts"): + return False + return not driver_is_native_app(driver) + + +def _action_capture_due(driver: Any, command: str) -> bool: + initial = _state["test"].awaiting_initial_frame and command in MAPPED_COMMANDS + if command in READ_ONLY_COMMANDS and not initial: + return False + return _action_capture_enabled(driver) + + def _capture_timer( driver: Any, command: str, poll: Any ) -> Optional[capture_timing.CaptureTimer]: @@ -438,7 +464,7 @@ def _capture_action_snapshot( timestamp: int, shot: Optional[str], timer: Optional[capture_timing.CaptureTimer] = None, -) -> None: +) -> Optional[ActionSnapshot]: """Read the page's element tree beside one action, for the trace's A11y tab. Trace mode only, and only with the scripts in hand — the JS adapters run @@ -452,13 +478,9 @@ def _capture_action_snapshot( Skipped entirely on a native app: both reads are page script, they run on every action, and neither can do anything but fail there. """ - if not _state["trace"] or not _state["a11y"]: - return - if driver_is_native_app(driver): - return + if not _action_capture_enabled(driver): + return None scripts = _state.get("element_scripts") - if not scripts: - return run = _guarded_execute_script(driver) snapshot: ActionSnapshot = {"timestamp": timestamp, "command": command} if shot: @@ -483,8 +505,136 @@ def _capture_action_snapshot( if isinstance(value, list) and value: snapshot[key] = value if "elements" not in snapshot and "accessibilityTree" not in snapshot: - return + return None _state["action_snapshots"].append(snapshot) + return snapshot + + +def _maybe_capture_action_snapshot( + driver: Any, + command: str, + timestamp: int, + shot: Optional[str], + timer: Optional[capture_timing.CaptureTimer] = None, +) -> None: + """Capture on every command that is not a known read, and on a test's first + mapped command as its initial frame. Reads and waits inherit the capture + before them.""" + initial = _state["test"].awaiting_initial_frame and command in MAPPED_COMMANDS + _note_final_frame_driver(driver) + if not _action_capture_due(driver, command): + return + if initial: + _state["test"].awaiting_initial_frame = False + _state["test"].last_captured_ts = timestamp + _state["test"].final_frame_driver = weakref.ref(driver) + snapshot = _capture_action_snapshot(driver, command, timestamp, shot, timer) + if not initial or _state["test"].initial_frame_retried: + return + if snapshot is None or "screenshot" not in snapshot: + _state["test"].initial_frame_retried = True + _state["test"].awaiting_initial_frame = True + + +def _capture_command(driver_command: str, params: Any) -> str: + """The command a script-run atom stands for: Selenium's `WebElement` methods + reach `execute` as marked scripts, not as the command they perform.""" + script = params.get("script") if isinstance(params, dict) else None + if isinstance(script, str): + for marker, command in _ATOM_SCRIPT_COMMANDS.items(): + if script.startswith(marker): + return command + return driver_command + + +def _final_frame_stamp( + rows: List[tuple], test_started_ms: int, last_captured_ts: Optional[int] +) -> Optional[tuple]: + """Mirrors core's `finalFrameStamp`: the (command, timestamp) the end-of-test + capture replaces, or None when no row follows the last capture.""" + in_test = [row for row in rows if row[1] >= test_started_ms] + if not in_test: + return None + if last_captured_ts is None or last_captured_ts < test_started_ms: + return in_test[-1] + if not any(ts > last_captured_ts for _, ts in in_test): + return None + at_or_before = [row for row in in_test if row[1] <= last_captured_ts] + owner = at_or_before[-1] if at_or_before else in_test[0] + return (owner[0], last_captured_ts) + + +def _note_final_frame_driver(driver: Any) -> None: + """Until a capture is taken, the test's stamp belongs to the driver that ran its latest row.""" + if _state["test"].last_captured_ts is None: + _state["test"].final_frame_driver = weakref.ref(driver) + + +def _capture_final_frame(driver: Any) -> bool: + if not _action_capture_enabled(driver): + return False + stamp = _final_frame_stamp( + _state["test"].rows, _state["test"].started_ms, _state["test"].last_captured_ts + ) + if stamp is None: + return False + command, timestamp = stamp + timer = ( + capture_timing.CaptureTimer(command, "final", driver_is_native_app(driver)) + if capture_timing.enabled() + else None + ) + try: + # Frames are screenshot-backed: panes merged over an earlier image would pair two documents. + shot = timer.read("screenshot", _take_screenshot, driver) if timer else _take_screenshot(driver) + if not shot: + return False + snapshot = _capture_action_snapshot(driver, command, timestamp, shot, timer) or { + "timestamp": timestamp, + "command": command, + "screenshot": shot, + } + superseded: dict = {} + for kept in _state["action_snapshots"]: + if kept["timestamp"] == timestamp and kept is not snapshot: + superseded.update(kept) + _state["action_snapshots"][:] = [ + kept for kept in _state["action_snapshots"] if kept["timestamp"] != timestamp + ] + _state["action_snapshots"].append({**superseded, **snapshot}) + _state["test"].last_captured_ts = timestamp + return True + finally: + if timer: + timer.finish() + + +def capture_final_frame(driver: Any = None) -> None: + """Replace the test's last capture with the page the test ended on, when + reads followed it. + + The last page-changing command can return before its navigation commits, + so its capture shows the page it left; the reads after it saw the new one. + Once per test: the call report and the driver's quit both reach here. Only + the driver that owns the stamp acts, so another session quitting mid-test + neither captures from nor consumes the test's one final frame. + """ + if _state["test"].final_frame_taken: + return + owner_ref = _state["test"].final_frame_driver + owner = owner_ref() if owner_ref is not None else None + if owner is None or (driver is not None and driver is not owner): + return + try: + if _capture_final_frame(owner): + _state["test"].final_frame_taken = True + except Exception as exc: # noqa: BLE001 — a missing end frame, not a failed run + _log.debug("final frame capture failed: %s", exc) + + +def reset_test_capture() -> None: + """Re-arm the initial frame and start the row ledger for the test about to start.""" + _state["test"].reset(now_ms()) def _begin_screencast_run(entry: Optional[dict], shot: Optional[str] = None) -> None: @@ -1032,6 +1182,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN # Finalize the screencast BEFORE quit tears the session down — after # orig_execute the driver's WebSocket/session is gone. if driver_command == "quit": + capture_final_frame(self) _on_quit(capturer, self) elif driver_command == "newSession": _enable_bidi_capability(params) # request BiDi before the session opens @@ -1060,6 +1211,10 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, failed) + _note_final_frame_driver(self) + _state["test"].rows.append( + (_capture_command(driver_command, params), failed["timestamp"]) + ) raise # WebDriver.execute returns the full response dict; the useful payload @@ -1068,7 +1223,8 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN # One screenshot per command on this (main) thread — attached to the # command (so selecting it shows the page) AND reused as a screencast # frame, so we pay for only a single screenshot round-trip either way. - timer = _capture_timer(self, driver_command, poll) + command = _capture_command(driver_command, params) + timer = _capture_timer(self, command, poll) shot = timer.read("screenshot", _take_screenshot, self) if timer else _take_screenshot(self) selector = element_locators.selector_for_command(driver_command, params, value) _remember_read_value(driver_command, params, value, selector) @@ -1084,7 +1240,9 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN parent_id=wait_tracking.parent_of(poll), ) wait_tracking.note_poll_row(poll, row) - _capture_action_snapshot(self, driver_command, row.get("timestamp", start), shot, timer) + stamp = row.get("timestamp", start) + _state["test"].rows.append((command, stamp)) + _maybe_capture_action_snapshot(self, command, stamp, shot, timer) if timer: timer.finish() if driver_command in NAVIGATION_COMMANDS: @@ -1116,6 +1274,7 @@ def patched_execute(self, driver_command: str, params: Any = None): # noqa: ANN trace=trace, filmstrip=filmstrip, filmstrip_frames=[], a11y=a11y, element_scripts=None, page_scripts=None, action_snapshots=[], ) + _state["test"].reset(now_ms()) def _executor_host(driver: Any) -> Optional[str]: diff --git a/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py b/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py new file mode 100644 index 000000000..76b7ecc3a --- /dev/null +++ b/packages/selenium-devtools-py/src/selenium_devtools/per_test_capture.py @@ -0,0 +1,29 @@ +"""The state one test's capture is decided from: its initial frame and its end frame.""" + +from __future__ import annotations + +import weakref +from dataclasses import dataclass, field +from typing import List, Optional + + +@dataclass +class PerTestCapture: + awaiting_initial_frame: bool = True + initial_frame_retried: bool = False + # The test's command rows as (command, timestamp). + rows: List[tuple] = field(default_factory=list) + started_ms: int = 0 + # The timestamp of the last capture attempted. + last_captured_ts: Optional[int] = None + final_frame_driver: Optional[weakref.ref] = None + final_frame_taken: bool = False + + def reset(self, started_ms: int) -> None: + self.awaiting_initial_frame = True + self.initial_frame_retried = False + self.rows = [] + self.started_ms = started_ms + self.last_captured_ts = None + self.final_frame_driver = None + self.final_frame_taken = False diff --git a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py index b89662d01..b4191cca1 100644 --- a/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py +++ b/packages/selenium-devtools-py/src/selenium_devtools/pytest_plugin.py @@ -15,7 +15,7 @@ from typing import Dict, Optional import selenium_devtools as devtools -from . import assertions, frames, read_value_locators, rerun +from . import assertions, frames, instrumentation, read_value_locators, rerun from .constants import ENV_OPT_IN, ENV_PORT, LOGGER_NAME from .utils import now_ms @@ -589,6 +589,7 @@ def pytest_runtest_logstart(nodeid, location) -> None: # noqa: ANN001 return _registry.mark_start(nodeid) read_value_locators.reset() + instrumentation.reset_test_capture() capturer = devtools.get_capturer() if capturer is None: return @@ -643,6 +644,8 @@ def pytest_runtest_logreport(report) -> None: # noqa: ANN001 return from . import instrumentation + if report.when == "call": + instrumentation.capture_final_frame() instrumentation.drain_native_network(capturer) file, line, name = report.location _registry.record( diff --git a/packages/selenium-devtools-py/tests/test_a11y_elements.py b/packages/selenium-devtools-py/tests/test_a11y_elements.py index ad0a9e47f..ecad43f3c 100644 --- a/packages/selenium-devtools-py/tests/test_a11y_elements.py +++ b/packages/selenium-devtools-py/tests/test_a11y_elements.py @@ -6,13 +6,17 @@ same flow carried 12; this is the gap. """ +import contextlib import json import unittest from unittest import mock from selenium_devtools import element_scripts, instrumentation, trace_export +from selenium_devtools.per_test_capture import PerTestCapture from selenium_devtools._contract import ( ELEMENT_SCRIPTS_PATH, + MAPPED_COMMANDS, + READ_ONLY_COMMANDS, RUNNER_ID, SCOPE_ACTION_SNAPSHOTS, ) @@ -217,6 +221,277 @@ def test_an_empty_or_failed_read_records_no_snapshot(self): self.assertEqual(self._capture(self.Driver(elements=None)), []) self.assertEqual(self._capture(self.Driver(raises=True)), []) + @contextlib.contextmanager + def _armed(self, driver=None): + with mock.patch.dict( + instrumentation._state, + { + "trace": True, + "a11y": True, + "element_scripts": SCRIPTS, + "action_snapshots": [], + "test": PerTestCapture(), + }, + ): + yield driver or self.Driver(elements=[{"selector": "#go"}]) + + def _gate(self, driver, command, shot="c2hvdA=="): + instrumentation._maybe_capture_action_snapshot(driver, command, 1200, shot) + + def _captured(self): + return [snap["command"] for snap in instrumentation.action_snapshots()] + + def test_a_click_is_captured_and_the_reads_after_it_are_not(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "getElementText") + self._gate(driver, "wait") + self.assertEqual(self._captured(), ["clickElement"]) + self.assertEqual(len(driver.scripts_run), 2, "read the page for a read") + + def test_a_leading_read_is_the_initial_frame_once_until_the_next_test(self): + with self._armed() as driver: + self._gate(driver, "getElementText") + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["getElementText"]) + + instrumentation.reset_test_capture() + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["getElementText", "getElementText"]) + + def test_a_missing_initial_screenshot_is_retried_exactly_once(self): + with self._armed() as driver: + self._gate(driver, "getElementText", shot=None) + self._gate(driver, "getElementText", shot="c2hvdA==") + self._gate(driver, "getElementText", shot="c2hvdA==") + snaps = instrumentation.action_snapshots() + self.assertEqual(len(snaps), 2) + self.assertNotIn("screenshot", snaps[0]) + self.assertIn("screenshot", snaps[1]) + + def test_a_form_submit_is_captured_as_a_page_change(self): + submit = {"script": "/* submitForm */var form = arguments[0];", "args": []} + other = {"script": "return 1", "args": []} + self.assertEqual( + instrumentation._capture_command("w3cExecuteScript", submit), "submit" + ) + self.assertEqual( + instrumentation._capture_command("w3cExecuteScript", other), + "w3cExecuteScript", + ) + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", submit)) + self.assertEqual(self._captured(), ["clickElement", "submit"]) + + def test_a_second_missing_initial_screenshot_is_not_retried_again(self): + with self._armed() as driver: + self._gate(driver, "clickElement", shot=None) + self._gate(driver, "getElementText", shot=None) + self._gate(driver, "getElementText", shot="c2hvdA==") + self.assertEqual(self._captured(), ["clickElement", "getElementText"]) + + def test_the_initial_frame_waits_for_a_capture_that_can_run(self): + class NativeDriver(self.Driver): + capabilities = {"platformName": "Android", "appium:app": "/app.apk"} + + with self._armed(NativeDriver(elements=[{"selector": "#go"}])) as driver: + self._gate(driver, "getElementText") + self.assertTrue(instrumentation._state["test"].awaiting_initial_frame) + self.assertEqual(driver.scripts_run, []) + + # Deny-list: a command the set does not name is captured, so only commands + # known not to change the page may be listed. + def test_the_generated_sets_agree_with_the_gate(self): + self.assertIn("findElement", READ_ONLY_COMMANDS) + self.assertIn("getElementText", READ_ONLY_COMMANDS) + self.assertIn("isDisplayed", READ_ONLY_COMMANDS) + self.assertIn("getElementAttribute", READ_ONLY_COMMANDS) + self.assertTrue( + READ_ONLY_COMMANDS.isdisjoint({"get", "clickElement", "submit", "actions", "w3cExecuteScript"}) + ) + self.assertIn("getElementText", MAPPED_COMMANDS) + self.assertNotIn("findElement", MAPPED_COMMANDS) + + def test_a_script_the_user_runs_is_a_page_change_and_captured(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "w3cExecuteScript") + self._gate(driver, "actions") + self.assertEqual(self._captured(), ["clickElement", "w3cExecuteScript", "actions"]) + + def test_a_lookup_and_a_text_read_after_a_click_are_not_captured(self): + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, "findElement") + self._gate(driver, "getElementText") + self.assertEqual(self._captured(), ["clickElement"]) + + def test_selenium_atom_scripts_are_read_as_the_reads_they_are(self): + is_displayed = {"script": "/* isDisplayed */return (function(){}).apply(null, arguments);", "args": []} + get_attribute = {"script": "/* getAttribute */return (function(){}).apply(null, arguments);", "args": []} + self.assertEqual(instrumentation._capture_command("w3cExecuteScript", is_displayed), "isDisplayed") + self.assertEqual(instrumentation._capture_command("w3cExecuteScript", get_attribute), "getElementAttribute") + with self._armed() as driver: + self._gate(driver, "clickElement") + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", is_displayed)) + self._gate(driver, instrumentation._capture_command("w3cExecuteScript", get_attribute)) + self.assertEqual(self._captured(), ["clickElement"]) + self.assertEqual(len(driver.scripts_run), 2, "read the page for an atom read") + + +class TestTheEndOfATest(unittest.TestCase): + """The last page change can return before its navigation commits, so its + capture shows the page it left. The reads after it saw the new page, and + the end of the test is captured again to replace that stale capture.""" + + class Driver: + session_id = "s" + + def __init__(self): + self.tree = [{"selector": "#before"}] + self.scripts_run = 0 + + def get_screenshot_as_base64(self): + return "END" + + def execute_script(self, script, *args): + self.scripts_run += 1 + return self.tree + + @contextlib.contextmanager + def _ended(self): + with mock.patch.dict( + instrumentation._state, + { + "trace": True, + "a11y": True, + "element_scripts": SCRIPTS, + "action_snapshots": [], + "test": PerTestCapture(awaiting_initial_frame=False, started_ms=1000), + }, + ): + yield + + def _step(self, driver, command, timestamp): + instrumentation._state["test"].rows.append((command, timestamp)) + instrumentation._maybe_capture_action_snapshot(driver, command, timestamp, None) + + def test_the_reads_after_a_click_replace_its_stale_capture(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + driver.tree = [{"selector": "#after"}] + self._step(driver, "getElementText", 1300) + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([s["command"] for s in snaps], ["clickElement"]) + self.assertEqual(snaps[0]["timestamp"], 1200) + self.assertEqual(snaps[0]["elements"][0]["selector"], "#after") + + def test_a_test_that_ends_on_a_capture_takes_no_final_one(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + reads = driver.scripts_run + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, reads, "re-read a page nothing followed") + self.assertEqual(len(instrumentation.action_snapshots()), 1) + + def test_a_test_with_no_capture_is_stamped_at_its_last_command(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "findElement", 1200) + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([(s["command"], s["timestamp"]) for s in snaps], [("findElement", 1200)]) + + def test_a_test_with_no_rows_takes_no_final_capture(self): + driver = self.Driver() + with self._ended(): + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, 0) + self.assertEqual(instrumentation.action_snapshots(), []) + + def test_the_final_capture_is_taken_once_per_test(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "clickElement", 1200) + self._step(driver, "getElementText", 1300) + instrumentation.capture_final_frame(driver) + reads = driver.scripts_run + instrumentation.capture_final_frame(driver) + self.assertEqual(driver.scripts_run, reads, "the quit repeated the call report's capture") + + def test_a_final_capture_without_a_screenshot_leaves_the_record_unchanged(self): + driver = self.Driver() + with self._ended(): + instrumentation._state["test"].rows.append(("clickElement", 1200)) + instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") + before = dict(instrumentation.action_snapshots()[0]) + driver.tree = [{"selector": "#after"}] + self._step(driver, "getElementText", 1300) + with mock.patch.object(instrumentation, "_take_screenshot", return_value=None): + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual(snaps, [before]) + + def test_a_screenshot_only_final_capture_supersedes_the_screenshot(self): + driver = self.Driver() + with self._ended(): + instrumentation._state["test"].rows.append(("clickElement", 1200)) + instrumentation._maybe_capture_action_snapshot(driver, "clickElement", 1200, "SHOT") + driver.tree = [] + self._step(driver, "getElementText", 1300) + with mock.patch.object(instrumentation, "_take_screenshot", return_value="END"): + instrumentation.capture_final_frame(driver) + snaps = instrumentation.action_snapshots() + self.assertEqual([(s["timestamp"], s["screenshot"]) for s in snaps], [(1200, "END")]) + self.assertEqual(snaps[0]["elements"][0]["selector"], "#before") + + def test_the_final_frame_comes_from_the_driver_that_captured_last(self): + owner, other = self.Driver(), self.Driver() + with self._ended(): + self._step(owner, "clickElement", 1200) + before = owner.scripts_run + self._step(other, "getElementText", 1300) + instrumentation.capture_final_frame() + self.assertEqual(owner.scripts_run, before + 2, "the owner was not re-read") + self.assertEqual(other.scripts_run, 0, "a read on another driver took the frame") + + def test_another_drivers_quit_leaves_the_final_frame_to_the_owner(self): + owner, other = self.Driver(), self.Driver() + with self._ended(): + self._step(owner, "clickElement", 1200) + self._step(other, "getElementText", 1300) + instrumentation.capture_final_frame(other) + self.assertFalse(instrumentation._state["test"].final_frame_taken) + before = owner.scripts_run + instrumentation.capture_final_frame(owner) + self.assertEqual(owner.scripts_run, before + 2) + self.assertTrue(instrumentation._state["test"].final_frame_taken) + + def test_window_and_aria_reads_take_no_capture(self): + driver = self.Driver() + with self._ended(): + self._step(driver, "w3cGetCurrentWindowHandle", 1200) + self._step(driver, "getElementAriaRole", 1300) + self.assertEqual(driver.scripts_run, 0) + self.assertEqual(instrumentation.action_snapshots(), []) + + def test_a_row_before_the_test_started_does_not_stamp_it(self): + self.assertIsNone( + instrumentation._final_frame_stamp([("clickElement", 500)], 1000, None) + ) + + def test_the_stamp_follows_core_final_frame_stamp(self): + stamp = instrumentation._final_frame_stamp + rows = [("clickElement", 1200), ("getElementText", 1300)] + self.assertEqual(stamp(rows, 1000, None), ("getElementText", 1300)) + self.assertEqual(stamp(rows, 1000, 1200), ("clickElement", 1200)) + self.assertIsNone(stamp(rows[:1], 1000, 1200)) + self.assertIsNone(stamp([], 1000, None)) + class TestStreamingTheSnapshots(unittest.TestCase): def test_they_go_out_under_the_action_snapshots_scope(self): diff --git a/packages/selenium-devtools-py/tests/test_capture_timing.py b/packages/selenium-devtools-py/tests/test_capture_timing.py index 3a79019e8..de9db4d50 100644 --- a/packages/selenium-devtools-py/tests/test_capture_timing.py +++ b/packages/selenium-devtools-py/tests/test_capture_timing.py @@ -10,7 +10,11 @@ from unittest import mock from selenium_devtools import capture_timing, instrumentation +from selenium_devtools.capturer import SessionCapturer from selenium_devtools.constants import ENV_CAPTURE_TIMING +from selenium_devtools.per_test_capture import PerTestCapture + +from wire_contract import RecordingTransport SCRIPTS = {"accessibilityTree": "(function(){})", "elements": "(function(){})"} @@ -155,6 +159,47 @@ def test_both_page_reads_are_timed(self): ["screenshot", "elements", "accessibilityTree"], ) + class ReadDriver: + session_id = "s" + caps = {"browserName": "chrome"} + + def execute(self, command, params=None): + return {"value": "Logout"} + + def get_screenshot_as_base64(self): + return "AA" + + def execute_script(self, script, *args): + return [{"selector": "#go"}] + + def test_a_read_is_timed_with_its_screenshot_alone_and_the_end_frame_as_final(self): + self.env("1") + instrumentation.uninstall() + self.addCleanup(instrumentation.uninstall) + instrumentation.install(SessionCapturer(RecordingTransport()), self.ReadDriver) + driver = self.ReadDriver() + with mock.patch.dict( + instrumentation._state, + { + "trace": True, + "a11y": True, + "element_scripts": SCRIPTS, + "action_snapshots": [], + "test": PerTestCapture(awaiting_initial_frame=False, started_ms=0), + }, + ): + driver.execute("getElementText", {"value": "x"}) + instrumentation._capture_final_frame(driver) + records, _ = _drain(self.out) + read, final = records + self.assertEqual((read["command"], read["trigger"]), ("getElementText", "action")) + self.assertEqual([p["probe"] for p in read["probes"]], ["screenshot"]) + self.assertEqual((final["command"], final["trigger"]), ("getElementText", "final")) + self.assertEqual( + [p["probe"] for p in final["probes"]], + ["screenshot", "elements", "accessibilityTree"], + ) + if __name__ == "__main__": unittest.main() diff --git a/packages/selenium-devtools-py/tests/test_mirror_vectors.py b/packages/selenium-devtools-py/tests/test_mirror_vectors.py index 1e234c274..6b73f03a6 100644 --- a/packages/selenium-devtools-py/tests/test_mirror_vectors.py +++ b/packages/selenium-devtools-py/tests/test_mirror_vectors.py @@ -22,6 +22,7 @@ capture_timing, capturer, device, + instrumentation, output_dir, performance, read_value_locators, @@ -51,6 +52,7 @@ def _vectors_path() -> Path: "nestedWaitIdentity", "performancePayload", "waitRowNaming", + "finalFrameStamp", "captureTimingSummary", } @@ -350,6 +352,18 @@ def test_cases(self): self.assertEqual({"command": command, "selector": selector}, case["expect"]) +class TestFinalFrameStamp(unittest.TestCase): + def test_cases(self): + for case in _cases("finalFrameStamp"): + with self.subTest(case["name"]): + rows = [(row["command"], row["timestamp"]) for row in case["commands"]] + stamp = instrumentation._final_frame_stamp( + rows, case["testStartMs"], case["lastCapturedTimestamp"] + ) + expect = None if case["expect"] is None else (case["expect"]["command"], case["expect"]["timestamp"]) + self.assertEqual(stamp, expect) + + class TestCaptureTimingSummary(unittest.TestCase): def test_cases(self): for case in _cases("captureTimingSummary"): diff --git a/packages/selenium-devtools-py/tests/test_pytest_plugin.py b/packages/selenium-devtools-py/tests/test_pytest_plugin.py index d54e726b3..40f17d133 100644 --- a/packages/selenium-devtools-py/tests/test_pytest_plugin.py +++ b/packages/selenium-devtools-py/tests/test_pytest_plugin.py @@ -11,7 +11,8 @@ import unittest from unittest import mock -from selenium_devtools import assertions +from selenium_devtools import assertions, instrumentation +from selenium_devtools.per_test_capture import PerTestCapture from selenium_devtools import pytest_plugin as plugin from selenium_devtools.pytest_plugin import _SuiteRegistry, reported_state from selenium_devtools.utils import iso @@ -550,3 +551,45 @@ def test_the_memoized_value_is_dropped_so_the_rewriter_re_reads(self): plugin._enable_assertion_pass_hook(config) self.assertIs(config.getini("enable_assertion_pass_hook"), True) + + +class TestEachTestStartsWithItsOwnInitialFrame(unittest.TestCase): + def test_a_test_starting_re_arms_the_initial_frame(self): + with mock.patch.dict( + instrumentation._state, + {"test": PerTestCapture(awaiting_initial_frame=False, initial_frame_retried=True)}, + ), mock.patch.object(plugin, "_opted_in", return_value=True), mock.patch.object( + plugin.devtools, "get_capturer", return_value=None + ): + plugin.pytest_runtest_logstart("tests/t.py::test_a", ("tests/t.py", 1, "test_a")) + + self.assertIs(instrumentation._state["test"].awaiting_initial_frame, True) + self.assertIs(instrumentation._state["test"].initial_frame_retried, False) + + + +class TestTheFinalFrameIsTakenAtTheCallReport(unittest.TestCase): + """The call report lands while the driver is still alive, before the + fixture teardown that quits it. Setup and teardown reports are not the + test's last page.""" + + def _log(self, when): + report = _Report(when, "passed") + report.nodeid = "tests/t.py::test_a" + report.location = ("tests/t.py", 1, "test_a") + with mock.patch.object(plugin, "_opted_in", return_value=True), mock.patch.object( + plugin.devtools, "get_capturer", return_value=object() + ), mock.patch.object(plugin, "_publish"), mock.patch.object( + plugin, "_registry" + ), mock.patch.object(instrumentation, "drain_native_network"), mock.patch.object( + instrumentation, "capture_final_frame" + ) as final: + plugin.pytest_runtest_logreport(report) + return final + + def test_the_call_report_takes_the_final_frame(self): + self._log("call").assert_called_once_with() + + def test_the_setup_and_teardown_reports_do_not(self): + self._log("setup").assert_not_called() + self._log("teardown").assert_not_called() diff --git a/packages/selenium-devtools/src/driverPatcher.ts b/packages/selenium-devtools/src/driverPatcher.ts index 778baba3a..cf89c3a1a 100644 --- a/packages/selenium-devtools/src/driverPatcher.ts +++ b/packages/selenium-devtools/src/driverPatcher.ts @@ -12,6 +12,7 @@ import { TRACKED_ELEMENT_METHODS, NAVIGATION_COMMANDS } from './constants.js' +import { patchActions } from './helpers/actions-patcher.js' import { rememberElementLocator, selectorForElement @@ -524,6 +525,7 @@ export function patchSelenium(hooks: DriverPatcherHooks): boolean { ) log.info(`Wrapped ${wrappedDriver.length} WebDriver method(s)`) + patchActions(WebDriver, hooks) patchDriverQuit(WebDriver.prototype as Patchable, hooks) if (WebElement) { patchWebElement(WebElement, hooks) diff --git a/packages/selenium-devtools/src/helpers/actions-patcher.ts b/packages/selenium-devtools/src/helpers/actions-patcher.ts new file mode 100644 index 000000000..d9e93562e --- /dev/null +++ b/packages/selenium-devtools/src/helpers/actions-patcher.ts @@ -0,0 +1,72 @@ +import logger from '@wdio/logger' +import { errorMessage } from '@wdio/devtools-core' +import { PATCHED_SYMBOL } from '../constants.js' +import type { + DriverPatcherHooks, + Patchable, + SeleniumDriverLike +} from '../types.js' + +const log = logger('@wdio/selenium-devtools:actionsPatcher') + +/** The driver each action sequence was built on, read when its perform() settles. */ +const actionsOwners = new WeakMap() + +function patchActionsPerform( + actionsProto: Patchable, + hooks: DriverPatcherHooks +): void { + if (actionsProto[PATCHED_SYMBOL]) { + return + } + actionsProto[PATCHED_SYMBOL] = true + + const originalPerform = actionsProto.perform as ( + this: unknown + ) => Promise + actionsProto.perform = async function patchedPerform(this: unknown) { + try { + await originalPerform.call(this) + } finally { + const driver = actionsOwners.get(this as object) + if (driver && hooks.afterActions) { + try { + hooks.afterActions(driver) + } catch (err) { + log.warn(`afterActions hook threw: ${errorMessage(err)}`) + } + } + } + } +} + +/** perform() reaches the driver through execute(), so no wrapped method sees the page change it makes. */ +export function patchActions( + WebDriver: { prototype: object }, + hooks: DriverPatcherHooks +): void { + const driverProto = WebDriver.prototype as Patchable + const originalActions = driverProto.actions + if ( + typeof originalActions !== 'function' || + PATCHED_SYMBOL in originalActions + ) { + return + } + const original = originalActions as ( + this: unknown, + options?: unknown + ) => object + const patchedActions = function patchedActions( + this: SeleniumDriverLike, + options?: unknown + ) { + const actions = original.call(this, options) + actionsOwners.set(actions, this) + patchActionsPerform(Object.getPrototypeOf(actions) as Patchable, hooks) + return actions + } + driverProto.actions = Object.assign(patchedActions, { + [PATCHED_SYMBOL]: true + }) +} diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index e452a94b5..df5acae34 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -5,12 +5,11 @@ import { errorMessage, isSessionGoneError, mapAssertCommand, - mapCommandToAction, toError, - upsertRichestSnapshot, type CapturedPerformancePayload, type RetryTracker } from '@wdio/devtools-core' +import { skipsCapture, type CaptureTrigger } from '@wdio/devtools-shared' import { ELEMENT_LOCATOR_METHODS } from '../constants.js' import { getDriverOriginals, getElementOriginals } from '../driverPatcher.js' import { captureOrReplaceCommand } from './captureOrReplaceCommand.js' @@ -347,10 +346,33 @@ export async function handleOnCommand( } maybeDrainAfterDomCommand(ctx, capturer, cmd) maybeDrainAfterLiveCommand(ctx, capturer, cmd) - queueActionSnapshot(ctx, capturer, cmd, entry.timestamp, error) + queueActionSnapshot( + ctx, + capturer, + cmd.command, + entry.timestamp, + error, + cmd.waitScope ? 'wait-poll' : 'action' + ) void capturer.nativeNetwork.forward({ throttle: true }) } +/** A driver-level action sequence's page change, stamped at completion for later reads to inherit. */ +export function handleActionsPerformed(ctx: OnCommandCtx): void { + const capturer = ctx.sessionCapturer + if (!capturer) { + return + } + queueActionSnapshot( + ctx, + capturer, + 'performActions', + Date.now(), + undefined, + 'action' + ) +} + /** Fire-and-forget post-action snapshot, drained at finalize. Stamped with the * logged command's timestamp, not the capture's own: the capture resolves after * the command completes, and export claims snapshots by exact equality with the @@ -358,29 +380,28 @@ export async function handleOnCommand( function queueActionSnapshot( ctx: OnCommandCtx, capturer: SessionCapturer, - cmd: CapturedCommand, + command: string, timestamp: number, - error: unknown + error: unknown, + trigger: CaptureTrigger ): void { - if ( - ctx.options.mode !== 'trace' || - error || - !ctx.driver || - !mapCommandToAction(cmd.command) - ) { + if (ctx.options.mode !== 'trace' || error || !ctx.driver) { return } - ctx.snapshotCaptures.push( - captureActionSnapshot( - ctx.driver, - cmd.command, - timestamp, - capturer.isNativeAppSession, - cmd.waitScope ? 'wait-poll' : 'action' - ).then((snap) => { - if (snap) { - upsertRichestSnapshot(ctx.actionSnapshots, snap) - } - }) + const driver = ctx.driver + capturer.testCapture.queue( + command, + timestamp, + !skipsCapture(command), + ctx.actionSnapshots, + ctx.snapshotCaptures, + () => + captureActionSnapshot( + driver, + command, + timestamp, + capturer.isNativeAppSession, + trigger + ) ) } diff --git a/packages/selenium-devtools/src/helpers/finalFrame.ts b/packages/selenium-devtools/src/helpers/finalFrame.ts new file mode 100644 index 000000000..19aa7874a --- /dev/null +++ b/packages/selenium-devtools/src/helpers/finalFrame.ts @@ -0,0 +1,38 @@ +import logger from '@wdio/logger' +import { errorMessage } from '@wdio/devtools-core' +import { captureActionSnapshot } from '../action-snapshot.js' +import type { SessionLifecycleCtx } from '../session-lifecycle.js' + +const log = logger('@wdio/selenium-devtools:finalFrame') + +/** Rows after the test's last capture saw a state nothing captured; one more capture, stamped at that capture, shows the test's end. */ +export async function captureFinalFrame( + ctx: SessionLifecycleCtx +): Promise { + const capturer = ctx.sessionCapturer + if ( + ctx.options.mode !== 'trace' || + ctx.finalized || + !ctx.driver || + !capturer + ) { + return + } + await capturer.testCapture.captureEndFrame( + capturer.commandsLog, + ctx.snapshotCaptures, + ctx.actionSnapshots, + // ctx.driver is read after the pending captures settle: a driver swap meanwhile must be honoured. + (stamp) => + ctx.driver + ? captureActionSnapshot( + ctx.driver, + stamp.command, + stamp.timestamp, + capturer.isNativeAppSession, + 'final' + ) + : Promise.resolve(null), + (err) => log.debug(`End-of-test frame capture failed: ${errorMessage(err)}`) + ) +} diff --git a/packages/selenium-devtools/src/index.ts b/packages/selenium-devtools/src/index.ts index e607676be..1fc5f6009 100644 --- a/packages/selenium-devtools/src/index.ts +++ b/packages/selenium-devtools/src/index.ts @@ -6,7 +6,10 @@ import logger from '@wdio/logger' import { startDetachedBackend } from './helpers/detachedBackend.js' import { openDashboard } from './helpers/dashboardLauncher.js' import { getDriverOriginals } from './driverPatcher.js' -import { handleOnCommand } from './helpers/commandPostActions.js' +import { + handleActionsPerformed, + handleOnCommand +} from './helpers/commandPostActions.js' import { logConfigSummary } from './helpers/configSummary.js' import { registerProcessHooks } from './helpers/processHooks.js' import { patchSelenium } from './driverPatcher.js' @@ -25,6 +28,7 @@ import { recordTraceBoundary, flushCurrentTestTrace } from './session-lifecycle.js' +import { captureFinalFrame } from './helpers/finalFrame.js' import type { SpecRange, TraceArtifact } from '@wdio/devtools-core' import { startTest as tmStartTest, @@ -480,6 +484,7 @@ class SeleniumDevToolsPlugin { async endTest(state: TestStats['state'] = 'passed') { const ended = this.#testManager?.getCurrentTest() ?? null tmEndTest(this.#getInternals(), state) + await captureFinalFrame(this.#getInternals()) const flushed = flushCurrentTestTrace(this.#getInternals()) await this.#emitTestArtifacts(ended, state === 'failed', flushed) } @@ -493,6 +498,7 @@ class SeleniumDevToolsPlugin { async endScenario(state: TestStats['state'] = 'passed') { const ended = this.#testManager?.getCurrentTest() ?? null tmEndScenario(this.#getInternals(), state) + await captureFinalFrame(this.#getInternals()) const flushed = flushCurrentTestTrace(this.#getInternals()) await this.#emitTestArtifacts(ended, state === 'failed', flushed) } @@ -548,6 +554,10 @@ class SeleniumDevToolsPlugin { return handled } + onActionsPerformed(driver: SeleniumDriverLike): void { + handleActionsPerformed({ ...this.#getInternals(), driver }) + } + /** Fallback path only: under the preload the destination instrumented itself. */ async instrumentNavigationTarget() { const capturer = this.#sessionCapturer @@ -687,6 +697,7 @@ const patched = patchSelenium({ onCommand: (cmd) => plugin.onCommand(cmd), afterNavigation: () => plugin.instrumentNavigationTarget(), beforeInput: () => plugin.focusForInput(), + afterActions: (driver) => plugin.onActionsPerformed(driver), // Per-scenario cleanup ONLY here (finalizes that driver's screencast, // clears per-driver state). The session-wide teardown — set-finalized, // session summary, capturer cleanup, interactive shutdown — lives in diff --git a/packages/selenium-devtools/src/test-management.ts b/packages/selenium-devtools/src/test-management.ts index 285611487..b4df38756 100644 --- a/packages/selenium-devtools/src/test-management.ts +++ b/packages/selenium-devtools/src/test-management.ts @@ -150,6 +150,7 @@ export function startTest( } ctx.testManager!.startMarkedTest(name, resolvedMeta) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (file) { ctx.sessionCapturer?.captureSource(file).catch(() => {}) } @@ -202,6 +203,7 @@ export function startScenario( const file = meta.file ?? root?.file ?? process.cwd() ctx.suiteManager.startScenarioSuite(name, file, meta.callSource, meta.file) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (meta.file) { ctx.sessionCapturer?.captureSource(meta.file).catch(() => {}) } diff --git a/packages/selenium-devtools/src/types.ts b/packages/selenium-devtools/src/types.ts index 0dac4297a..e57912f85 100644 --- a/packages/selenium-devtools/src/types.ts +++ b/packages/selenium-devtools/src/types.ts @@ -159,6 +159,8 @@ export interface DriverPatcherHooks { afterNavigation?: (command: string) => Promise // Awaited before an element click, sendKeys, clear or submit is sent. beforeInput?: () => Promise + // Fired after perform() settles, failed too: a partly performed sequence may already have moved the page. + afterActions?: (driver: SeleniumDriverLike) => void } // Unwrapped WebDriver methods used by the plugin's own internal calls diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index 7e75bcd8a..60cda6d15 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -1,4 +1,5 @@ -import { afterEach, describe, expect, it, vi } from 'vitest' +import type * as DevtoolsCore from '@wdio/devtools-core' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { resetSignatureCounters } from '../src/helpers/utils.js' import { TestManager } from '../src/helpers/testManager.js' @@ -10,11 +11,25 @@ import { type OnCommandCtx } from '../src/helpers/commandPostActions.js' import { captureActionSnapshot } from '../src/action-snapshot.js' +import { captureFinalFrame } from '../src/helpers/finalFrame.js' +import { handleActionsPerformed } from '../src/helpers/commandPostActions.js' +import type { SessionLifecycleCtx } from '../src/session-lifecycle.js' import { getDriverOriginals } from '../src/driverPatcher.js' -import { RetryTracker } from '@wdio/devtools-core' +import { + RetryTracker, + captureActionSnapshot as coreCapture +} from '@wdio/devtools-core' import type { ActionSnapshot } from '@wdio/devtools-shared' import type { CapturedCommand, SeleniumDriverLike } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + const capturers: SessionCapturer[] = [] afterEach(() => { @@ -26,13 +41,16 @@ afterEach(() => { /** A driver whose probes resolve to empty results — enough for the snapshot to * be produced without a browser. The patcher's `originals` bag is empty here * (nothing was patched), so the adapter falls back to these directly. */ -function fakeDriver(): SeleniumDriverLike { +type FakeDriver = SeleniumDriverLike & + Required> + +function fakeDriver(): FakeDriver { return { executeScript: vi.fn().mockResolvedValue([]), takeScreenshot: vi.fn().mockResolvedValue('AA'), getCurrentUrl: vi.fn().mockResolvedValue('http://example.test/'), getTitle: vi.fn().mockResolvedValue('Example') - } as unknown as SeleniumDriverLike + } as unknown as FakeDriver } function makeCtx(driver: SeleniumDriverLike) { @@ -94,7 +112,7 @@ describe('selenium action-snapshot timestamp binding', () => { ) }) - it('does not capture for a non-mapped command', async () => { + it('does not capture for a read-only lookup', async () => { const { ctx, snapshotCaptures, actionSnapshots } = makeCtx(fakeDriver()) await handleOnCommand( ctx, @@ -115,6 +133,209 @@ describe('selenium action-snapshot timestamp binding', () => { }) }) +describe('selenium action-snapshot capture gate', () => { + // The capture reads through the patcher's originals; route them to the fake + // driver so its screenshot probe is the one that can fail. + const patched = ['takeScreenshot'] as const + beforeEach(() => { + const originals = getDriverOriginals() as Record + for (const key of patched) { + originals[key] = (d: FakeDriver) => d[key]() + } + }) + afterEach(() => { + const originals = getDriverOriginals() as Record + for (const key of patched) { + delete originals[key] + } + }) + + async function run(commands: string[], ctxBag = makeCtx(fakeDriver())) { + let t = 1 + for (const command of commands) { + await handleOnCommand(ctxBag.ctx, cmd({ command, timestamp: t++ })) + } + await Promise.all(ctxBag.snapshotCaptures) + return ctxBag + } + + it('captures a click, but not a read or wait that follows it', async () => { + const { actionSnapshots } = await run([ + 'click', + 'getText', + 'waitForDisplayed' + ]) + expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + }) + + it('captures an unmapped command that can change the page', async () => { + const { actionSnapshots } = await run(['executeScript', 'actions']) + expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const bag = makeCtx(fakeDriver()) + await run(['getText', 'isDisplayed'], bag) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + bag.capturer.resetTestCapture() + await handleOnCommand(bag.ctx, cmd({ command: 'getText', timestamp: 9 })) + await Promise.all(bag.snapshotCaptures) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 9]) + }) + + it('retries the initial frame when the first read captures no screenshot', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValueOnce(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = makeCtx(driver) + for (const [i, command] of [ + 'getText', + 'isDisplayed', + 'getText' + ].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + expect(bag.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) + }) + + it('attempts the initial frame at most twice when every screenshot fails', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValue(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = makeCtx(driver) + for (const [i, command] of [ + 'getText', + 'getText', + 'getText', + 'getText' + ].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + }) + + it('retries the initial frame when the first capture resolves null', async () => { + vi.mocked(coreCapture).mockClear() + vi.mocked(coreCapture).mockResolvedValueOnce(null) + const bag = makeCtx(fakeDriver()) + for (const [i, command] of ['getText', 'getText', 'getText'].entries()) { + await handleOnCommand(bag.ctx, cmd({ command, timestamp: i + 1 })) + await Promise.all(bag.snapshotCaptures) + } + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + }) + + it('retries the initial frame when a page-changing first command loses its screenshot', async () => { + const driver = fakeDriver() + vi.mocked(driver.takeScreenshot).mockRejectedValueOnce(new Error('gone')) + vi.mocked(coreCapture).mockClear() + const bag = await run(['click', 'getText', 'getText'], makeCtx(driver)) + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(2) + expect(bag.actionSnapshots.map((s) => !!s.screenshot)).toEqual([ + false, + true + ]) + }) + + it('does not re-arm the initial frame from a capture that resolves after the next test starts', async () => { + type Snap = Awaited> + let resolveFirst!: (snap: Snap) => void + vi.mocked(coreCapture) + .mockClear() + .mockImplementationOnce( + () => + new Promise((resolve) => { + resolveFirst = resolve + }) + ) + const bag = makeCtx(fakeDriver()) + await handleOnCommand(bag.ctx, cmd({ command: 'getText', timestamp: 1 })) + bag.capturer.resetTestCapture() + resolveFirst({ timestamp: 1, command: 'getText' } as Snap) + await Promise.all(bag.snapshotCaptures) + + expect(bag.capturer.testCapture.initialFrameRetried).toBe(false) + expect(bag.capturer.testCapture.awaitingInitialFrame).toBe(true) + }) + + describe('end-of-test frame', () => { + const endOfTest = (bag: ReturnType) => + captureFinalFrame(bag.ctx as unknown as SessionLifecycleCtx) + + it('replaces the last capture with the end state when reads follow it', async () => { + const bag = await run(['click', 'getText']) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + vi.mocked(coreCapture).mockClear() + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'FINAL', + elements: [{ selector: '#final' }] + } as unknown as ActionSnapshot) + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(vi.mocked(coreCapture).mock.calls[0]?.[0]).toMatchObject({ + trigger: 'final' + }) + expect(bag.actionSnapshots).toHaveLength(1) + expect(bag.actionSnapshots[0]!.screenshot).toBe('FINAL') + expect(bag.actionSnapshots[0]!.elements).toEqual([{ selector: '#final' }]) + }) + + it('takes no end-of-test frame when the test ends on a capturing command', async () => { + const bag = await run(['getText', 'click']) + vi.mocked(coreCapture).mockClear() + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 2]) + }) + + it('takes no end-of-test frame for a test with no rows', async () => { + const bag = makeCtx(fakeDriver()) + vi.mocked(coreCapture).mockClear() + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).not.toHaveBeenCalled() + }) + + it('keeps the earlier capture when the end state saw no page', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + screenshot: 'GOOD', + elements: [{ selector: '#good' }] + } as unknown as ActionSnapshot) + const bag = await run(['click', 'getText']) + const before = { ...bag.actionSnapshots[0]! } + expect(before.screenshot).toBe('GOOD') + + vi.mocked(coreCapture).mockClear() + vi.mocked(coreCapture).mockResolvedValueOnce({ + timestamp: 1, + command: 'click', + snapshotText: '[Page]', + elements: [] + } as unknown as ActionSnapshot) + await endOfTest(bag) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(bag.actionSnapshots).toEqual([before]) + }) + }) +}) + describe('selenium action-snapshot locator dialect', () => { it('injects the XPath text form, which By.xpath resolves', async () => { // `a*=Logout` is WebdriverIO-only syntax; a Selenium user pastes the @@ -194,3 +415,18 @@ describe('the per-action snapshot on a native session', () => { } }) }) + +describe('driver-level action sequences', () => { + it('captures the page a perform left without adding a timeline row', async () => { + const bag = makeCtx(fakeDriver()) + vi.mocked(coreCapture).mockClear() + + handleActionsPerformed(bag.ctx) + await Promise.all(bag.snapshotCaptures) + + expect(vi.mocked(coreCapture)).toHaveBeenCalledTimes(1) + expect(bag.capturer.commandsLog).toHaveLength(0) + expect(bag.actionSnapshots).toHaveLength(1) + expect(bag.actionSnapshots[0]!.command).toBe('performActions') + }) +}) diff --git a/packages/selenium-devtools/tests/driverPatcher.test.ts b/packages/selenium-devtools/tests/driverPatcher.test.ts index 3ad7b38a5..7fa59c3ce 100644 --- a/packages/selenium-devtools/tests/driverPatcher.test.ts +++ b/packages/selenium-devtools/tests/driverPatcher.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect } from 'vitest' +import { describe, it, expect, vi } from 'vitest' import { isWebElementLike, safeSerialize, @@ -6,6 +6,8 @@ import { getDriverOriginals, getElementOriginals } from '../src/driverPatcher.js' +import { patchActions } from '../src/helpers/actions-patcher.js' +import type { DriverPatcherHooks } from '../src/types.js' describe('isWebElementLike', () => { it('detects an object with getId + click as a WebElement', () => { @@ -122,3 +124,76 @@ describe('getDriverOriginals / getElementOriginals', () => { expect(e).not.toBeNull() }) }) + +describe('patchActions', () => { + // A fresh class per test: patching marks the prototype, so a shared one would + // keep the first test's hooks. + function fakeDriverClass() { + class FakeActions { + constructor( + private readonly executor_: unknown, + private readonly fails = false + ) {} + async perform() { + if (this.fails) { + throw new Error('stale element') + } + return this.executor_ + } + } + return class FakeDriver { + actions(options?: { fails?: boolean }) { + return new FakeActions(this, options?.fails) + } + } + } + + function hooksWith( + afterActions = vi.fn(), + onCommand = vi.fn() + ): DriverPatcherHooks { + return { + onDriverCreated: () => {}, + onCommand, + afterActions + } as unknown as DriverPatcherHooks + } + + it('reports the owning driver once a sequence performs, with no command row', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + const onCommand = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions, onCommand)) + const driver = new FakeDriver() + + await driver.actions().perform() + + expect(afterActions).toHaveBeenCalledTimes(1) + expect(afterActions.mock.calls[0]![0]).toBe(driver) + expect(onCommand).not.toHaveBeenCalled() + }) + + it('still reports a sequence that moved the page before it failed, and rethrows its error', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions)) + const driver = new FakeDriver() + + await expect(driver.actions({ fails: true }).perform()).rejects.toThrow( + 'stale element' + ) + expect(afterActions).toHaveBeenCalledTimes(1) + expect(afterActions.mock.calls[0]![0]).toBe(driver) + }) + + it('wraps perform once however often it is patched', async () => { + const FakeDriver = fakeDriverClass() + const afterActions = vi.fn() + patchActions(FakeDriver, hooksWith(afterActions)) + patchActions(FakeDriver, hooksWith(afterActions)) + + await new FakeDriver().actions().perform() + + expect(afterActions).toHaveBeenCalledTimes(1) + }) +}) diff --git a/packages/selenium-devtools/tests/mirror-vectors.test.ts b/packages/selenium-devtools/tests/mirror-vectors.test.ts index b91298100..84a18bac9 100644 --- a/packages/selenium-devtools/tests/mirror-vectors.test.ts +++ b/packages/selenium-devtools/tests/mirror-vectors.test.ts @@ -17,6 +17,7 @@ import { } from '@wdio/devtools-shared' import { applyPerformanceData, + finalFrameStamp, formatCaptureTimingSummary, resolveAdapterOutputDir, resolveRunId, @@ -79,6 +80,7 @@ const RUNNERS = [ 'nestedWaitIdentity', 'performancePayload', 'waitRowNaming', + 'finalFrameStamp', 'captureTimingSummary' ] @@ -412,6 +414,27 @@ describe('waitRowNaming', () => { } }) +describe('finalFrameStamp', () => { + type Row = { command: string; timestamp: number } + for (const c of section< + Named & { + commands: Row[] + testStartMs: number + lastCapturedTimestamp: number | null + expect: Row | null + } + >('finalFrameStamp').cases) { + it(c.name, () => { + const stamp = finalFrameStamp( + c.commands, + c.testStartMs, + c.lastCapturedTimestamp ?? undefined + ) + expect(stamp).toEqual(c.expect) + }) + } +}) + describe('captureTimingSummary', () => { for (const c of section< Named & { diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index f21d970b0..5e23b1fd5 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -39,6 +39,7 @@ import { } from './action-snapshot.js' import { isWaitCommand, + skipsCapture, type ActionSnapshot, type TestMetadataMap } from '@wdio/devtools-shared' @@ -79,6 +80,7 @@ export * from './types.js' export const launcher = DevToolsAppLauncher const log = logger('@wdio/devtools-service') +const INITIAL_CAPTURE_MAX_ATTEMPTS = 2 export { setupForDevtools } from './standalone.js' @@ -159,6 +161,8 @@ export default class DevToolsHookService implements Services.ServiceInstance { /** Wall-clock ms at the current test's start, set in beforeTest/beforeScenario; * the lower bound of that test's video frame window (per-test slicing). */ #currentTestStartWallTime = 0 + #awaitingInitialCapture = true + #initialCaptureAttempts = 0 /** Map of testUid → metadata for trace group events and per-spec partitioning. */ #testMetadata: TestMetadataMap = new Map() @@ -380,6 +384,8 @@ export default class DevToolsHookService implements Services.ServiceInstance { }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true + this.#initialCaptureAttempts = 0 this.#currentStepIndex = 0 this.#currentStepUid = undefined @@ -413,6 +419,8 @@ export default class DevToolsHookService implements Services.ServiceInstance { beforeTest(test?: { file?: string; title?: string; fullTitle?: string }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true + this.#initialCaptureAttempts = 0 // Track test identity for command tagging. Generate a stable UID // from file + title so commands can be partitioned across reruns. @@ -583,7 +591,13 @@ export default class DevToolsHookService implements Services.ServiceInstance { // action's has no successor and this is the only capture of it. A session // that ran no action names it FINAL_SNAPSHOT_COMMAND, which the per-test // screenshot reads as "post-teardown frame, do not use". - const lastAction = this.#lastAction() + const lastChange = this.#lastCommandWhere( + (command) => !skipsCapture(command) + ) + const lastAction = + lastChange && lastChange.timestamp >= this.#currentTestStartWallTime + ? lastChange + : this.#lastAction() // A session with no action has no timestamp of its own to key on, so its // frame is recognised by the marker instead — otherwise `Date.now()` differs // between the per-test finalize and `after()` and both capture. @@ -641,20 +655,22 @@ export default class DevToolsHookService implements Services.ServiceInstance { * self-scopes, and a slot already filled at its own boundary converges on * `alreadyCaptured`. */ #lastAction(): { command: string; timestamp: number } | undefined { + return this.#lastCommandWhere(() => true) + } + + #lastCommandWhere( + predicate: (command: string) => boolean + ): { command: string; timestamp: number } | undefined { const commands = this.#sessionCapturer.commandsLog for (let i = commands.length - 1; i >= 0; i--) { const cmd = commands[i]! - if (this.#isActionCommand(cmd.command)) { + if (this.#isActionCommand(cmd.command) && predicate(cmd.command)) { return cmd } } return undefined } - #lastActionTimestamp(): number { - return this.#lastAction()?.timestamp ?? Date.now() - } - private resetStack() { this.#commandStack.reset() // No command is in flight after a reset; without this a window stranded by a @@ -767,8 +783,11 @@ export default class DevToolsHookService implements Services.ServiceInstance { // the previous action left behind. Taken HERE, before the command is issued, // because that is the one moment the driver is guaranteed idle and the app // at rest — a capture taken the instant a command returns catches whatever - // transition it started. Stamped at the previous action's end (or now, for - // the first, which makes it the initial frame). Trace mode only. + // transition it started. Stamped at the previous page-changing command's + // end (or now, for the first, which makes it the initial frame). Trace mode + // only. + // + // Reads/waits inherit the last page-changing capture; capturing them costs a page-source round trip. // // Not while a mobile-web Appium session has a document to probe — the // re-entrant probes can deadlock a serialising driver (#374). A native @@ -781,16 +800,19 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#browser, this.#sessionCapturer.currentContext ) && - this.#isActionCommand(command) + this.#isActionCommand(command) && + (!skipsCapture(command) || this.#awaitingInitialCapture) ) { - // Stamped at the previous action's end so this capture IS that action's + // Stamped at the previous page-changing command's end so this capture IS that command's // result — except across a test boundary, where that slot already holds // its own finalize capture and a second frame at the same timestamp lets // the richer-screenshot merge replace it (with a reloadSession between // the tests, the row then replays the post-reload page). The first // capture of a test stamps now instead, the same rule the session's first // capture uses to become the initial frame. - const previousEnd = this.#lastActionTimestamp() + const previousEnd = + this.#lastCommandWhere((cmd) => !skipsCapture(cmd))?.timestamp ?? + Date.now() const snap = await captureActionSnapshot( this.#browser, command, @@ -803,6 +825,13 @@ export default class DevToolsHookService implements Services.ServiceInstance { if (snap) { upsertRichestSnapshot(this.#actionSnapshots, snap) } + if ( + this.#awaitingInitialCapture && + (snap?.screenshot || + ++this.#initialCaptureAttempts >= INITIAL_CAPTURE_MAX_ATTEMPTS) + ) { + this.#awaitingInitialCapture = false + } } } diff --git a/packages/service/tests/trace-action-capture.test.ts b/packages/service/tests/trace-action-capture.test.ts index 188122e70..8de93bd50 100644 --- a/packages/service/tests/trace-action-capture.test.ts +++ b/packages/service/tests/trace-action-capture.test.ts @@ -19,7 +19,7 @@ vi.mock('stack-trace', () => ({ /** * The command has to be in the log by the time `afterCommand` returns: - * `#lastActionTimestamp()` reads it to stamp the NEXT action's capture, and that + * `#lastPageChange()` reads it to stamp the NEXT action's capture, and that * stamp is what makes an action's result the following action's "before". */ const commandsLog: { command: string; timestamp: number }[] = [] @@ -46,6 +46,7 @@ const capturer = { isReportingUpstream: false, metadata: {}, setBrowser: vi.fn(), + openWait: vi.fn(() => 0), /** Set by a test to stand in for the drain having brought a document the * session had not anchored before. */ replacedDocumentInLastDrain: false @@ -202,6 +203,107 @@ describe('trace mode: one capture per action, taken before it', () => { expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) }) + describe('reads and waits do not capture', () => { + const run = async ( + service: DevToolsHookService, + commands: string[] + ): Promise => { + for (const command of commands) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + } + const started = async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + vi.mocked(captureActionSnapshot).mockClear() + return service + } + + it('captures only the page-changing commands of click, getText x3, click', async () => { + const service = await started() + await run(service, ['click', 'getText', 'getText', 'getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('click') + expect(namedAt(1)).toBe('click') + }) + + it('stamps the capture after a wait at the preceding click end', async () => { + const service = await started() + await run(service, ['click', 'waitForDisplayed', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) + }) + + it('takes an initial capture on a leading read, then one on the click', async () => { + const service = await started() + await run(service, ['getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('getText') + expect(namedAt(1)).toBe('click') + }) + + it('retries the initial capture when the first read captures no screenshot', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + await run(service, ['getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('attempts the initial capture at most twice when no screenshot lands', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'getText' }) + .mockResolvedValueOnce({ timestamp: 2, command: 'getText' }) + await run(service, ['getText', 'getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('retries the initial capture when the first capture resolves null', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce(null) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + await run(service, ['getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('retries when a page-changing first command loses its screenshot', async () => { + const service = await started() + vi.mocked(captureActionSnapshot) + .mockResolvedValueOnce({ timestamp: 1, command: 'click' }) + .mockResolvedValueOnce({ + timestamp: 2, + command: 'getText', + screenshot: 'AA' + }) + await run(service, ['click', 'getText', 'getText']) + expect(namedAt(1)).toBe('getText') + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('takes the initial capture again for a new test', async () => { + const service = await started() + service.beforeTest({ file: '/spec/a.ts', title: 'one' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + service.beforeTest({ file: '/spec/a.ts', title: 'two' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + }) + it('captures the final state when the session ends', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) @@ -388,6 +490,51 @@ describe('trace mode: one capture per action, taken before it', () => { ]) }) + it('stamps the final capture at the test last page change, past later reads', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const test = { file: '/spec/a.ts', title: 'submit' } + service.beforeTest(test as never) + clock = Date.now() + 1000 + for (const command of ['click', 'getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(test as never, {} as never, {} as never) + + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + expect(namedAt(0)).toBe('click') + expect(stampedAt(0)).toBe(commandsLog[0]!.timestamp) + }) + + it('stamps the final capture at the last read when the test changed no page', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const first = { file: '/spec/a.ts', title: 'first' } + service.beforeTest(first as never) + await service.beforeCommand('click' as never, []) + await service.afterCommand('click' as never, [], undefined) + await service.afterTest(first as never, {} as never, {} as never) + + const second = { file: '/spec/a.ts', title: 'second' } + service.beforeTest(second as never) + clock = Date.now() + 1000 + for (const command of ['getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(second as never, {} as never, {} as never) + + expect(namedAt(0)).toBe('getText') + expect(stampedAt(0)).toBe(commandsLog.at(-1)!.timestamp) + }) + it('does not merge the next test first capture into the previous test slot', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) diff --git a/packages/shared/mirror-vectors.json b/packages/shared/mirror-vectors.json index 2d1d1291a..aea07d5e1 100644 --- a/packages/shared/mirror-vectors.json +++ b/packages/shared/mirror-vectors.json @@ -604,6 +604,58 @@ } ] }, + "finalFrameStamp": { + "mirrors": { + "ts": "core/final-frame.ts finalFrameStamp", + "py": "instrumentation.py _final_frame_stamp" + }, + "notes": "commands are the test's rows in completion order. expect is the stamp the end-of-test capture takes, or null when no row follows the last capture. A stamp at the last capture names the last row at or before it, and a stamp at a row names that row.", + "cases": [ + { + "name": "no capture yet stamps the test's last row", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": null, + "expect": { "command": "getText", "timestamp": 1200 } + }, + { + "name": "a capture from before the test started does not count", + "commands": [ + { "command": "click", "timestamp": 500 }, + { "command": "getText", "timestamp": 1200 }, + { "command": "click", "timestamp": 1300 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 900, + "expect": { "command": "click", "timestamp": 1300 } + }, + { + "name": "no row after the capture takes none", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 1200, + "expect": null + }, + { + "name": "rows after the capture stamp it, owned by the last row at or before it", + "commands": [ + { "command": "click", "timestamp": 1100 }, + { "command": "getText", "timestamp": 1200 }, + { "command": "click", "timestamp": 1300 }, + { "command": "getText", "timestamp": 1400 } + ], + "testStartMs": 1000, + "lastCapturedTimestamp": 1250, + "expect": { "command": "getText", "timestamp": 1250 } + } + ] + }, "captureTimingSummary": { "mirrors": { "ts": "core/capture-timing.ts summarizeCaptureTiming + formatCaptureTimingSummary", "py": "capture_timing.py summarize + format_summary" }, "notes": "wall and file are null when not given. Halves round up in both languages, which Python's round() does not do on its own; the second case is the one that tells them apart.", diff --git a/packages/shared/scripts/wire-schema.ts b/packages/shared/scripts/wire-schema.ts index a0d088fc2..cdc6ffc7c 100644 --- a/packages/shared/scripts/wire-schema.ts +++ b/packages/shared/scripts/wire-schema.ts @@ -4,6 +4,11 @@ import { realpathSync, writeFileSync } from 'node:fs' import path from 'node:path' import { fileURLToPath } from 'node:url' import ts from 'typescript' +import { + ACTION_MAP, + READ_ONLY_COMMANDS, + skipsCapture +} from '../src/trace-actions.js' export type JsonSchema = { [key: string]: unknown } @@ -12,6 +17,11 @@ export interface WireSchema { $comment: string scopes: Record $defs: Record + // Runner-native command names, evaluated from ACTION_MAP rather than typed out. + actionCommands: { + mapped: string[] + readOnly: string[] + } } const HERE = path.dirname(fileURLToPath(import.meta.url)) @@ -239,7 +249,18 @@ export function buildWireSchema(): WireSchema { $comment: 'GENERATED by packages/shared/scripts/wire-schema.ts from shared types. Do not edit.', scopes: sortKeys(scopes), - $defs: sortKeys(Object.fromEntries(builder.defs)) + $defs: sortKeys(Object.fromEntries(builder.defs)), + actionCommands: actionCommandSets() + } +} + +// Every ACTION_MAP key is mapped by mapCommandToAction (its fallback only adds assert.*, which no WebDriver hook sees). +function actionCommandSets(): WireSchema['actionCommands'] { + const commands = Object.keys(ACTION_MAP).sort() + const known = [...new Set([...commands, ...READ_ONLY_COMMANDS])].sort() + return { + mapped: commands, + readOnly: known.filter((c) => skipsCapture(c)) } } diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index f8ed5a066..a60ce0ed1 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -47,7 +47,58 @@ export function mapAssertCommand(command: string): TraceAction | null { return match ? { class: ASSERT_ACTION_CLASS, method: match[1] } : null } -export const ACTION_MAP: Record = { +export type ActionEffect = 'changes' | 'reads' + +export const METHOD_EFFECT = { + navigate: 'changes', + goBack: 'changes', + goForward: 'changes', + reload: 'changes', + goto: 'changes', + click: 'changes', + dblclick: 'changes', + fill: 'changes', + selectOption: 'changes', + hover: 'changes', + scrollIntoViewIfNeeded: 'changes', + dragTo: 'changes', + clear: 'changes', + submit: 'changes', + press: 'changes', + evaluate: 'changes', + tap: 'changes', + switchToWindow: 'changes', + getText: 'reads', + getValue: 'reads', + getAttribute: 'reads', + getProperty: 'reads', + getCSSProperty: 'reads', + getTagName: 'reads', + getLocation: 'reads', + getSize: 'reads', + getRect: 'reads', + isDisplayed: 'reads', + isExisting: 'reads', + isEnabled: 'reads', + isSelected: 'reads', + isClickable: 'reads', + isFocused: 'reads', + waitForDisplayed: 'reads', + waitForHidden: 'reads', + waitForExist: 'reads', + waitForNotExist: 'reads', + waitForEnabled: 'reads', + waitForClickable: 'reads', + waitForFunction: 'reads', + getTitle: 'reads', + getUrl: 'reads', + getPageSource: 'reads' +} as const satisfies Record + +export const ACTION_MAP: Record< + string, + TraceAction & { method: keyof typeof METHOD_EFFECT } +> = { // WDIO browser-level url: { class: 'Page', method: 'navigate' }, navigateTo: { class: 'Page', method: 'navigate' }, @@ -217,3 +268,87 @@ export function isInputDispatchingCommand(command: string): boolean { const action = ACTION_MAP[command] return action !== undefined && POINTABLE_METHODS.has(action.method) } + +/** Driver lookups and reads that never change the page and have no ACTION_MAP + * entry; the ACTION_MAP reads are covered by METHOD_EFFECT in `skipsCapture`. */ +export const READ_ONLY_COMMANDS: ReadonlySet = new Set([ + 'findElement', + 'findElements', + 'findChildElement', + 'findChildElements', + '$', + '$$', + 'findElementFromShadowRoot', + 'findElementsFromShadowRoot', + 'getElementShadowRoot', + 'getActiveElement', + 'getWindowHandle', + 'getWindowHandles', + 'getAllWindowHandles', + 'getCurrentContextHandle', + 'getContextHandles', + 'getCookies', + 'getAllCookies', + 'getCookie', + 'getCredentials', + 'getDownloadableFiles', + 'getWindowRect', + 'getLog', + 'getAvailableLogTypes', + 'getAlertText', + 'getTimeouts', + 'getScreenOrientation', + 'status', + 'takeScreenshot', + 'screenshot', + 'elementScreenshot', + 'printPage', + 'getHTML', + 'getComputedRole', + 'getComputedLabel', + 'getElementAriaLabel', + 'getElementAriaRole', + 'getShadowRoot', + 'w3cGetActiveElement', + 'w3cGetAlertText', + 'w3cGetCurrentWindowHandle', + 'w3cGetWindowHandles', + 'getNetworkConnection', + 'isVisible', + 'isPresent', + 'title', + 'source', + 'getCssProperty', + 'getElementSize', + 'getAriaRole', + 'getAccessibleName', + 'getFirstElementChild', + 'getLastElementChild', + 'getNextSibling', + 'getPreviousSibling', + 'hasDescendants', + 'element', + 'elements', + 'windowHandle', + 'windowHandles', + 'getWindowSize', + 'getWindowPosition', + 'getLogTypes', + 'isLogAvailable', + 'getDomAttribute', + 'getId', + 'getCapabilities', + 'getSession' +]) + +/** Whether a command is known not to change the page, so its capture can be + * skipped. Deny-list by design: an unlisted command still captures. */ +export function skipsCapture(command: string): boolean { + if (READ_ONLY_COMMANDS.has(command) || mapAssertCommand(command) !== null) { + return true + } + if (!Object.prototype.hasOwnProperty.call(ACTION_MAP, command)) { + return false + } + return METHOD_EFFECT[ACTION_MAP[command].method] === 'reads' +} diff --git a/packages/shared/tests/skips-capture.test.ts b/packages/shared/tests/skips-capture.test.ts new file mode 100644 index 000000000..69a48e120 --- /dev/null +++ b/packages/shared/tests/skips-capture.test.ts @@ -0,0 +1,54 @@ +import { describe, it, expect } from 'vitest' +import { ACTION_MAP, METHOD_EFFECT, skipsCapture } from '@wdio/devtools-shared' + +describe('skipsCapture', () => { + it.each([ + 'getText', + 'isDisplayed', + 'waitForDisplayed', + 'waitForElementVisible', + 'waitUntil', + 'getTitle', + 'findElement', + 'getAllWindowHandles', + 'printPage', + 'getCredentials', + 'getDownloadableFiles', + 'getCurrentUrl', + 'assert.equal', + 'expect.toHaveText', + 'getElementText', + 'w3cGetCurrentWindowHandle', + 'getElementAriaRole', + 'isVisible', + 'title', + 'getDomAttribute' + ])('is true for %s', (command) => { + expect(skipsCapture(command)).toBe(true) + }) + + it.each([ + 'click', + 'setValue', + 'url', + 'execute', + 'keys', + 'sendKeys', + 'switchToWindow', + 'executeScript', + 'actions', + 'submitForm', + 'acceptAlert', + 'w3cExecuteScript', + 'someUnknownCommand', + 'constructor' + ])('is false for %s', (command) => { + expect(skipsCapture(command)).toBe(false) + }) + + it('classifies the method of every ACTION_MAP entry', () => { + for (const { method } of Object.values(ACTION_MAP)) { + expect(Object.keys(METHOD_EFFECT)).toContain(method) + } + }) +}) diff --git a/packages/shared/tests/wire-schema.test.ts b/packages/shared/tests/wire-schema.test.ts index fa0952c97..80b061246 100644 --- a/packages/shared/tests/wire-schema.test.ts +++ b/packages/shared/tests/wire-schema.test.ts @@ -39,6 +39,17 @@ describe('wire schema', () => { }) }) + it('lists the read-only commands the capture gate skips', () => { + expect(schema.actionCommands.readOnly).toEqual( + expect.arrayContaining([ + 'findElement', + 'getElementText', + 'waitForDisplayed' + ]) + ) + expect(schema.actionCommands.readOnly).not.toContain('click') + }) + it('describes a Date as the ISO string it serializes to', () => { const props = (schema.$defs.TestStats.properties ?? {}) as Record< string, diff --git a/packages/shared/wire-schema.json b/packages/shared/wire-schema.json index 3616f4638..14d50cadb 100644 --- a/packages/shared/wire-schema.json +++ b/packages/shared/wire-schema.json @@ -1433,5 +1433,187 @@ ], "additionalProperties": false } + }, + "actionCommands": { + "mapped": [ + "addValue", + "back", + "clear", + "clearElement", + "click", + "clickElement", + "doubleClick", + "dragAndDrop", + "execute", + "executeAsync", + "forward", + "get", + "getAttribute", + "getCSSProperty", + "getCssValue", + "getCurrentUrl", + "getElementAttribute", + "getElementProperty", + "getElementRect", + "getElementTagName", + "getElementText", + "getElementValueOfCssProperty", + "getLocation", + "getPageSource", + "getProperty", + "getRect", + "getSize", + "getTagName", + "getText", + "getTitle", + "getUrl", + "getValue", + "goBack", + "goForward", + "isClickable", + "isDisplayed", + "isElementEnabled", + "isElementSelected", + "isEnabled", + "isExisting", + "isFocused", + "isSelected", + "keys", + "moveTo", + "navigateTo", + "newWindow", + "refresh", + "scrollIntoView", + "selectByVisibleText", + "sendKeys", + "sendKeysToElement", + "setValue", + "submit", + "switchToFrame", + "switchToParentFrame", + "switchToWindow", + "to", + "touchAction", + "url", + "wait", + "waitForClickable", + "waitForDisplayed", + "waitForElementLocated", + "waitForElementNotPresent", + "waitForElementNotVisible", + "waitForElementPresent", + "waitForElementVisible", + "waitForEnabled", + "waitForExist", + "waitUntil" + ], + "readOnly": [ + "$", + "$$", + "element", + "elementScreenshot", + "elements", + "findChildElement", + "findChildElements", + "findElement", + "findElementFromShadowRoot", + "findElements", + "findElementsFromShadowRoot", + "getAccessibleName", + "getActiveElement", + "getAlertText", + "getAllCookies", + "getAllWindowHandles", + "getAriaRole", + "getAttribute", + "getAvailableLogTypes", + "getCSSProperty", + "getCapabilities", + "getComputedLabel", + "getComputedRole", + "getContextHandles", + "getCookie", + "getCookies", + "getCredentials", + "getCssProperty", + "getCssValue", + "getCurrentContextHandle", + "getCurrentUrl", + "getDomAttribute", + "getDownloadableFiles", + "getElementAriaLabel", + "getElementAriaRole", + "getElementAttribute", + "getElementProperty", + "getElementRect", + "getElementShadowRoot", + "getElementSize", + "getElementTagName", + "getElementText", + "getElementValueOfCssProperty", + "getFirstElementChild", + "getHTML", + "getId", + "getLastElementChild", + "getLocation", + "getLog", + "getLogTypes", + "getNetworkConnection", + "getNextSibling", + "getPageSource", + "getPreviousSibling", + "getProperty", + "getRect", + "getScreenOrientation", + "getSession", + "getShadowRoot", + "getSize", + "getTagName", + "getText", + "getTimeouts", + "getTitle", + "getUrl", + "getValue", + "getWindowHandle", + "getWindowHandles", + "getWindowPosition", + "getWindowRect", + "getWindowSize", + "hasDescendants", + "isClickable", + "isDisplayed", + "isElementEnabled", + "isElementSelected", + "isEnabled", + "isExisting", + "isFocused", + "isLogAvailable", + "isPresent", + "isSelected", + "isVisible", + "printPage", + "screenshot", + "source", + "status", + "takeScreenshot", + "title", + "w3cGetActiveElement", + "w3cGetAlertText", + "w3cGetCurrentWindowHandle", + "w3cGetWindowHandles", + "wait", + "waitForClickable", + "waitForDisplayed", + "waitForElementLocated", + "waitForElementNotPresent", + "waitForElementNotVisible", + "waitForElementPresent", + "waitForElementVisible", + "waitForEnabled", + "waitForExist", + "waitUntil", + "windowHandle", + "windowHandles" + ] } } diff --git a/packages/trace/src/trace-snapshots.ts b/packages/trace/src/trace-snapshots.ts index 5edbd8189..18457b609 100644 --- a/packages/trace/src/trace-snapshots.ts +++ b/packages/trace/src/trace-snapshots.ts @@ -70,6 +70,27 @@ export function upsertRichestSnapshot( mergeSnapshotInto(snapshots[idx]!, snap) } +function definedFields(snap: ActionSnapshot): Partial { + return Object.fromEntries( + Object.entries(snap).filter(([, value]) => value !== undefined) + ) +} + +/** Supersede the record holding the snapshot's timestamp field by field: a later capture's defined fields win, and one it lacks (a screenshot that timed out) keeps the earlier value. */ +export function supersedeSnapshotAt( + snapshots: ActionSnapshot[], + snap: ActionSnapshot +): void { + let merged = snap + for (let i = snapshots.length - 1; i >= 0; i--) { + if (snapshots[i]!.timestamp === snap.timestamp) { + merged = { ...snapshots[i]!, ...definedFields(merged) } + snapshots.splice(i, 1) + } + } + snapshots.push(merged) +} + /** Name under `resources/` of a snapshot's screenshot, extension sniffed from its bytes. */ export function screenshotResourceName( pageId: string, diff --git a/packages/trace/tests/trace-frame-snapshots.test.ts b/packages/trace/tests/trace-frame-snapshots.test.ts index bee30161d..a43fa5e3e 100644 --- a/packages/trace/tests/trace-frame-snapshots.test.ts +++ b/packages/trace/tests/trace-frame-snapshots.test.ts @@ -71,6 +71,25 @@ describe('FrameSnapshotIndex', () => { expect(index.refs()).toEqual([]) }) + it('resolves every read row to the capture of the latest earlier page change', () => { + const index = new FrameSnapshotIndex([ + snap({ timestamp: 100, command: 'click', screenshot: 'A1' }), + snap({ timestamp: 200, command: 'click', screenshot: 'B2' }) + ]) + expect(index.claimAfter(50, 'read@0')).toBeUndefined() + const reads: [number, string][] = [ + [150, 'A1'], + [160, 'A1'], + [250, 'B2'] + ] + for (const [ts, shot] of reads) { + const id = `read@${ts}` + expect(index.claimAfter(ts, id)).toBe(`after@${id}`) + const ref = index.refs().find((r) => r.callId === id)! + expect(ref.snapshot.screenshot).toBe(shot) + } + }) + it('ignores snapshots without a screenshot', () => { const index = new FrameSnapshotIndex([snap({ screenshot: undefined })]) expect(index.claimAfter(2000, 'call@2')).toBeUndefined() diff --git a/packages/trace/tests/trace-snapshots.test.ts b/packages/trace/tests/trace-snapshots.test.ts index 7147f19ec..57eb9b223 100644 --- a/packages/trace/tests/trace-snapshots.test.ts +++ b/packages/trace/tests/trace-snapshots.test.ts @@ -6,6 +6,7 @@ import { buildFilmstripEvents, buildSnapshotResources, screenshotResourceName, + supersedeSnapshotAt, upsertRichestSnapshot, writeTraceZip, type TraceCapturer @@ -226,3 +227,47 @@ describe('snapshot binding contract (capture → export)', () => { await fs.rm(outputDir, { recursive: true, force: true }) }) }) + +describe('supersedeSnapshotAt', () => { + it('takes the later capture’s defined fields and keeps the rest', () => { + const list = [ + snap(50, REAL), + snap(100, REAL, { elements: [{ selector: '#old' }] }) + ] + supersedeSnapshotAt(list, snap(100, BLANK, { snapshotText: 'tree' })) + expect(list.filter((s) => s.timestamp === 100)).toEqual([ + { + timestamp: 100, + command: 'click', + screenshot: BLANK, + elements: [{ selector: '#old' }], + snapshotText: 'tree' + } + ]) + expect(list).toHaveLength(2) + }) + + it('keeps the earlier screenshot when a partial final capture has none', () => { + const list = [snap(100, REAL, { elements: [{ selector: '#old' }] })] + supersedeSnapshotAt(list, { + timestamp: 100, + command: 'click', + screenshot: undefined, + elements: [{ selector: '#new' }] + }) + expect(list).toEqual([ + { + timestamp: 100, + command: 'click', + screenshot: REAL, + elements: [{ selector: '#new' }] + } + ]) + }) + + it('appends when no record holds the timestamp', () => { + const list = [snap(50, REAL)] + supersedeSnapshotAt(list, snap(100, BLANK)) + expect(list.map((s) => s.timestamp)).toEqual([50, 100]) + }) +})