Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/capture-only-page-changes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@wdio/devtools-service': patch
'@wdio/selenium-devtools': patch
'@wdio/nightwatch-devtools': patch
---

Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for reads, waits and lookups such as `getText`, `isDisplayed`, `waitForDisplayed` or `findElement`. Those rows still appear in the trace and show the page as of the last command that took a capture (any command except reads, waits, lookups and asserts). Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module).

Selenium and Nightwatch now capture every other command, including ones that can navigate but are not in the action map (`executeScript`, `actions`, Nightwatch `submitForm`), and take one more capture at a test's end when reads follow its last capture, so a click that returns before its navigation commits no longer leaves the reads showing the page it left. The WDIO service keeps its page-changing gate.
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ The test for a new helper: *would the backend ever need this to build or read a

### One DOM snapshot per action, taken before it

The per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag.
In the WDIO service the per-action trace snapshot is captured in `beforeCommand`, stamped at the previous action's end, so an action's result IS the next action's "before" and every row resolves to a state the driver was idle for. Its `afterCommand` captures nothing in trace mode. An eager post-action capture — added and removed twice — lands while the screen is still moving, which is the only reason `waitForActionResult` and the `__wdioSnapMark` document tag existed; both are deleted, and both should stay that way. The last action has no successor, so `#finalizePerScenario` supplies it and names its capture after that action — `lastRenderedScreenshot` skips a capture named `__final__`, which is reserved for a session that ran no action at all. What that one capture waits on is `settleAfterLastAction`, and the wait is **gated, not timed**: the drain that runs immediately before it (`captureTrace(browser, true)`) anchors each document once, so `SessionCapturer.replacedDocumentInLastDrain` says whether the last action navigated to a document the session had not seen. No → return at once (the app has been at rest since the last action, so the test's own teardown is the gap that let the paint land). Yes → `waitUntil(document.readyState === 'complete')`, which is only meaningful once you know the document being described is the incoming one: ungated, the OUTGOING document already reports 'complete' right after a click, so a blind poll returns instantly and captures the page the test just left, and the old `body.childElementCount > 0` clause that papered over that made a legitimately blank destination a guaranteed 8 s timeout per test. Native pauses 250 ms instead (no document to poll; a capture at a 0 s gap measures 359–476 KB against 1.87 MB settled on Appium). Nightwatch re-captures a command whose capture predates the destination document's `performance.timeOrigin` (a navigation that committed after it returned), and discards that re-capture if a newer page capture was queued meanwhile. Residual: a navigation that has not *committed* when the anchor is read still reads as "no navigation" — the same blind spot the deleted document tag had. Don't reintroduce a poll without a gate, or a tag.

Selenium, Nightwatch and Python skip a capture only for a command `skipsCapture` (`shared/trace-actions.ts`) knows is a read, wait or lookup, and capture after every other command, including unmapped ones that can navigate (`executeScript`, `actions`, Nightwatch `submitForm`). A test's first mapped command always takes one. Selenium and Nightwatch (selenium `commandPostActions.ts` `queueActionSnapshot`, nightwatch `SessionCapturer#queueActionSnapshot`) capture right after the command itself. Rows after a test's last capture mean that capture saw a state the test later left, so the test takes one more capture at its end, stamped at that last capture (`core/final-frame.ts` `finalFrameStamp`) and superseding that record field by field, only when it has a screenshot; this covers a click that returns before its navigation commits. A test ending on a capturing command pays nothing. WDIO applies the same gate, but only to commands it renders as action rows, and captures before the next command rather than after its own, so it takes no end-of-test frame. In the JS adapters and Python a read, wait or lookup takes no capture, and inherits the latest earlier one through `claimAfter`.

### Adapters are thin and isolated

Expand Down Expand Up @@ -398,7 +400,7 @@ Documented divergences from the conventions above. They exist today as debt to b
- **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2.
- A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch.
- **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop.
- Residual: **a polling matcher fires one capture per poll** — each poll is a top-level mapped read — so "one capture per action" undercounts a real suite. Pre-existing in `d924a02`, unmeasured on a device, and the likeliest next lever.
- **Only a command that can change the page is captured** (`shared/trace-actions.ts` `skipsCapture`, a deny-list: reads, waits, lookups and assertions are skipped, anything unlisted captures; the ACTION_MAP reads come from the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit.
- The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands.

### File-size (raw line counts; soft cap is 500 logic lines)
Expand All @@ -409,7 +411,7 @@ Most entries below don't trigger the `max-lines` lint rule after `skipBlankLines
- `packages/nightwatch-devtools/src/index.ts` (783 raw / 676 logic). Cucumber/test/run-lifecycle, session-init, event-hub and now the screencast seam (`plugin-screencast.ts`, 105 raw / 60 logic) are extracted; the remainder is the `PluginInternals` accessor bag plus per-method delegators plus the factory. The bag is deliberately declarative — accept as-is.
- `packages/selenium-devtools/src/index.ts` (~644 raw, down from ~758 — the dead `scriptInjected` accessor pair and setter are gone). Session/test-lifecycle **and** the per-test-artifact seam are now extracted: the sink cache + input snapshot + produce/attach flow live in `selenium-devtools/src/test-artifacts.ts` as `SeleniumTestArtifacts` (mirrors Nightwatch's twin — a typed input bag threading the Allure sink + flushed-trace promise), and the plugin keeps only a thin bag-building delegator. Remainder is the `PluginInternals` accessor bag plus onCommand/onDriverCreated wiring. Still over the 500 **raw** soft cap (under the logic-line cap after `skipBlankLines`/`skipComments`); the accessor bag / command wiring is the next extraction candidate if it grows.
- `packages/nightwatch-devtools/src/session.ts` (519 raw, under the logic-line cap after `skipBlankLines`/`skipComments`). `captureNetworkFromPerformanceLogs` + `drainCollector` are tightly coupled to NightwatchBrowser state. Coverage at 78% after recent backfill; further extraction would need rewriting the browser-coupling.
- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (975 raw). Every seam here reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` itself should split it per concern first; `action_snapshot.py` is then a clean lift.
- `packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py` (1297 raw). The per-test frame state is split out (`per_test_capture.py` `PerTestCapture`, one `_state["test"]` entry); every other seam here still reads the module-level `_state` bag — the per-action snapshot capture, its two accessors and the element-scripts handoff all key off `trace`/`a11y`/`element_scripts` alongside the screencast and session entries — so an extraction is a move of state ownership, not a lift of a function. The next change that touches `_state` should split another concern out the same way; `action_snapshot.py` is then a clean lift.

### Test coverage gaps (worst-risk-first)

Expand Down
2 changes: 1 addition & 1 deletion docs/ADAPTERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ All times are epoch milliseconds except the suite/test `start`/`end` ISO strings

### Generating your types

`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with:
`packages/shared/wire-schema.json` is a JSON Schema (draft 2020-12) generated from shared's TypeScript types by `packages/shared/scripts/wire-schema.ts`: one entry per scope under `scopes`, shared shapes under `$defs`, and the command names the Python capture gate reads under `actionCommands`. A vitest test (`packages/shared/tests/wire-schema.test.ts`) fails when the committed copy is stale; regenerate with:

```sh
pnpm --filter @wdio/devtools-shared gen:wire-schema
Expand Down
36 changes: 36 additions & 0 deletions packages/core/src/final-frame.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import type { ActionSnapshot } from '@wdio/devtools-shared'

/** Frames are screenshot-backed: a url-only capture merged over an earlier record would pair the new url with the old image and DOM. */
export function capturedPageState(snap: ActionSnapshot): boolean {
return Boolean(snap.screenshot)
}

export interface FinalFrameCommand {
command: string
timestamp: number
}

/** Rows after the last capture saw a state no capture holds; the end-of-test capture is stamped at that capture's timestamp so those rows resolve to it. */
export function finalFrameStamp(
commands: ReadonlyArray<FinalFrameCommand>,
testStartMs: number,
lastCapturedTimestamp: number | undefined
): FinalFrameCommand | null {
const inTest = commands.filter((c) => c.timestamp >= testStartMs)
const last = inTest[inTest.length - 1]
if (!last) {
return null
}
if (
lastCapturedTimestamp === undefined ||
lastCapturedTimestamp < testStartMs
) {
return { timestamp: last.timestamp, command: last.command }
}
if (!inTest.some((c) => c.timestamp > lastCapturedTimestamp)) {
return null
}
const atOrBefore = inTest.filter((c) => c.timestamp <= lastCapturedTimestamp)
const owner = atOrBefore[atOrBefore.length - 1] ?? inTest[0]!
return { timestamp: lastCapturedTimestamp, command: owner.command }
}
1 change: 1 addition & 0 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ export * from './request-type.js'
export * from './stack.js'
export * from './terminal-throttle.js'
export * from './error.js'
export * from './final-frame.js'
export * from './finalize-screencast.js'
export * from './input-dispatch.js'
export * from './output-dir.js'
Expand Down
8 changes: 8 additions & 0 deletions packages/core/src/session-capturer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import {
} from '@wdio/devtools-shared'
import { resolveRunId } from './run-id.js'
import { reattributeDomAnchors } from '@wdio/devtools-trace/trace-mutations'
import { TestCapture } from './test-capture.js'
import {
CONSOLE_METHODS,
LOG_SOURCES,
Expand Down Expand Up @@ -139,6 +140,13 @@ export abstract class SessionCapturerBase {
return sessionHasDocument(this.metadata?.capabilities, this.currentContext)
}

/** Per-test capture state for adapters that capture after each command. */
readonly testCapture = new TestCapture()

resetTestCapture(): void {
this.testCapture.reset()
}

// ── Construction ────────────────────────────────────────────────────────
constructor(opts: SessionCapturerOptions = {}) {
const { hostname, port, reconnect } = opts
Expand Down
Loading
Loading