Skip to content

Trace mode: measure where per-action capture time goes, per platform #516

Closed

Description

@vishnuv688

Problem

A trace-mode run is slower than a live one, and the cost is the per-action capture: before each action the adapter reads the URL, title, accessibility tree, element list and a screenshot (plus the page source on native mobile) through the driver. Nothing measured yet says where that time goes on each platform, so any proposal to make capture cheaper is a guess.

What is measured today, from CLAUDE.md:

  • Android, Appium 3.7 / UiAutomator2: screenshot 1.18 to 2.22 s (steady about 1.2 s), page source 0.09 to 0.53 s. Two captures per action took 2.41 s against 1.19 s for one; the native example ran 12.4 s in trace mode against 5.9 s live.
  • iOS, XCUITest: page source 0.75 to 1.2 s per action. Screenshot unmeasured.
  • Desktop Chrome, Firefox and Safari: unmeasured.
  • A polling matcher takes one capture per poll, so "one capture per action" undercounts a real suite. Unmeasured.

What is not a lever

packages/core/src/action-snapshot.ts already starts every probe together in one Promise.all. A driver serves one session's requests one at a time, in the order issued, so starting them together does not overlap them. The order is deliberate: the cheap reads go first, the screenshot last, because a slow screenshot issued first delayed the reads long enough for the next command to navigate.

Proposal

  1. Time each probe of a capture per platform (desktop Chrome, Firefox, Safari; Android and iOS native and mobile web), on the four adapters' examples, and report the share each probe takes of an action and of the whole run, including the polls of waiting matchers.
  2. Only then propose changes, ranked by measured time saved.

Constraint: default capture loses nothing

The default must keep capturing everything it captures today. Two ideas are lossy and are not acceptable as defaults:

  • Skipping the screenshot when the DOM did not change: canvas, video, CSS animation, an image or font finishing loading, and scrolling all change pixels with no DOM change, and comparing against the last screenshot needs the screenshot taken.
  • Capturing only the first and last poll of a waiting matcher: the polls in between are the page while it waited, which is often what explains a timeout.

A change that trades detail for speed may only ship as an opt-in option, documented as lossy.

Activity

  1. self-assigned this
    on Oct 30, 2026
  2. vishnuv688 commented on Oct 30, 2026

    @vishnuv688
    MemberAuthor

    Step 1 (opt-in timing) in #519, stacked on #518. Step 2 is running the README's measurement commands per platform and posting the numbers here.

  3. vishnuv688 commented on Nov 10, 2026

    @vishnuv688
    MemberAuthor

    Step 2 results

    Measured on v11.2.0 (main at 4e77f78), trace mode with DEVTOOLS_CAPTURE_TIMING=1, one run at a time on one macOS machine. Desktop runs drove the local fixture site. Machine load was high (security software, and a WebDriverAgent build during the iOS phase), so treat absolute numbers as rough; the ranking held across repeated runs.

    Platform / adapter Captures per run Median capture What dominates Capture share of wall
    Chrome, WebdriverIO 15 50 ms a11y tree 1.5-3.6%
    Chrome, Selenium 14 89 ms url read (queued) ~5%
    Chrome, Nightwatch 12 106 ms screenshot 9-10%
    Chrome, Python 42 45 ms screenshot 14-17%
    Safari, Selenium 11 92 ms url read (queued) ~4%
    Android native, WebdriverIO 8 384-471 ms page source and screenshot, about half each 11-12%
    Android native, Selenium 13 665 ms screenshot, behind the filmstrip poller 26%
    Android native, Nightwatch 11 598 ms screenshot, behind the filmstrip poller 15%
    Android native, Python 42 192 ms screenshot 44%
    iOS native, WebdriverIO 6 938 ms page source 817 ms (87%) ~21%
    iOS Safari, WebdriverIO 4 128 ms screenshot 3-5%

    Not measured: Firefox (not installed), Android Chrome (a local Appium chromedriver permission problem), the other adapters on iOS Safari.

    Levers, ranked by measured time

    1. The 200 ms polling filmstrip on native. Selenium and Nightwatch keep filmstrip on by default and, with no CDP screencast on Appium or Safari, poll a screenshot every 200 ms: 95-100 polled shots per Android run against 11-13 per-action captures (67 vs 7 on iOS, 93-103 vs 11 on desktop Safari). With the poller running, a per-action screenshot took 600-665 ms on the emulator, against 229 ms with no recorder: about 400 ms per capture spent waiting behind polls. This agrees with the PR perf(trace): capture page state only for commands that change it #442 trace-performance dataset Vince shared. Lossless: skip a poll while a command or a capture is in flight and reuse the per-action screenshot as a frame. Opt-in: a slower or adaptive interval, or no filmstrip on mobile.
    2. iOS page source is 87% of an iOS capture; Android splits about evenly. Candidate: XCUITest's page-source settings (excluded attributes, snapshot depth), only if nothing the trace or core/locators reads depends on what is excluded. Needs a check first.
    3. Python captures on every command, reads included (findElements, getElementText, isElementEnabled, getCurrentUrl), 42 captures for a flow the JS adapters capture 8 times. Applying the JS adapters' rule (capture only commands that can change the page) is lossless and would save about 5.7 s of a 22 s Android run.
    4. Desktop: the first read of a capture waits 40-70 ms on Selenium, Nightwatch and Safari behind adapter work issued just before it (WebdriverIO's url read takes 3 ms). Small.

    Also found: on native, Selenium and Nightwatch capture only the screenshot (no page source, so no element or accessibility tree), and the capture-timing report does not count the filmstrip's own screenshots, so it understates lever 1.

    Follow-ups: #561, #562, #563, #564.

  4. vishnuv688 commented on Nov 10, 2026

    @vishnuv688
    MemberAuthor

    Both steps are done: opt-in capture timing (#519) and the per-platform measurements above. The improvements they point to are tracked in #561, #562, #563 and #564.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions