Skip to content

Nightwatch on a cloud grid can lose the whole trace when a screenshot lands after the session ended #524

Closed

Description

@Winify

Summary

A Nightwatch run in trace mode against BrowserStack (iOS native, XCUITest) sometimes writes no trace zip at all. The test passes, and the log shows:

WARN @wdio/nightwatch-devtools: trace write failed: base64.startsWith is not a function

The whole session's trace is lost, not one row. Reproduced on main (b9791e5) and on the #442 branch: 1 of 4 runs on main, 2 of 5 on the branch, with the same mechanism. #442 does not introduce it.

Cause

  1. The last action fires several screenshot requests at once: the row's screenshot, the per-action snapshot and the screencast poller. The device serves them one at a time, about 0.5 s each.
  2. Nightwatch deletes the session about 1 s after the last command. A screenshot request that reaches the hub after the DELETE is answered with HTTP 404 and the message Session not started or terminated, with no error field.
  3. packages/core/src/webdriver-http.ts isWebdriverError only treats a payload as an error when value.error is a string. This reply has no error field, so webdriverGet<string>(browser, 'screenshot') returns the object as if it were the screenshot.
  4. The object travels through packages/nightwatch-devtools/src/action-snapshot.ts, core's captureActionSnapshot (screenshot: shot ?? undefined) and upsertRichestSnapshot into ActionSnapshot.screenshot.
  5. packages/trace/src/trace-frame-snapshots.ts imageDocument guards only null/undefined (snap.screenshot ?? ''), so packages/shared/src/image.ts imageMime calls .startsWith on an object and throws. trace-finalizer.ts catches it as trace write failed, and no zip is written.

The error-shape check came in with 772ff82.

Evidence

  • Probe logs that record every /screenshot response with its HTTP status and typeof. In a failing run on main, the three requests started within 4 ms of each other and ended +658/+1012/+1282 ms later. The last one was http=404 typeof=object.
  • In another run, the same object also reached the screencast poller. That run's trace was saved, so the poller survived it, but how the poller handles a non-string frame was not checked.
  • Every successful response was a base64 PNG string (320-760 KB).

Scope

Seen with Nightwatch on a BrowserStack iOS native session. Any adapter that probes a screenshot over raw WebDriver HTTP while the session is being deleted can hit it, when the grid answers with a body that has no error field. Local chromedriver answers {value: {error: ...}}, which is already filtered, so this is mostly a cloud-grid issue.

Proposed fix

  • core/webdriver-http.ts: reject a non-2xx response before parsing it as a value. Treat a body without value, or a value with a string message and no payload, as an error.
  • Defence in depth where the value is consumed: captureActionSnapshot (core) keeps screenshot only when it is a non-empty string, and imageDocument/FrameSnapshotIndex skip a non-string screenshot. One bad frame should cost a frame, never the whole zip.
  • Check how the screencast poller handles a non-string frame in the same change.

Tests

  • webdriverGet against a fake 404 {"message": ...} response rejects, or returns null like the existing error path.
  • A trace export with one snapshot whose screenshot is an object still writes a zip, and that row resolves to the previous frame.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions