Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/hosted-player-parse-in-worker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@wdio/devtools-app': patch
---

The hosted trace player reads a trace `.zip` in a Web Worker instead of on the page's main thread, so the tab no longer freezes while a large trace opens. While it works, the landing screen names the step it is on (reading the file, unpacking the archive, building the timeline) under a progress bar that does not claim a percentage. The worker loads from the player's own origin, within the existing content security policy. The live dashboard is unchanged.
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Contains:
- `buildImageFrameSnapshots` + `buildScreenshotResourceSnapshots` / `upsertRichestSnapshot` / `buildDenseScreencast` / `thinScreencastFrames` — filmstrip and per-action frame snapshots (each a page whose image is the stored screenshot, reached through a `resourceOverrides` entry and the one recorded response it patches).
- `buildMutationsNdjson` / `reattributeDomAnchors` — the DOM mutation stream and its anchor repositioning.
- `buildConsoleEvents`, `networkRequestToHar`, `buildSourceResources` / `sourceResourceName` / `callSourceToStack`, `generateTranscript`, `sha1Hex`.
- `parseTraceZip` (`trace-reader.ts` and its `trace-reader-*` siblings) — the reverse direction: a `trace.zip`, ours or a foreign one, back into the `TracePlayerData` the player renders. Imported by subpath (`@wdio/devtools-trace/trace-reader`) and kept out of the package index, so adapters that reach `trace` through `core` never inline it. It reaches no Node built-in, because the hosted player runs it in the browser (`tests/trace-reader-browser-safe.test.ts` walks its imports); the disk read is `readTraceZip` in `read-trace-zip.ts`, and source paths are hashed with the pure `portableSha1Hex` so the reader needs no `node:crypto`; resource bytes keep native `sha1Hex`, ~30x faster.
- `parseTraceZip` (`trace-reader.ts` and its `trace-reader-*` siblings) — the reverse direction: a `trace.zip`, ours or a foreign one, back into the `TracePlayerData` the player renders. Imported by subpath (`@wdio/devtools-trace/trace-reader`) and kept out of the package index, so adapters that reach `trace` through `core` never inline it. `parseTraceZip` is `unzipTraceZip` then `parseTraceEntries`, exported separately so the hosted player can report each as a phase. It reaches no Node built-in, because the hosted player runs it in the browser, in a Web Worker (`tests/trace-reader-browser-safe.test.ts` walks its imports); the disk read is `readTraceZip` in `read-trace-zip.ts`, and source paths are hashed with the pure `portableSha1Hex` so the reader needs no `node:crypto`; resource bytes keep native `sha1Hex`, ~30x faster.

Imports from: `shared`. Imported by: `core` (which re-exports it, so adapters reach it unchanged), `backend`, and `app`, which may import `trace-reader` alone (ESLint enforces it).

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ They import from `shared` and `core`, never from each other. They aren't importe

### Backend and app are framework-agnostic

`backend` and `app` import from `shared` (for contracts) and from each other via the WS/HTTP boundary. Neither imports an adapter package. `backend` also imports `trace`, to write and read zips; `app` imports only `trace`'s reader, which the hosted player (`trace.webdriver.io`, built with `pnpm player:hosted`, deployed by Vercel from `vercel.json`) runs in the browser to open a dropped zip with no backend at all. That build is `HOSTED_PLAYER` in `app/src/controller/constants.ts`; it never writes the trace to browser storage, and `vercel.json` is also what `vite preview` serves, so its CSP is checked locally before it ships.
`backend` and `app` import from `shared` (for contracts) and from each other via the WS/HTTP boundary. Neither imports an adapter package. `backend` also imports `trace`, to write and read zips; `app` imports only `trace`'s reader, which the hosted player (`trace.webdriver.io`, built with `pnpm player:hosted`, deployed by Vercel from `vercel.json`) runs in the browser to open a dropped zip with no backend at all. It runs the reader in a module Web Worker loaded from the player's own origin (`app/src/controller/trace-parse.worker.ts`, which `script-src 'self'` already admits), so the tab stays responsive and the landing screen names the phase the open is on; the live build never references the worker. That build is `HOSTED_PLAYER` in `app/src/controller/constants.ts`; it never writes the trace to browser storage, and `vercel.json` is also what `vite preview` serves, so its CSP is checked locally before it ships.

Framework-specific behavior in the backend is contained in two files: `runner.ts` and `framework-filters.ts`. Both branch on a typed `TestRunnerId` from shared, never on a magic string. The `framework-filters` dispatch is a `switch` over `TestRunnerId` (not a table lookup) so CodeQL's `unvalidated-dynamic-method-call` query trusts the call site.

Expand Down
5 changes: 3 additions & 2 deletions packages/app/src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ onThemeChange(applyDarkMode)
export class WebdriverIODevtoolsApplication extends Element {
dataManager = new DataManagerController(this)
#traceOpen = HOSTED_PLAYER
? new TraceOpenController(this, (file) =>
this.dataManager.openTraceArchive(file)
? new TraceOpenController(this, (file, onPhase) =>
this.dataManager.openTraceArchive(file, onPhase)
)
: undefined

Expand Down Expand Up @@ -256,6 +256,7 @@ export class WebdriverIODevtoolsApplication extends Element {
if (traceOpen && (traceOpen.opening || !this.dataManager.hasConnection)) {
return html`<wdio-devtools-open-trace
.busy="${traceOpen.opening}"
.phase="${traceOpen.phase}"
.error="${traceOpen.error}"
></wdio-devtools-open-trace>`
}
Expand Down
54 changes: 50 additions & 4 deletions packages/app/src/components/onboarding/open-trace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,18 @@ import { Element } from '@core/element'
import { html, css, nothing } from 'lit'
import { customElement, property, query, state } from 'lit/decorators.js'

import { TRACE_FILE_ACCEPT } from '../../controller/constants.js'
import {
TRACE_FILE_ACCEPT,
TRACE_OPEN_PHASE_LABEL
} from '../../controller/constants.js'
import type { TraceOpenPhase } from '../../controller/trace-parse-protocol.js'
import { announceTraceFile, takePickedFile } from '../../utils/trace-file.js'

/** Landing screen of the hosted trace player: pick or drop a trace.zip. */
@customElement('wdio-devtools-open-trace')
export class DevtoolsOpenTrace extends Element {
@property({ type: Boolean }) busy = false
@property() phase?: TraceOpenPhase
@property() error?: string

@state() dragging = false
Expand Down Expand Up @@ -61,6 +66,41 @@ export class DevtoolsOpenTrace extends Element {
cursor: progress;
}

/* Indeterminate: the reader cannot say how far through a step it is. */
.progress {
height: 3px;
margin-top: 12px;
overflow: hidden;
border-radius: 2px;
background: var(--vscode-panel-border);
}

.progress::after {
content: '';
display: block;
width: 40%;
height: 100%;
background: var(--accent);
animation: progress-slide 1.2s ease-in-out infinite;
}

@keyframes progress-slide {
from {
transform: translateX(-100%);
}
to {
transform: translateX(250%);
}
}

@media (prefers-reduced-motion: reduce) {
.progress::after {
width: 100%;
animation: none;
opacity: 0.5;
}
}

.error {
color: var(--vscode-charts-red);
}
Expand Down Expand Up @@ -106,6 +146,13 @@ export class DevtoolsOpenTrace extends Element {
this.#open(takePickedFile(this.input))
}

#status(): string {
if (!this.busy) {
return 'Drop a trace.zip here'
}
return this.phase ? TRACE_OPEN_PHASE_LABEL[this.phase] : 'Opening trace…'
}

render() {
return html`
<div
Expand All @@ -117,9 +164,8 @@ export class DevtoolsOpenTrace extends Element {
<img src="/robot.png" width="120" alt="" />
<h2 class="text-3xl font-bold">WebdriverIO Trace Viewer</h2>
<div class="drop ${this.dragging ? 'dragging' : ''}">
<p class="text-lg">
${this.busy ? 'Opening trace…' : 'Drop a trace.zip here'}
</p>
<p class="text-lg" role="status">${this.#status()}</p>
${this.busy ? html`<div class="progress"></div>` : nothing}
<button
?disabled="${this.busy}"
@click="${() => this.input?.click()}"
Expand Down
4 changes: 2 additions & 2 deletions packages/app/src/controller/DataManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,8 @@ export class DataManagerController implements ReactiveController {
this.#host.requestUpdate()
}

async openTraceArchive(file: Blob) {
this.loadPlayerData(await readTraceArchive(file))
async openTraceArchive(...args: Parameters<typeof readTraceArchive>) {
this.loadPlayerData(await readTraceArchive(...args))
}

#connectWebSocket() {
Expand Down
7 changes: 7 additions & 0 deletions packages/app/src/controller/constants.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
import type { LogSource } from '@wdio/devtools-shared'

import type { TraceOpenPhase } from './trace-parse-protocol.js'

export const CACHE_ID = 'wdio-trace-cache'
/** Built with `--mode hosted`: the static trace player, with no backend. */
export const HOSTED_PLAYER = import.meta.env.MODE === 'hosted'
export const TRACE_OPEN_EVENT = 'app-trace-open'
export const TRACE_FILE_ACCEPT = '.zip,application/zip'
export const TRACE_OPEN_PHASE_LABEL: Record<TraceOpenPhase, string> = {
reading: 'Reading the file…',
unpacking: 'Unpacking the archive…',
building: 'Building the timeline…'
}
export const SIDEBAR_MIN_WIDTH = 250
export const DARK_MODE_KEY = 'darkMode'
/**
Expand Down
78 changes: 70 additions & 8 deletions packages/app/src/controller/trace-archive.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,77 @@
import type { TracePlayerData } from '@wdio/devtools-shared'

/** Parse a trace.zip the viewer picked, entirely in this tab. */
export async function readTraceArchive(file: Blob): Promise<TracePlayerData> {
import { HOSTED_PLAYER } from './constants.js'
import type {
TraceOpenProgress,
TraceParseMessage,
TraceParseRequest,
TraceParseResult
} from './trace-parse-protocol.js'

/** Runs the trace reader on a zip's bytes; rejects only if the reader itself fails. */
export type TraceParser = (
bytes: ArrayBuffer,
onPhase: TraceOpenProgress
) => Promise<TraceParseResult>

function relay(
onPhase: TraceOpenProgress,
settle: (result: TraceParseResult) => void
): (message: TraceParseMessage) => void {
return (message) =>
message.type === 'phase' ? onPhase(message.phase) : settle(message)
}

/** Parses off the main thread, so the tab stays responsive for the whole read. */
export const parseInWorker: TraceParser = (bytes, onPhase) => {
const worker = new Worker(
new URL('./trace-parse.worker.ts', import.meta.url),
{ type: 'module' }
)
return new Promise<TraceParseResult>((resolve, reject) => {
const answer = relay(onPhase, resolve)
worker.addEventListener('message', (event) => answer(event.data))
worker.addEventListener('messageerror', () =>
reject(new Error('The trace reader sent a reply this page cannot read'))
)
// The reader catches its own errors, so this is a worker that never started.
worker.addEventListener('error', (event) =>
reject(
new Error(
`The trace reader could not start${event.message ? `: ${event.message}` : ''}`
)
)
)
const request: TraceParseRequest = { bytes }
worker.postMessage(request, [bytes])
}).finally(() => worker.terminate())
}

/** The same reader in this thread, for a runtime with no Worker. */
export const parseInPage: TraceParser = async (bytes, onPhase) => {
// Loaded on demand so the live dashboard never downloads the reader.
const { parseTraceZip } = await import('@wdio/devtools-trace/trace-reader')
let data: TracePlayerData
try {
data = parseTraceZip(new Uint8Array(await file.arrayBuffer()))
} catch (error) {
throw new Error(`Not a readable trace: ${(error as Error).message}`)
const { answerParseRequest } = await import('./trace-parse.js')
return new Promise((resolve) =>
answerParseRequest(bytes, relay(onPhase, resolve))
)
}

// Gated on the build flag too, so the live dashboard's bundle never references the worker.
const defaultParser = (): TraceParser =>
HOSTED_PLAYER && typeof Worker === 'function' ? parseInWorker : parseInPage

/** Parse a trace.zip the viewer picked, entirely in this tab. */
export async function readTraceArchive(
file: Blob,
onPhase: TraceOpenProgress = () => {},
parse: TraceParser = defaultParser()
): Promise<TracePlayerData> {
onPhase('reading')
const result = await parse(await file.arrayBuffer(), onPhase)
if (result.type === 'error') {
throw new Error(`Not a readable trace: ${result.message}`)
}
const { data } = result
const { commands, consoleLogs, networkRequests, mutations } = data.trace
// A zip with no trace stream in it parses to a payload with nothing to show.
if (
Expand Down
24 changes: 17 additions & 7 deletions packages/app/src/controller/trace-open.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,26 @@
import type { ReactiveController, ReactiveControllerHost } from 'lit'

import { TRACE_OPEN_EVENT } from './constants.js'
import type {
TraceOpenPhase,
TraceOpenProgress
} from './trace-parse-protocol.js'

type Host = ReactiveControllerHost & HTMLElement

/** The hosted player's open flow: a file picked in the UI or dropped anywhere. */
export class TraceOpenController implements ReactiveController {
opening = false
phase?: TraceOpenPhase
error?: string

readonly #host: Host
readonly #load: (file: File) => Promise<void>
readonly #load: (file: File, onPhase: TraceOpenProgress) => Promise<void>

constructor(host: Host, load: (file: File) => Promise<void>) {
constructor(
host: Host,
load: (file: File, onPhase: TraceOpenProgress) => Promise<void>
) {
this.#host = host
this.#load = load
host.addController(this)
Expand All @@ -37,16 +45,13 @@ export class TraceOpenController implements ReactiveController {
this.opening = true
this.error = undefined
this.#host.requestUpdate()
// Lets "Opening trace…" paint before the synchronous unzip holds the thread.
await new Promise((resolve) =>
requestAnimationFrame(() => setTimeout(resolve))
)
try {
await this.#load(file)
await this.#load(file, this.#onPhase)
} catch (error) {
this.error = `Could not open ${file.name}: ${(error as Error).message}`
} finally {
this.opening = false
this.phase = undefined
this.#host.requestUpdate()
}
}
Expand All @@ -56,6 +61,11 @@ export class TraceOpenController implements ReactiveController {
this.#host.requestUpdate()
}

#onPhase = (phase: TraceOpenPhase): void => {
this.phase = phase
this.#host.requestUpdate()
}

#onAnnounced = (event: CustomEvent<File>): void => {
void this.open(event.detail)
}
Expand Down
19 changes: 19 additions & 0 deletions packages/app/src/controller/trace-parse-protocol.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import type { TracePlayerData } from '@wdio/devtools-shared'

/** The step an open is on; no step can say how far through it is. */
export type TraceOpenPhase = 'reading' | 'unpacking' | 'building'

export type TraceOpenProgress = (phase: TraceOpenPhase) => void

/** What the page posts to the trace reader: the zip's bytes, transferred. */
export interface TraceParseRequest {
bytes: ArrayBuffer
}

/** How a parse ended: the payload, or why the bytes are not a trace. */
export type TraceParseResult =
{ type: 'done'; data: TracePlayerData } | { type: 'error'; message: string }

/** What the trace reader posts back: any number of phases, then one result. */
export type TraceParseMessage =
{ type: 'phase'; phase: TraceOpenPhase } | TraceParseResult
24 changes: 24 additions & 0 deletions packages/app/src/controller/trace-parse.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import {
parseTraceEntries,
unzipTraceZip
} from '@wdio/devtools-trace/trace-reader'

import type { TraceParseMessage } from './trace-parse-protocol.js'

/** The trace reader's side of the protocol, run in the worker or, without one, in the page. */
export function answerParseRequest(
bytes: ArrayBuffer,
post: (message: TraceParseMessage) => void
): void {
try {
post({ type: 'phase', phase: 'unpacking' })
const entries = unzipTraceZip(new Uint8Array(bytes))
post({ type: 'phase', phase: 'building' })
post({ type: 'done', data: parseTraceEntries(entries) })
} catch (error) {
post({
type: 'error',
message: error instanceof Error ? error.message : String(error)
})
}
}
6 changes: 6 additions & 0 deletions packages/app/src/controller/trace-parse.worker.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import { answerParseRequest } from './trace-parse.js'
import type { TraceParseRequest } from './trace-parse-protocol.js'

addEventListener('message', (event: MessageEvent<TraceParseRequest>) => {
answerParseRequest(event.data.bytes, (message) => postMessage(message))
})
Loading
Loading