OpenScreen 2.1 introduces Off-Menu. The built-in functions stay: they are the menu. Next to them sits a new layer: a low-level code interface on the video itself, where an AI agent writes the function you are missing. It has far more freedom than any menu allows, and it still works inside the editor's infrastructure: the timeline, the cursor and the clicks, the zooms and the 3D camera, one renderer for the preview and the export.

Our first implementation is visual effects. You describe an effect, the agent writes it as a GPU shader, OpenScreen compiles it, and it lands on the timeline with its own controls. The preview and the export show the same pixels, on Windows, macOS and Linux.

A ripple on every click. The agent wrote it from one sentence, in two iterations.

Two halves of one release​

OpenScreen 2.1 is also the release where an agent drives the app end to end. Through its MCP server, an agent records the screen, cuts, zooms, captions, annotates, sets the format and exports, and you never open the app. Those are the editor's own functions, exposed as tools.

Off-Menu is the other half. The tools let the agent use every function the editor has. The code layer lets it write the ones the editor does not have. Together they move the limit of what the editor can do from its UI to what you can describe.

What Off-Menu means​

Ordering off the menu means asking the kitchen for a dish that isn't listed. The kitchen still cooks it, with its own tools and its own rules. That is the model. The menu is the editor's UI, a finite list of functions. The order goes to the agent. The kitchen is the editor's infrastructure, and its rules are the guardrails.

We learned quickly why the kitchen matters. An LLM writes a shader in seconds. What it cannot know is where the cursor sits in the final frame after an animated zoom, or how the recording is tilted under a moving camera. Only the editor knows that. So Off-Menu needs two things:

• A low-level surface, close to the pixels, where free code can do what no predefined function anticipated.
• A contract that hands that code everything only the engine knows, so the code stays short and the result stays correct.

And the layer is built for agents. Nobody writes code there by hand or pastes it in. The user's interface is the conversation: a coding agent writes the function through the editor's MCP tools, and only through them. That is what lets the editor put guardrails around it: every function is compiled before it is accepted, its loops are bounded, it runs on the GPU only, with no access to files or the network, and its cost is estimated from the code before it ever renders.

The idea is close to what Ink & Switch call malleable software: tools people reshape themselves, instead of waiting for their developers. The difference is in who touches the code. In malleable software, people change their tools by hand. Off-Menu keeps people out of the code: the agent is the only way in, and that is what makes the guardrails possible.

The built-in functions remain the quick path for the basics. The code layer covers the rest.

Prior art​

The idea of letting an LLM write shaders instead of picking presets has been published. A defensive publication from September 2025 describes it for images and video. It describes no editor around the idea: no timeline, no preview, no export.

AI video editing today falls into three families:

Off-Menu sits between the last two: code, inside an editor, picking up where the catalog stops. Three products come close.

• invideo's Agentic Effects, announced on September 30, 2026. Its agent assembles an adjustable effect on invideo's timeline from the editor's building blocks: layers, keyframes, rearranged footage, generated images, and, according to its FAQ, shader-based effects. The examples are editor-level treatments: a boomerang, a paper tear, a grid transition, highlight streaks. Its pages describe no layer below those blocks, only invideo's own agent builds them, and each job costs credits. It is a closed, cloud product.
• Remotion createEffect(), since version 4.0.479, defines project-specific effects in 2D, WebGL2 or WebGPU, and a June 2026 change teaches its agents to write them. Rendering goes through a browser, nothing promises the same pixels in preview and render, and Remotion's license is not an open-source one.
• Mosaic lets you build editing agents on a canvas, each node configured in plain language. Each node is still one of the editor's predefined operations.

Elsewhere, shaders extend a catalog as add-ons (DaVinci Resolve DCTLs, obs-shaderfilter), or agents write them for live video with the loop open: the MCP server of Resolume Wire writes ISF shaders it can neither compile nor render, a TouchDesigner MCP writes GLSL in a node graph with no timeline, and ShadAR has an LLM write HLSL for a headset's passthrough video.

What OpenScreen puts together:

1. A layer below the catalog: free code on the video itself, where an agent writes what no built-in function anticipated.
2. Built for any coding agent, through MCP, inside guardrails: compiled before it is accepted, bounded loops, GPU only, cost known in advance.
3. A contract that carries what only the engine knows, down to the recording's surface under the 3D camera.
4. One native renderer for the preview and the export, with measured parity.
5. A closed loop: the agent looks at frames the editor rendered before it stops.
6. Open source, free and local, on three systems.

As far as we could find on October 15, 2026, no other open-source video editor works this way. "Not found" is not "absent".

The first implementation: effects​

An effect has no rendering path of its own. It is one more pass in OpenScreen's native compositor, which already composes the preview and encodes the export.

• One path for every pixel. The compositor is written in Rust, on D3D11 (Windows), Metal (macOS) and wgpu (Linux). The preview and the export call the same compose function with the same scene. The effect is inserted once, so it shows up everywhere, GIF export included.
• A one-function contract. The agent writes fn effect(uv: vec2<f32>) -> vec4<f32>, called for every output pixel. The compositor writes the rest (inputs, bindings, entry points) and places it after the agent's code. WGSL declarations can come in any order, so compiler errors point at the lines the agent wrote. The full contract, with worked examples, is served to agents as the MCP resource openscreen://effects/contract.
• One language. The agent's WGSL is translated by naga to HLSL on Windows and to MSL on macOS; wgpu takes it as is on Linux. We ruled out three other routes:
  • Slang: its Metal and WGSL outputs were still experimental, it weighs 30 to 60 MB, and it has no maintained Rust bindings;
  • a bytecode interpreter inside a shader: no compilation, but several times slower per pixel;
  • linking precompiled functions (D3D11 function linking, Metal stitching): it only combines nodes written in advance, not free code.
• Nothing compiles while rendering. A 60-line effect compiles in 10 ms. It is compiled twice:
  • when it is written, for real, so that "accepted" means "will draw";
  • when the scene is set, on a background thread. The preview skips the effect while it compiles, then redraws. The export waits for it and never renders a frame without its effect.
• The project keeps the source. Code, name, description and declared parameters are stored, never a binary. An effect is a region on its own timeline row, anchored to clips like a zoom. Click it and the inspector shows a control for each parameter the agent declared, where the effect applies (the image, the text, or both), and its cost. Any change to the function itself goes through the agent.

The agent loops on the top row until the code compiles, then around the whole loop until the frames match the request. The bottom row is the path every frame in OpenScreen already takes.

What we measured​

On an RTX 4070 Ti (Windows 11, D3D11), the heaviest effect we tried adds 0.04 ms to a 1080p frame, and export time does not move. That effect combines chromatic aberration, barrel distortion, grain, scanlines and a vignette.

Cost is predictable from the code. An effect pays a fixed 0.03 ms for the copy and the pass, then about 4 µs per frame() read at 1080p (17 µs at 4K). That is why the cost tiers count image reads, not lines of code.

The preview and the export show the same picture. We compared the live preview's frame with the exported MP4's frame at the same instant:

• inside the effect region, the mean difference is 2.2 to 2.7 out of 255 (32 to 34 dB), the same as outside it: that is H.264 codec noise;
• against the export without the effect, it rises to 14 to 35 out of 255. So the effect is really drawn, and drawn the same on both sides.
• A measurement trap. Decoded without stating the BT.709 matrix, the MP4 shows a false difference of 12 out of 255 that has nothing to do with the effect.

One agent iteration takes seconds. On the prompt "ripple effect when the mouse clicks":

1. Broken code is refused in 7 ms, with naga's error at the agent's own line and column, here a typo in strenght.
2. The first version compiles on the first try, but draws a single glowing ring, closer to a halo.
3. The second version sends concentric waves out from the click, refracting the picture. It matches the prompt.

The agent saw nothing but the tool's description. Each iteration chained writing, compiling, rendering and looking at a few frames.

What the contract learned​

The first real prompt exposed a hole in the contract: a ripple on click could not be written. The effect saw the picture and the time, but not where the mouse was in the final frame.

The agent cannot compute that position. Between the recorded mouse and the output pixel sit the animated zoom, the layout, the frame and the optional 3D tilt. Only the compositor knows all of them, at every frame. So the cursor tip and the latest clicks went into the contract, already placed in output coordinates. The ripple then took two iterations.

The second lesson came from a water drop. Under the orbit camera, the recording is a tilted plane, and a wave computed in output coordinates drew perfect circles floating over it. The contract now exposes the recording's surface itself, through the exact warp the compositor draws it with: bilinear under a fixed angle, the homography of its corners under the real camera. fx_to_footage(uv) and fx_from_footage(p) map between the picture and that surface. The contract decides nothing: the effect picks the space that fits it. The water drop now lies on the recording, in perspective, and a vignette still works on the whole frame.

The rule that came out of it: what only the compositor knows goes into the contract, and the contract informs without deciding. Version 1 therefore provides, on top of the picture and the time:

• the cursor and the four latest clicks, in output coordinates;
• where the recording and the webcam sit in the final frame, and the recording's own surface, through its warp;
• the effect's progress across its whole region, even when a cut runs through it;
• named, typed parameters (params.radius rather than an index), so an agent rereads an effect without decoding it, and the inspector knows which control to show;
• deterministic randomness, identical on every GPU.

That last point is a lesson of its own. The classic shader hash fract(sin(x) * 43758.5453) gives different values from one GPU to another, so a project opened on another machine would no longer have the same grain.

Limits and open questions​

The real risk of agent-written code is not its cost. It is a loop that never ends: it hangs the GPU until Windows resets it, and the app loses its device.

• GPU hangs. The guard is static: every loop needs a literal bound, with a ceiling of 4,096 iterations per pixel. Recovering a lost device is a separate project that concerns the whole app.
• Cost is not capped. A budget measured on the machine that writes the effect would refuse a heavy effect on a modest laptop, even though the export would render it fine. So cost is estimated from the code: image reads per pixel, multiplied by loop bounds. That gives a tier, identical on every machine, which the agent receives with a recommendation.
• Pixels across machines. Floating-point results can differ slightly between DXBC, MSL and SPIR-V, as for the rest of the rendering. Strict equality holds on one machine.
• The translator's version. From naga 25 on, HLSL output declares samplers in a D3D12 heap, which FXC rejects for D3D11 (error X3721). We stay on naga 24, and a test guards that line.
• The scope of version 1. An effect acts on the whole composed picture, on the text layer alone, or on both. Three things stay out of reach for now:
  • effects on one layer only, the screen or the webcam;
  • transitions, which need two pictures as input;
  • temporal effects (trails, persistence), which would need previous frames as explicit inputs, never kept state.
• Shared code. Opening someone else's project compiles code you did not write. It runs only on the GPU, with no access to files, the network or the app's memory, and the same guards apply when the project opens.
• Measurements. The numbers above come from one Windows machine. The Metal and wgpu passes ship in the same release; we have not benchmarked them on comparable hardware yet.

Where Off-Menu goes next​

Effects are the first function we opened to code. A shader computes each pixel from pictures and time, so anything that can be described that way can move from the menu to a request, as long as the contract gives it the right inputs.

Beyond pixels. Cuts, timing and the order of clips are functions too. Today the agent drives them through the editor's built-in tools, and they do the job. Opening them to free code the same way, with the timeline as the infrastructure, is where the idea goes next: an edit where any function, not only the look, can be written on demand.

What stays out. Understanding the content (tracking a face, reading on-screen text) needs a model; the code consumes its result as a mask or coordinates. Inventing pictures that are not in the footage is the generative family's job, with its price: no determinism, and no source to edit.

The model that comes out of it. An edit becomes the editor's built-in functions plus functions written on demand. Between the two, a contract supplies everything only the engine knows. The agent works on both through the same interface, and every edit stays an object you can reread, adjust or delete. The menu stays for the basics. It stops being the edge of what an edit can do.

What would prove it better than a demo. A benchmark of effect prompts, scored on the rendered result like ShaderMatch, with two checks we already know how to run:

• parity between preview and export;
• an agent that looks at its own frames before it stops.

Try it​

OpenScreen 2.1 is free and open source, on Windows, macOS and Linux: getopenscreen.com.

• In the app, connect a provider in AI settings and ask the chat for an effect.
• From Claude Code, paste: Install the OpenScreen demo skill with npx skills add getopenscreen/openscreen --skill openscreen-demo -a claude-code, then follow it to make a demo video of this feature.
• Docs: Custom effects and AI editing.
• For agent authors: the contract is the MCP resource openscreen://effects/contract, and the design is in docs/shader-effects.md.

Sources​

Pages opened between October 13 and 15, 2026.

The idea

• Malleable software: restoring user agency in a world of locked-down apps, Ink & Switch, 2025
• AI-driven Special Effects Generation and Application Framework for Visual Content, Colvin Pitts, TDCommons, September 29, 2025

Research

• Timeline-Bench, arXiv, September 28, 2026
• ShaderMatch, LLM4Code @ ICSE, May 2025
• ShadAR, arXiv, February 2026
• Code2Video, October 2025
• ELLMPEG, MMSys 2026
• EditDuet, SIGGRAPH 2025
• LAVE, IUI 2024

Tools and products

• invideo Agentic Effects, September 30, 2026, its custom effects guide and its FAQ
• Remotion createEffect(), and the change that opens it to agents
• Mosaic
• HyperFrames and its shader transitions
• Resolume Wire MCP server
• TouchDesigner MCP
• DaVinci Resolve DCTL
• obs-shaderfilter
• Runway Aleph
• Google Vids and Gemini Omni
• Descript Underlord
• Kapwing Kai
• OpusClip MCP

OpenScreen

• The effects design: docs/shader-effects.md
• The compositor and the WGSL contract: crates/compositor/src/shader_fx.rs

The result was two rules kept out of ten, and thirty findings. The fixes are gathered in one branch, #814, which is what v2 will be.

The audit​

Ten rules, each written as something a user should be able to rely on. Defaults that look good. Every value bounded, with a way back to its default. A format change or a cut that breaks nothing. What you see in the preview is what you export.

Measured on the build of that day, the defaults failed first:

• The default cursor was 82 px tall in a 1080p export, about four times the system arrow, and it could be pushed to 272 px.
• The default shadow drew at 9% opacity, which on most wallpapers means no shadow at all.
• A 16:9 recording exported as 9:16 filled 20% of the frame.

None of those is a bug in the usual sense. Each one is a setting that works as written and makes the video worse.

Twenty-one pull requests, one branch​

The findings split into six workstreams, and each became a pull request against a shared integration branch instead of main. They touch the same panels and the same compositor, and landing them one by one would have meant a week of broken intermediate states on main. Merged into one branch in dependency order, they can be tested together and land together.

Defaults. The cursor is half its old size, 41 px in 1080p. The shadow is at 27% opacity. New projects start from the look of your last one.

Bounds in the model. One table clamps every appearance value, whoever writes it: the panel, the AI agent, a preset or a hand-edited project file. Every slider has a reset.

The render tells the truth. Two findings were the kind you only see by measuring. Every text font exported as Segoe UI, pixel for pixel, because the picker offered fonts the app never shipped; five open-licensed families are embedded now. Gradients used only their first and last colour stops, so a red, green and blue gradient drew purple at its midpoint where green belonged.

Export. The bitrate the app computed never reached the encoder, so every export ran at 8 Mb/s whatever the frame rate. It is 18.7 Mb/s at 1080p60 now. GIFs are dithered, and the dialog offers four destinations (Web / YouTube, Social, Studio, README GIF) with the codec details folded under Advanced.

Audio. Voice is normalised to −16 LUFS with a soft limiter instead of a hard clip, and music ducks under it.

Framing. A 16:9 take in a 9:16 export now fills 64% of the frame, in a window that follows the cursor. Zooms are capped at twice the source's own resolution, so a deep zoom no longer magnifies blur. Silence cuts keep 70 ms of breath on each side, so a cut doesn't clip the start of a word.

One thing went the other way. The 17 third-party cursor packs are gone: their terms don't allow redistribution. Five original themes replace them.

Then the interface​

With the rules holding, the panels were the next thing to make ugly by accident. The branch this post was written on carries about a hundred and twenty more commits, and most of them delete something.

One click per choice. The selects and dropdowns in the right panel became rows of buttons, so every option is visible and picking one is a single click: format, frame style, the zoom's 3D camera, focus mode, cursor style, caption language and font. Only the fifteen translation targets stay a list.

Names instead of numbers. A shadow is None, Light, Medium or Strong. Click bounce is None, Light or Strong. A slider stays where the amount is the point (padding, background blur, cursor size), and a slider with no unit shows no number.

No explanations under settings. If the label needs a sentence to be understood, the label is wrong. An option that does nothing in the current context is hidden instead of greyed out with its reason.

Fewer controls. The Fit button rewrote four settings without naming any, and padding at 0 already does its job, so it's gone. The seek bar under the transport repeated the ruler right below it, so that went too.

A timeline you read by colour. The recording is the main object on the timeline, so it wears the brand's green. Zoom pills are blue, so the most common effect never reads as the clip under it. Every block is a solid card with dark text, because mint can't carry white: 2.3:1 contrast with white text, 8.2:1 with dark ink.

The walkthrough on the home page and this site's docs were redrawn to match.

The 3D camera​

The 3D work followed the same rule. A camera that rolls during a zoom looks broken rather than cinematic, so a moving camera never rolls now. The fixed angles went through three versions in a day before landing back on the v1.13.0 angle, checked against a real v1.13.0 export rather than from memory. The follow camera became 3D Orbit, which obeys the zoom's focus mode like the flat zoom does. The 3D cursor dives onto each click in depth, and the click impact that used to live on the zoom is a cursor setting for every camera.

Before v2 ships​

#814 is open, and green except for one compositor test on Linux and macOS that checks the 3D cursor's footprint. After that, it needs a manual pass on real Windows and macOS machines, then the usual release candidates.

It is still not production-grade, and rough edges are expected. If a default still makes your demo worse, that is exactly the bug this release is about: Discord or an issue, either works.

So the comparison is public now: screen-recorder-benchmark. Same clip, same edit, same machine, across the desktop apps built to turn a recording into a finished demo. OpenScreen is one of the tools in it, alongside Cap, FocuSee, Recordly and Screen Studio, and the standings are whatever the submissions say.

The current standings (lower is better)​

Read the ratio, not seconds. Each figure is an export divided by what a plain ffmpeg transcode of the same clip needed on the same machine, measured minutes earlier. A tool at 1.5× did 50% more work than a bare re-encode, on whatever hardware you have. Seconds only ever compare a machine to itself.

The scenario is a finished demo rather than a transcode: a sampled wallpaper, padding, rounded corners, a drop shadow, three animated zooms, motion blur, a cursor redrawn from telemetry, a webcam inset with mask and shadow, and the recording's audio, all pinned to 1920x1080 at 60 fps in H.264. How the current OpenScreen release renders that is in the export docs.

Four decisions that make it hard to fake​

ffmpeg is the 1x, and it is not a competitor. The reference is re-measured on your machine, under your load, immediately before each tool runs. Nothing is normalised against a number I recorded once on a laptop I own.

No tool is required in a submission. Tools are nodes, every pair measured together on one machine is an edge, and the ranking is recovered by least squares across the whole graph. Two people who never measured the same app still contribute to the same ordering, as long as the submissions overlap somewhere.

The verifier decides what happened, not the app. Every export is re-probed before it counts: resolution, frame rate, codec and duration against the target, then wallpaper, padding, corner radius, zooms, cursor and camera checked in the pixels, then audio by loudness, because a silent track passes any check that only asks whether audio exists. A tool reports what it configured. The verifier reports what came out, and the verifier wins.

The disagreement is published above the ranking. Where one pair of tools was measured on more than one machine, the two ratios can differ by more than the gap the ranking puts between them, and that spread is printed at the top of the page rather than in a footnote. A single global order is not yet what this data supports, and the page says so before it shows you one.

Roster membership follows what a tool is for, not whether I managed to automate it. An app I cannot drive is an empty cell, and an empty cell is a result.

Add a machine​

The hardware coverage is thin, and it shows in the spread above. A run needs any two tools and takes one command:

node bench.mjs run --bundle commons-upload --apps <two or more tools>
node bench.mjs submit --run <runId> --as "your name" > submissions/<platform>/<chip>-<date>.json

Open a pull request. CI validates it against the schema and regenerates the site, so the table on this page moves as soon as it merges. The footage comes from permanent Wikimedia URLs with a hash, so two machines can prove they measured the same thing. It is MIT, and the results page is the repo's own output: EtienneLescot/screen-recorder-benchmark.

Building it this way meant accepting the answer before knowing it, on every machine anyone adds. A benchmark whose author always wins is a marketing page.

That is v1.10.0, August 24. Before it, v1.9.0 through v1.9.6 in under two weeks.

Losing a recording used to be possible​

A screen recorder that can lose the take is not a screen recorder. The old capture path wrote a single MP4 whose index is finalised at the end, so a helper crash, a forced quit or a dead battery left a file no player could open. The recording had happened. It was just unreadable.

Capture on Windows and macOS writes fragmented MP4 now. The file is valid at every fragment boundary, so whatever was captured before the process died is still there and still plays. Linux still writes a plain MP4, so a crash there can still cost the file.

The same release stopped two other ways to lose work. The native webcam stream writes to disk while it records, so killing a recording can't take the camera track with it, and the Windows capture helper stopped hanging on stop, which used to require killing the process and losing the file.

Where you can install it now​

Most of v1.10.0 was packaging. Distribution is where free desktop software quietly fails: the build works, and nobody can get it.

Windows. The appx goes to the Microsoft Store from the release build, with branded tiles. That is the route I'd recommend on Windows now, and the installation docs walk through it. winget stopped skipping silently. There is no Visual C++ Redistributable dependency any more, and the OpenMP runtime the transcription backends actually import is bundled, so transcription works on a clean machine.

Linux. Fedora RPM (@Mundo-Dev0ps), a source build that completes on ARM64 machines (@zebster-cmd), and DMA-BUF negotiation so capture works on niri and other wlroots compositors, which had no working screen recorder from this project at all. Packages now respect the glibc floor of the distros they claim to target, declared and proven on a clean machine, and AppStream metadata is in place for Flathub. The released packages, all x64, are on the download page.

macOS. The Homebrew cask job now reports that it published nothing, instead of passing silently.

Capture is also DPI-aware and stops guessing which monitor you meant, and there is a GPU DXGI encode path behind a flag, opt-in until it earns the default.

v1.9.0, three weeks earlier​

v1.9.0 shipped August 5, the day after v1.8.0 was promoted. That is a backed-up queue rather than speed: the 1.8.0 RC window ran through nine candidates while main kept moving. The frozen release branch is the only reason the two didn't tangle.

Two features in it:

• Teleprompter mode in the notes window. Your script scrolls next to the capture, mirrored so it reads right in a webcam (@My-Denia).
• A CLI with record, export and info, driving the same engine the app does, which makes OpenScreen usable from a script (@PeterTakahashi). Recording still needs a real desktop session, as the CLI docs explain.

It also removed a PID-file instance lock that could permanently brick startup, notarized macOS RCs like stable builds, and made the AppX package declare all 13 locales instead of one.

What changed in the patch releases​

v1.9.1, v1.9.2, v1.9.5 and v1.9.6 went out over ten days. 1.9.3 and 1.9.4 never left RC and their fixes rolled forward.

Roundness, shadow and motion blur are on by default now, which is the look most people were assembling by hand on every project. The Windows cursor sampler became DPI-aware. The chat agent learned addTrims and addZooms, so asking it for a cut stopped costing a round trip through a slower path.

Every release's regression pass is written up in the repo's testing docs, including the ones that caught a candidate breaking something. This site was rebuilt in the same window, and gained its light theme.

On main right now​

Webcam background effects have landed on all three compositor backends. Blur or replace what is behind you, including an AI cutout that doesn't need a green screen. The transcription helper reports its real timing and which compute backend it used, so "how long will this take" has an answer instead of a progress bar with no scale.

Still open: hardware encode on Linux, and measurements on discrete GPUs and QSV. It is still not production-grade, so rough edges are expected and bug reports are welcome.

Three months, ten releases. Discord is open if you want to argue with any of it.

v1.8.0 shipped that on August 4, alongside an optional AI editing layer. It transcribes on your own machine, and chat edits go to the LLM provider you connect.

The AI layer​

It is off by default. If you never turn it on, nothing downloads, no model is contacted, and you have exactly the 1.7.0 feature set. That was in the roadmap from week one and I don't intend to change it.

Turn it on and you get four things.

Transcription on your machine. Whisper via whisper.cpp, with Metal on Apple Silicon, Vulkan on Windows and Linux, CPU everywhere else. DTW word timestamps. Nothing is uploaded, and after a one-time model download it works offline.

Editing through the transcript. Select words, delete them, the span is cut from playback and export. Word boundaries get re-anchored against the audio so the cut lands where the word actually starts.

Captions derived from the transcript, instead of generated once and then maintained by hand. Restyle or regroup them with no regeneration step. Translation into 15 languages if you want it, through the LLM provider you connect for chat. The captions docs cover styling and translation.

Editing by chat. Describe an edit in plain language and an agent applies real timeline operations: cuts, zooms, speed ramps, annotations, camera framing. Ctrl/Cmd + Z undoes an agent edit the same way it undoes yours, with per-message rewind.

The chat uses your own LLM key. Anthropic, OpenAI, Google, Mistral, OpenRouter, MiniMax, or anything OpenAI-compatible. Keys are stored encrypted through your OS's credential protection, and requests go from your machine straight to the provider. There is no OpenScreen server in the middle, because there is no OpenScreen server. Setup is in the AI editing docs.

1.8.0 also removed the ChatGPT and GitHub Copilot sign-in options the pre-fork codebase had. Using someone's existing subscription there meant shipping GitHub's and OpenAI's own client IDs against endpoints they reserve for their own clients, from inside a signed installer. I didn't want to be in that position. If those vendors open a sanctioned surface, the integrations come back.

The compositor​

Export at 1080p60 with full effects ran at roughly 8 fps. The same project now runs at roughly 126 fps on the same laptop, an AMD Ryzen 5 7520U with its passive integrated Radeon. That machine is deliberately weak. Your number will be different. The order of magnitude is the point.

It is one GPU-resident chain with no CPU readback between stages:

demux -> hardware decode -> GPU composite -> RGB to NV12 -> hardware encode -> MP4 mux

I didn't set out to rewrite it. I first rebuilt the Canvas2D compositor to cache by what invalidates a cache instead of by layer. That was worth about 2x, and I verified it byte-identical (SSIM 1.000000 across all 1418 frames of the reference export). What it really did was make the profile readable: compositing was 79% of the time, encoding was 4.5%. The bottleneck was the compositor, running on the CPU inside a browser.

Preview and export now both consume the same SceneDescription the app builds, so there is no second renderer to drift. The frame in the editor is the frame that gets written.

Three backends​

There is a CPU backend too, software render and decode, picked automatically when there is no usable GPU. The UI tells you when you are on it, so a slow export has a visible reason.

The macOS decode detour​

Profiling the first working Metal export showed 74% of wall time in decode. The compositor was at 0.2 ms/frame. The hardware decoder pays a fixed per-frame latency, and OpenScreen's own capture writes Constrained Baseline H.264, which software decodes in a few hundred microseconds.

So the decoder is picked by profile now. Baseline goes to software, which came out 2.4x faster end-to-end on the test export (182 fps against 76). High, HEVC and 4K go to VideoToolbox, where the arbitration flips the other way.

Still open​

Hardware encode on Linux. It is correct today and slower for being software. And every number above comes from one weak laptop, so discrete GPUs and Intel QSV still need real measurements. Update, September 2026: since v1.11.0, H.264 export on Linux uses VAAPI when the driver and GPU support it, and falls back to software otherwise. H.265 on Linux is still software. The export docs cover the render path, and the download page has the current release.

Next: v1.9.0 through v1.9.6 in twelve days, and v1.10.0.

v1.6.0, so that merging is safe​

The fork inherited a codebase with no release candidate process. Merging a stranger's PR into a project with no RC step means the first person to find the regression is a user, so this came first: RC tags, a promote workflow, release notes generated from the PR history.

The Windows Graphics Capture helper, the native code that does the actual recording on Windows, got per-step stop timing and a diagnostic tool. "Recording fails on stop" is not a fixable bug report when all you know is that it failed, and it was the most common one.

The Linux HUD got its drag handle back. The overlay's pointer-events handling made the bar unmovable there, which nobody had noticed because nobody on the original team ran Linux.

Three PRs from outside landed in this one: the macOS cursor offset in single-window capture (@giulio333), copy/paste for timeline region attributes (@446f6e6e79), and a CI refactor (@psychosomat).

The release branch rule​

If you maintain something with a promote step, this is the part to steal. One release branch per stable version, cut at the RC, frozen until promote. Only cherry-picked bugfixes land on it, and anything merged to main after the cut ships in the next cycle.

I wrote that rule after the promote workflow tagged the tip of main instead of the RC snapshot people had tested. The release went out containing commits nobody had signed off on, and I replayed it. The rule is in the repo docs now and every release since has followed it.

v1.7.0​

Eight first-time contributors:

• Large recordings stream instead of loading into memory. The editor and exporter used to run out of RAM on long captures, which made the app unusable for exactly the recordings people care most about (@rainyflash).
• A software H.264 fallback for Windows machines whose hardware encoder is missing or locked, which is a large share of corporate laptops (@My-Denia).
• The webcam recorder now starts after native macOS capture does, so the first seconds of camera footage stop vanishing (@josiahcoad).
• A fix for the cursor drifting out of place after a crop (@SakuraiSatoru).
• A notes window that capture ignores, so you can keep your script beside the recording without it showing up in the video (@Itzadetunji).
• Playback speed up to 100x, for finding the one moment you want in an hour of footage (@rainyflash).
• The Full Camera timeline effect. Press a key and the webcam PiP grows to fullscreen for a stretch, then eases back (@rodrvc).
• A custom gradient editor for backgrounds (@psychosomat).

The platform work in that release was mine. Preview recovery from WebGL context loss on Linux and Wayland. Vulkan off on Wayland so PipeWire capture can import DMA-BUF frames. webm duration patching rewritten so hour-long recordings stop killing the editor on load.

Full Camera and speed regions are covered in the timeline docs, and the download page has the current build.

Why this matters more than the feature list​

An archived repo with 39k stars has a lot of people sitting on fixes they wrote for themselves and never upstreamed, because there was nobody to merge them. Reopening the repo released about a month of accumulated work in two weeks, from people who had already done it.

That is the argument for continuing an archived project rather than starting a new one. The users are already there and some of them are already contributors.

Next: v1.8.0, where an AI layer and a whole new rendering engine shipped under one version number.

I picked it up on June 15, with the original author's approval. Same name, same MIT license, new URL. This post starts a journal of what happens next.

Where it lives now​

github.com/getopenscreen/openscreen. It started as a personal fork and moved under the getopenscreen org in the first week. It stays there. Installers are on the download page, and the docs cover installing, recording, editing and export.

The archived original is still online and still read-only. Every commit of it is in this repo's history.

I tagged v1.5.0 on day one with no code changes at all. Same code, same installer, same version number. It only exists to prove the release pipeline works on my infrastructure before anything real ships through it. v1.6.0 is the first release with new work in it.

What I'm committing to​

Forking a popular archived project is a good way to quietly turn it into something else. So, written down where you can hold me to it:

• Free forever, MIT. No paid tier, no premium features, no usage caps. Nothing is gated on who you are.
• Stability before features. The recorder has to work on macOS, Windows and Linux. Bugs from real users go first.
• It's not production-grade, and I'll keep saying so. Expect rough edges and breaking changes, including to the project format.

The roadmap is public: record, edit, export, plus an optional AI editing layer that's off by default and never required. There's a Discord with a roadmap channel if you want to argue about any of it.

It already had contributors​

Four days in, before I'd shipped anything, someone I'd never met opened a fix for exports stalling when trim regions leave long decoder gaps. Real bug, correct diagnosis, regression test included.

That's roughly what the job turned out to be. Very little of it is features. Mostly it's release candidates that mean something, CI that fails loudly, and diagnostics for the parts that break on other people's machines. Native capture on Windows and macOS is as finicky as its reputation.

I'll write up what ships and what breaks here. Next: v1.6.0 and v1.7.0.