<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ishank Choudhary</title>
    <description>The latest articles on DEV Community by Ishank Choudhary (@ishank-dev).</description>
    <link>https://dev.to/ishank-dev</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1886713%2F6e03b2e8-4762-4569-9dea-f73f593e8a4a.png</url>
      <title>DEV Community: Ishank Choudhary</title>
      <link>https://dev.to/ishank-dev</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ishank-dev"/>
    <language>en</language>
    <item>
      <title>T3 Code Hands-On: A Coding Agent Restyles Its Own IDE Through MCP Theme Tools</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Wed, 18 Nov 2026 07:24:46 +0000</pubDate>
      <link>https://dev.to/ishank-dev/t3-code-hands-on-a-coding-agent-restyles-its-own-ide-through-mcp-theme-tools-4jl9</link>
      <guid>https://dev.to/ishank-dev/t3-code-hands-on-a-coding-agent-restyles-its-own-ide-through-mcp-theme-tools-4jl9</guid>
      <description>&lt;p&gt;On 14 November Theo posted that "T3 Code now lets you fully customize your UI in app. Ask your agent to make changes and it will update on the fly." The &lt;a href="https://x.com/theo/status/2121455838414201200" rel="noopener noreferrer"&gt;post&lt;/a&gt; had 5,213 likes when I checked. The feature had merged a few hours earlier in &lt;a href="https://github.com/pingdotgg/t3code/pull/35247" rel="noopener noreferrer"&gt;PR #35247&lt;/a&gt;, "agents can restyle T3 Code with theme colors and CSS".&lt;/p&gt;

&lt;p&gt;I've looked at T3 Code before as "one more UI for coding agents" and moved on. This feature is different. The agent isn't editing your project. It's editing the tool you're watching it work in. I wanted to know whether that holds up, so I ran it myself: how fast a change shows up, what survives a restart, and what happens when the agent gets the CSS wrong. It can't see the screen, after all.&lt;/p&gt;

&lt;p&gt;So I built a small lab on a headless Linux box, gave a free model seven restyle prompts, and measured every one from T3 Code's own database and a headless browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  What T3 Code is, and what's new
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/pingdotgg/t3code" rel="noopener noreferrer"&gt;T3 Code&lt;/a&gt; is an MIT-licensed TypeScript app that its README calls "the open-source control plane for coding agents." It drives agents you already have installed, such as Claude Code, Codex, Cursor and OpenCode, and shows their threads in one UI on desktop, web and phone. The repo had 63,163 stars on 17 November.&lt;/p&gt;

&lt;p&gt;The new part is a pair of MCP tools that T3 Code's server hands to the agents it runs (OpenCode saw them as &lt;code&gt;t3-code_t3_theme_read&lt;/code&gt; and &lt;code&gt;t3-code_t3_theme_set&lt;/code&gt;):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;t3_theme_read&lt;/code&gt;&lt;/strong&gt; is read-only. It returns the current theme, the five built-in themes (&lt;code&gt;t3-chat&lt;/code&gt;, &lt;code&gt;grove&lt;/code&gt;, &lt;code&gt;ocean&lt;/code&gt;, &lt;code&gt;ember&lt;/code&gt;, &lt;code&gt;iris&lt;/code&gt;), any published themes, the themes folder, and a 3,720-character styling guide.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;t3_theme_set&lt;/code&gt;&lt;/strong&gt; is marked destructive. It publishes a theme file and switches this server's web app, and the desktop app on the same machine, over to it. It can also flip light, dark or system mode.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A theme is a small JSON file. You give a &lt;code&gt;canvas&lt;/code&gt; and an &lt;code&gt;accent&lt;/code&gt; hex colour, and T3 Code derives the rest of the palette. You can also override any of 61 named colour roles, from &lt;code&gt;sidebarRowActive&lt;/code&gt; to &lt;code&gt;terminalCursor&lt;/code&gt;, and add up to 50,000 characters of custom CSS. The guide lists 29 stable selectors, such as &lt;code&gt;[data-chat-header]&lt;/code&gt; and &lt;code&gt;[data-slot="composer-host"]&lt;/code&gt;, because class names change between releases. The guide also tells the agent the important limit up front: "You cannot see the result. Make small changes, then ask the user how it looks."&lt;/p&gt;

&lt;p&gt;Per the docs, the thread has to run in &lt;strong&gt;Full access&lt;/strong&gt; mode. Mobile apps and desktop apps connected from other machines don't follow these themes.&lt;/p&gt;

&lt;h2&gt;
  
  
  My setup
&lt;/h2&gt;

&lt;p&gt;Everything ran on a Debian 13 box with 8 vCPUs, no GPU and no display. There's no desktop app in this test, only T3 Code's web mode, which is the same UI served by &lt;code&gt;t3 serve&lt;/code&gt;. I don't have Claude or Codex accounts there, so the agent was OpenCode 1.19.0 with the free &lt;code&gt;deepseek-v4.1-flash-free&lt;/code&gt; model. No paid key was involved anywhere.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm i t3@0.1.10                  &lt;span class="c"&gt;# tiny launcher; first run downloads a 171 MB binary&lt;/span&gt;
npx t3 --version                 &lt;span class="c"&gt;# t3 v0.1.10, 8.1 s the first time&lt;/span&gt;

&lt;span class="nb"&gt;cd&lt;/span&gt; ~/code/t3demo                 &lt;span class="c"&gt;# an empty git repo to act as the project&lt;/span&gt;
t3 serve --base-dir ~/t3lab --port 3773 --host 127.0.0.1
&lt;span class="c"&gt;# first boot runs 96 DB migrations: listening in ~2.5 s; restarts: 1.76 s&lt;/span&gt;
t3 project add --base-dir ~/t3lab ~/code/t3demo
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;OpenCode is off by default. I turned it on in the server's &lt;code&gt;settings.json&lt;/code&gt; (&lt;code&gt;userdata/settings.json&lt;/code&gt; under the data directory), which T3 Code picked up without a restart:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"providerInstances"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"opencode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"driver"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"opencode"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"enabled"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The best surprise of the setup is that T3 Code has a real CLI for threads, so I didn't have to click through the UI to run prompts. Each prompt was one command:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;t3 thread start --base-dir ~/t3lab . \
  --provider opencode --model opencode/deepseek-v4.1-flash-free \
  --wait --json &lt;span class="s2"&gt;"Restyle T3 Code itself using your T3 Code theme tools ..."&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To measure what actually reached the screen, I paired a headless Chrome page with the server and left it open on the app. A small Playwright script polled the page's computed styles about every 100 ms and logged each change with a timestamp. Tool start and finish times came from T3 Code's own SQLite projection of the thread. So the latency number below is: first visible style change, minus the moment the first successful &lt;code&gt;t3_theme_set&lt;/code&gt; call completed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run 1: dark, high-contrast
&lt;/h2&gt;

&lt;p&gt;The first prompt asked for a near-black canvas, pure white text and a bright yellow accent, published as &lt;code&gt;hc-dark&lt;/code&gt;. The agent read the guide once, then published the whole theme in one &lt;code&gt;t3_theme_set&lt;/code&gt; call: canvas &lt;code&gt;#0a0a0a&lt;/code&gt;, accent &lt;code&gt;#ffd400&lt;/code&gt;, 60 colour overrides, and some CSS for yellow focus rings and a yellow composer border. (It made a second call 5 s later that just switched to the same theme again.) This is a trimmed copy of the first call's input:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"themeId"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"hc-dark"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"appearance"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"dark"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"theme"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"High Contrast Dark"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"appearance"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"dark"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"canvas"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"#0a0a0a"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"accent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"#ffd400"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"colors"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"#ffffff"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"focus"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"#ffd400"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"sidebarRowSelected"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"#262626"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"css"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;".dark [data-slot=\"composer-host\"]:focus-within { border-color: #ffd400 !important; }"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The tool call itself took 37 ms on the server. My open browser page switched from the stock light theme to dark 171 ms after the call completed, with no reload. The whole prompt took 44.2 s, and 34 s of that was the model thinking before its first write. It then ended with "I can't see the result — how does it look?", which is exactly what the guide tells it to do.&lt;/p&gt;

&lt;p&gt;I ran the same prompt twice more with new theme IDs, to check whether it was repeatable rather than lucky:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Run 2&lt;/strong&gt; took 52.2 s. The first &lt;code&gt;t3_theme_set&lt;/code&gt; failed validation with "Give &lt;code&gt;themeId&lt;/code&gt;, &lt;code&gt;appearance&lt;/code&gt;, or both. &lt;code&gt;theme&lt;/code&gt; needs a &lt;code&gt;themeId&lt;/code&gt;." The model had left out the ID. It read the error, retried with the ID, and the change appeared 171 ms after the good call.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run 3&lt;/strong&gt; took 19.3 s. It went straight through, and the screen changed 34 ms after the call.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So 3 of 3 runs produced a working high-contrast theme, and one needed a retry it handled by itself.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run 2: my brand colours
&lt;/h2&gt;

&lt;p&gt;Next I asked for a deep navy canvas (&lt;code&gt;#0e1a2b&lt;/code&gt;), an orange accent (&lt;code&gt;#ff6b2c&lt;/code&gt;) and orange highlights on the active thread in the sidebar, published as &lt;code&gt;ishank-brand&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;This was the slowest run, at 101.0 s, because the model spent 27 s in one reasoning block planning the palette. The write itself took 47 ms, and the browser changed 172 ms later. The navy and orange looked right, and the active thread row turned a warm orange-brown.&lt;/p&gt;

&lt;p&gt;Then I checked how the active row got that colour. The agent had done two things. It set the &lt;code&gt;sidebarRowActive&lt;/code&gt; colour role, and it wrote CSS that guessed at the row's state attributes: &lt;code&gt;[data-thread-item][aria-current="true"]&lt;/code&gt;, &lt;code&gt;[data-active="true"]&lt;/code&gt;, &lt;code&gt;[data-state="active"]&lt;/code&gt; and &lt;code&gt;[data-selected="true"]&lt;/code&gt;. I queried the live page, and none of those four selectors matched a single element. The rows don't carry any of those attributes. The orange inset bar in that CSS never rendered. The colour role did all the visible work, and the CSS was dead code.&lt;/p&gt;

&lt;p&gt;That's the pattern for the rest of this test. When the agent sticks to the colour roles, it works. When it guesses at the DOM, it's guessing blind.&lt;/p&gt;

&lt;h2&gt;
  
  
  Run 3: a layout tweak
&lt;/h2&gt;

&lt;p&gt;Third, I kept the brand colours and asked for three layout changes: a 72 px chat header, 24 px rounded corners on the composer, and a sidebar of about 240 px. The agent read the guide, used a shell command to read its own theme file so it could keep the colours, and updated the theme in place. It took 20.3 s.&lt;/p&gt;

&lt;p&gt;I measured the DOM before and after:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Chat header:&lt;/strong&gt; 52 px before, 72 px after. This one worked.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Composer corners:&lt;/strong&gt; the &lt;code&gt;composer-host&lt;/code&gt; element now computed a 24 px radius, but the card you actually see is drawn by an inner element that stayed at 16 px. There was no visible change.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sidebar:&lt;/strong&gt; the outer &lt;code&gt;[data-slot="sidebar"]&lt;/code&gt; dropped from 256 px to 240 px. The &lt;code&gt;sidebar-container&lt;/code&gt; and &lt;code&gt;sidebar-gap&lt;/code&gt; inside it stayed at 256 px, and the chat column moved to start at x=240. The sidebar now overlapped the chat by 16 px. The layout was broken.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So one of the three layout changes worked as asked, one did nothing you could see, and one broke the page.&lt;/p&gt;

&lt;h2&gt;
  
  
  Asking it to fix its own bug
&lt;/h2&gt;

&lt;p&gt;This is the part I most wanted to test, because it's what you'd actually do. I told the agent exactly what was wrong: the outer sidebar is 240 px, the inner container is still 256 px, and they overlap by 16 px. Then I asked it to fix that.&lt;/p&gt;

&lt;p&gt;It answered in 16.2 s: "Fixed: &lt;code&gt;[data-slot="sidebar-inner"]&lt;/code&gt; now shares the same 240px sizing as the outer sidebar, with &lt;code&gt;border-box&lt;/code&gt; and &lt;code&gt;overflow: hidden&lt;/code&gt; so nothing spills."&lt;/p&gt;

&lt;p&gt;It wasn't fixed. The container was still 256 px. This is the CSS it wrote:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar-inner"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;240px&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;min-width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;240px&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;max-width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;240px&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;flex-basis&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;240px&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="cm"&gt;/* matches nothing: one element can't have two different data-slot values */&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar-inner"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;box-sizing&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;border-box&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nl"&gt;overflow&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;hidden&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The guide names &lt;code&gt;sidebar-inner&lt;/code&gt; as "sidebar surface", so the model used it. But the element that sets the width is &lt;code&gt;sidebar-container&lt;/code&gt;, which the guide doesn't list. The second rule chains two different &lt;code&gt;data-slot&lt;/code&gt; values onto one element, so it can never match. The model reported success because the tool call succeeded, but it had no way to check what the page actually did.&lt;/p&gt;

&lt;p&gt;To be sure the platform wasn't the problem, I fixed it myself. I added one rule to the theme file and renamed it into place, as the docs recommend:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar-gap"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;data-slot&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"sidebar-container"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;240px&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;important&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No command was needed. The open page picked up the file change within 3 s, and the sidebar, gap and container all measured 240 px with the chat column starting at x=240. The pipeline was fine. The agent was missing feedback.&lt;/p&gt;

&lt;h2&gt;
  
  
  Does it stick?
&lt;/h2&gt;

&lt;p&gt;Yes. I stopped the server and started it again (1.76 s to ready). &lt;code&gt;t3 theme show&lt;/code&gt; still reported &lt;code&gt;ishank-brand&lt;/code&gt; as the environment theme. The already-open page reloaded into it with the 72 px header and 240 px sidebar intact. Then I opened a brand-new browser profile, paired it with the restarted server, and it loaded straight into the brand theme. Themes are plain files in &lt;code&gt;userdata/themes/&amp;lt;id&amp;gt;.json&lt;/code&gt;, and the selection is saved in &lt;code&gt;settings.json&lt;/code&gt;. That makes them easy to diff, back up or commit.&lt;/p&gt;

&lt;h2&gt;
  
  
  Undo works, mostly
&lt;/h2&gt;

&lt;p&gt;Last, I asked the agent to switch back to the stock theme. It took 12.4 s with one write, and the page changed 122 ms later to the built-in &lt;code&gt;t3-chat&lt;/code&gt;. But it stayed in dark mode. My session had started in light mode, and nothing put that back. The four themes the agent had published were also still in the themes folder. There's no delete tool, only read and set, so cleanup is up to you.&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;High contrast, 3 runs:&lt;/strong&gt; 44.2, 52.2 and 19.3 s. 2, 2 (1 failed) and 1 writes. On screen 171, 171 and 34 ms after the write. 3 of 3 applied.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Brand colours:&lt;/strong&gt; 101.0 s, 1 write, on screen after 172 ms. The colours applied; the CSS matched nothing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Layout tweak:&lt;/strong&gt; 20.3 s, 1 write. 1 of 3 changes visible, and the sidebar broke.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fix the sidebar:&lt;/strong&gt; 16.2 s, 1 write. It said "Fixed". It wasn't.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Back to stock:&lt;/strong&gt; 12.4 s, 1 write, on screen after 122 ms. It switched but left dark mode on.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Across the five colour changes, a median of 171 ms passed between the tool finishing and the screen changing, and the slowest took 172 ms. The &lt;code&gt;t3_theme_set&lt;/code&gt; call itself never took more than 55 ms. Almost all of each prompt's time was the model thinking, which is about the model, not T3 Code.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd tell my team
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The plumbing is solid.&lt;/strong&gt; Every write went live on an already-open page without a reload. The theme survived a restart and reached a brand-new client. Validation caught a bad call and gave an error message the model could act on. Themes are plain JSON files you can review in a pull request. For a feature that's less than a week old, that's a good base.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Use colour roles, not CSS.&lt;/strong&gt; The guide says "Prefer colors over CSS", and my results back that up. All 5 colour changes landed. Of the 4 CSS changes I checked, only the taller header worked. The active-row highlight and the composer corners did nothing visible, the sidebar broke, and the follow-up fix didn't fix it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The agent can't check its work.&lt;/strong&gt; That's the real gap, and the guide is honest about it. A model that can't see the result will report success when the tool call succeeds. It did that once here, after I'd told it exactly what was broken. A tool that returns the computed size of a selector, or a screenshot, would close most of that gap. Until then, a person has to look.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I couldn't test:&lt;/strong&gt; the desktop app, macOS and Windows (my box is headless Linux), Claude Code and Codex (no accounts), any model stronger than a free one, mobile and remote clients, background images, and whether the tools refuse to run outside Full access mode. A stronger model may guess the DOM better. It still can't see it.&lt;/p&gt;

&lt;p&gt;If you want to try it yourself, these are the same commands I ran, without my lab's &lt;code&gt;--base-dir&lt;/code&gt; (enable OpenCode in &lt;code&gt;settings.json&lt;/code&gt; first, as above):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx t3@0.1.10 serve --port 3773          &lt;span class="c"&gt;# open the pairing URL it prints&lt;/span&gt;
npx t3@0.1.10 project add .              &lt;span class="c"&gt;# in a second terminal, inside your repo&lt;/span&gt;
npx t3@0.1.10 thread start . --wait \
  --provider opencode --model opencode/deepseek-v4.1-flash-free \
  &lt;span class="s2"&gt;"Use your T3 Code theme tools to restyle T3 Code. Change colours only, no CSS."&lt;/span&gt;
npx t3@0.1.10 theme &lt;span class="nb"&gt;set&lt;/span&gt; t3-chat          &lt;span class="c"&gt;# undo: back to a built-in theme&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All numbers come from my own runs on 17 Nov 2026 (IST) with T3 Code v0.1.10 and OpenCode 1.19.0. T3 Code ships fast, so some of these gaps may be closed by the time you read this.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/t3-code-hands-on-a-coding-agent-restyles-its-own-ide-through-mcp-theme-tools-a9fae1b261fc" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>mcp</category>
      <category>opensource</category>
      <category>typescript</category>
    </item>
    <item>
      <title>AWS Emulator Hands-On: Test S3, Lambda and DynamoDB Locally With a Coding Agent</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Tue, 17 Nov 2026 07:22:32 +0000</pubDate>
      <link>https://dev.to/ishank-dev/aws-emulator-hands-on-test-s3-lambda-and-dynamodb-locally-with-a-coding-agent-35bb</link>
      <guid>https://dev.to/ishank-dev/aws-emulator-hands-on-test-s3-lambda-and-dynamodb-locally-with-a-coding-agent-35bb</guid>
      <description>&lt;p&gt;On 11 November AWS shipped AWS Emulator, a local copy of the AWS APIs that you run on your own machine. The pitch is aimed squarely at coding agents: let the agent build and test against something that behaves like AWS, with no account, no credentials and no bill. Swami Sivasubramanian's &lt;a href="https://x.com/SwamiSivasubram/status/2120579757544755272" rel="noopener noreferrer"&gt;launch post&lt;/a&gt; passed 1,700 likes on X, and the &lt;a href="https://news.ycombinator.com/item?id=50456631" rel="noopener noreferrer"&gt;Hacker News thread&lt;/a&gt; went past 960 points in under two days.&lt;/p&gt;

&lt;p&gt;LocalStack and moto already cover local AWS testing, so I wanted to know three things. Does it start fast enough to be part of a test loop? Can a coding agent actually ship something against it without me helping? And where does it stop looking like AWS? So I ran it in Docker on a Linux box, had an agent build a small S3 → Lambda → DynamoDB service three times from scratch, and then went looking for the edges myself.&lt;/p&gt;

&lt;h2&gt;
  
  
  What AWS Emulator is (from the repo and docs)
&lt;/h2&gt;

&lt;p&gt;AWS Emulator (&lt;a href="https://github.com/aws/aws-emulator" rel="noopener noreferrer"&gt;github.com/aws/aws-emulator&lt;/a&gt;, &lt;a href="https://docs.aws.amazon.com/emulator/latest/userguide/what-is-service.html" rel="noopener noreferrer"&gt;docs&lt;/a&gt;) is a single Rust binary, &lt;code&gt;aws-emulator&lt;/code&gt;, that serves AWS service APIs over HTTPS on &lt;code&gt;127.0.0.1:9297&lt;/code&gt;. A few facts worth knowing up front:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It's free and open.&lt;/strong&gt; The code is CC0 1.0 (public domain), except the &lt;code&gt;vendor/&lt;/code&gt; framework crates, which are Apache 2.0. The repository doesn't take pull requests; you file issues instead.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It runs three ways.&lt;/strong&gt; On Linux you install an RPM or Debian package. On macOS and Windows you build a Docker image from a short Dockerfile that installs that same package (AWS doesn't publish a prebuilt image). For Rust tests there's an embedded mode that runs in-process.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Coverage comes from AWS's own API models.&lt;/strong&gt; The repo has 437 service crates, one per API model. Each release publishes fidelity scores. The v1.0.338 scores I downloaded cover 355 services with 43,548 scored tests, and 26,001 of them (59.7%) pass.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Agents built it.&lt;/strong&gt; AWS's &lt;a href="https://www.aboutamazon.com/news/aws/how-autonomous-ai-agents-built-aws-emulator" rel="noopener noreferrer"&gt;own write-up&lt;/a&gt; says autonomous agents wrote every line, with agent teams writing fidelity tests against real AWS, filing issues and fixing them. You can see this in the repo: &lt;code&gt;issues/things-that-do-not-work/&lt;/code&gt; held 6,936 files on the day I tested, and &lt;code&gt;issues/things-that-now-work/&lt;/code&gt; held 13,224.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;There's an agent skill.&lt;/strong&gt; &lt;code&gt;aws agent-toolkit add-skill --skill-name aws-emulator&lt;/code&gt; installs a skill that teaches an agent how to install, start, connect and troubleshoot the emulator.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Telemetry is on by default.&lt;/strong&gt; The docs list exactly what's collected (operation counts, error categories, SDK names and versions; no payloads, resource names or credentials). You turn it off with &lt;code&gt;--telemetry off&lt;/code&gt; or &lt;code&gt;EMULATOR_TELEMETRY=off&lt;/code&gt;. I ran with it off the whole time.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  My setup
&lt;/h2&gt;

&lt;p&gt;I tested version 1.0.338 (released 12 Nov) on a Debian 13 box with 8 vCPUs and 15 GiB of RAM. The coding agent was OpenCode 1.19 with OpenCode Zen's free &lt;code&gt;deepseek-v4.1-flash-free&lt;/code&gt; model. I didn't use Cursor or Claude Code, and I didn't use real AWS credentials at any point.&lt;/p&gt;

&lt;p&gt;Docker needed some work first. The Debian mirror returned HTTP 500, so I installed Docker 27.3.1 from the static binaries. The box also has no iptables, so the Docker daemon ran without its bridge network and every container used host networking. That meant I couldn't test the documented &lt;code&gt;-p 127.0.0.1:9297:9297&lt;/code&gt; port publishing. I bound the emulator to loopback inside the container instead, which has the same effect on a single host.&lt;/p&gt;

&lt;p&gt;I built the image from the Dockerfile in the docs, then started it like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Dockerfile from the docs: amazonlinux:2023-minimal + the aws-emulator RPM&lt;/span&gt;
docker build -t aws-emulator .              &lt;span class="c"&gt;# 30.0 s, image is 1.27 GB&lt;/span&gt;

mkdir -p ~/.aws/emulator/9297
docker run -d --rm --name aws-emulator --network host \
  --user &lt;span class="s2"&gt;"$(id -u):$(id -g)"&lt;/span&gt; --memory=2g --memory-swap=2g \
  --cap-drop ALL --security-opt no-new-privileges --read-only \
  -e &lt;span class="nv"&gt;HOME&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"$HOME"&lt;/span&gt; -e &lt;span class="nv"&gt;EMULATOR_TELEMETRY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;off \
  -v &lt;span class="s2"&gt;"$HOME/.aws/emulator/9297:$HOME/.aws/emulator/9297"&lt;/span&gt; \
  --entrypoint aws-emulator aws-emulator -b 127.0.0.1 -p 9297
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When it starts, the emulator writes a fresh access key, an SDK config file and a self-signed CA into &lt;code&gt;~/.aws/emulator/9297/&lt;/code&gt;. Requests must be SigV4-signed with that run's key. Pointing the AWS CLI or boto3 at it takes only environment variables, with no code changes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="kd"&gt;unset&lt;/span&gt; AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_ENDPOINT_URL&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;https://127.0.0.1:9297
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_SHARED_CREDENTIALS_FILE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;~/.aws/emulator/9297/emulator_credentials
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_CONFIG_FILE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;~/.aws/emulator/9297/config
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_CA_BUNDLE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;~/.aws/emulator/9297/ca.pem
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_EC2_METADATA_DISABLED&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_DISABLE_HOST_PREFIX_INJECTION&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="kd"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;AWS_REGION&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;us-east-1

aws sts get-caller-identity   &lt;span class="c"&gt;# Account "000000000000", Arn ".../user/emulator"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Keep &lt;code&gt;127.0.0.1&lt;/code&gt; in the endpoint rather than &lt;code&gt;localhost&lt;/code&gt;. With an IP address, the SDKs put the S3 bucket name in the path, which is the only style the emulator accepts.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start-up numbers
&lt;/h2&gt;

&lt;p&gt;These are wall-clock times on the box. The full logs are in my lab notes.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Image build:&lt;/strong&gt; 30.0 s with the base image already pulled, and 28.7 s with &lt;code&gt;--no-cache&lt;/code&gt;, which re-downloads the 171.6 MB RPM.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cold start&lt;/strong&gt; (from &lt;code&gt;docker run&lt;/code&gt; to the first HTTP 200 on &lt;code&gt;/__admin/health&lt;/code&gt;): 272 ms to 1,007 ms across 10 starts, with a median of 375 ms. Three of the ten went over 500 ms, and the slowest was the very first one.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Idle memory:&lt;/strong&gt; 175.6 MiB.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Installing the skill:&lt;/strong&gt; 0.69 s. It installed for every agent it found: &lt;code&gt;~/.agents/skills&lt;/code&gt; (OpenCode, Codex, Gemini CLI and others), &lt;code&gt;~/.claude/skills&lt;/code&gt; and &lt;code&gt;~/.cursor/skills&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A sub-second start changes how you use it. You can start a clean emulator for every test run instead of sharing a long-lived one, and since state lives only in memory, a restart is also your reset.&lt;/p&gt;

&lt;h2&gt;
  
  
  The agent test: upload-and-index, three times from scratch
&lt;/h2&gt;

&lt;p&gt;I gave OpenCode one prompt and nothing else. Build a python3.12 Lambda that reads an object from S3, counts its lines and words, and writes an index item to a DynamoDB table. Give it a least-privilege IAM role and an idempotent &lt;code&gt;deploy.py&lt;/code&gt;. Write a pytest suite that uploads three files, invokes the function for each, and checks the DynamoDB items. Use the aws-emulator skill. The only hints were about my box: the image was already built, and Docker needed &lt;code&gt;--network host&lt;/code&gt;. Each run started with no emulator running and an empty folder.&lt;/p&gt;

&lt;p&gt;All three runs ended with passing tests:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Run 1:&lt;/strong&gt; 168.9 s, 27 tool calls, 3 tests passed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run 2:&lt;/strong&gt; 280.3 s, 39 tool calls, 2 tests passed (one test covers all three files, including an empty one and one with Unicode)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run 3:&lt;/strong&gt; 187.2 s, 20 tool calls, 2 tests passed (plus an idempotency test that deploys twice)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The median was 3 min 7 s. Afterwards I re-ran each suite myself against a fresh emulator, and all three passed again in 5–11 seconds.&lt;/p&gt;

&lt;p&gt;The skill made a visible difference to how the agent worked. In every run, it loaded the skill first and read the container and connect references. It started the emulator with telemetry off, and checked that &lt;code&gt;sts get-caller-identity&lt;/code&gt; returned account &lt;code&gt;000000000000&lt;/code&gt; before writing any code. All three runs also added a guard to &lt;code&gt;deploy.py&lt;/code&gt; that refuses to run unless the endpoint is loopback and the account is the emulator's. I didn't ask for that. It comes from the skill's "keep real credentials out" rules, and it's exactly the guard I'd want in a repo where an agent has a shell.&lt;/p&gt;

&lt;p&gt;This is the handler from run 1:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;boto3&lt;/span&gt;

&lt;span class="no"&gt;TABLE_NAME&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"DOC_INDEX_TABLE"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"doc-index"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;handler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"bucket"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"key"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;s3&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;boto3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;client&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"s3"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;s3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_object&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Bucket&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;bucket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Body"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"utf-8"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;item&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"pk"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"bytes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="s2"&gt;"lines"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;splitlines&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"words"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;boto3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resource&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dynamodb"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Table&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TABLE_NAME&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;put_item&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Item&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The most useful part was where the agents got stuck, because each time the emulator was behaving like AWS:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;CreateFunction&lt;/code&gt; rejected &lt;code&gt;AWS_REGION&lt;/code&gt; and &lt;code&gt;AWS_DEFAULT_REGION&lt;/code&gt; as reserved environment variables (runs 1 and 2).&lt;/li&gt;
&lt;li&gt;Calling &lt;code&gt;UpdateFunctionCode&lt;/code&gt; and then &lt;code&gt;UpdateFunctionConfiguration&lt;/code&gt; back to back returned &lt;code&gt;ResourceConflictException&lt;/code&gt; (runs 1 and 2). So did invoking a function that was still &lt;code&gt;Pending&lt;/code&gt; (run 3). Each agent fixed it by waiting for the update to finish or by retrying.&lt;/li&gt;
&lt;li&gt;Inside the function, S3 and DynamoDB calls returned &lt;code&gt;AccessDenied&lt;/code&gt; until the execution role had a policy that allowed them (seen in runs 2 and 3).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every one of those is a bug you'd otherwise find on your first real deploy. Finding them locally, in a three-minute agent run, is the whole value of the tool.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things I checked myself
&lt;/h2&gt;

&lt;p&gt;I started a second emulator on port 9300 and ran a set of probes against it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;DynamoDB tables are created asynchronously, as on AWS.&lt;/strong&gt; A &lt;code&gt;PutItem&lt;/code&gt; right after &lt;code&gt;CreateTable&lt;/code&gt; returned &lt;code&gt;ResourceNotFoundException&lt;/code&gt; while the table was still &lt;code&gt;CREATING&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;S3 → Lambda notification works.&lt;/strong&gt; With an &lt;code&gt;s3:ObjectCreated:*&lt;/code&gt; notification on the bucket, a &lt;code&gt;PutObject&lt;/code&gt; invoked the function, and the DynamoDB item appeared 515 ms later.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;DynamoDB passed everything I tried:&lt;/strong&gt; a GSI query, &lt;code&gt;TransactWriteItems&lt;/code&gt;, a conditional put that correctly failed with &lt;code&gt;ConditionalCheckFailedException&lt;/code&gt;, Streams &lt;code&gt;GetRecords&lt;/code&gt; and TTL.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The S3 basics passed too:&lt;/strong&gt; multipart upload, a presigned GET fetched with plain HTTPS, and versioning.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lambda behaved like Lambda:&lt;/strong&gt; a 2-second timeout produced &lt;code&gt;Sandbox.Timedout&lt;/code&gt; at 2.00 s, &lt;code&gt;LogType=Tail&lt;/code&gt; returned START/END/REPORT lines, and when the role allowed it, &lt;code&gt;/aws/lambda/&amp;lt;fn&amp;gt;&lt;/code&gt; appeared in CloudWatch Logs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Speed (boto3, 8 threads, same box):&lt;/strong&gt; 1,000 S3 puts of 10 KiB each took 1.44 s (693 requests/s), and 5,000 DynamoDB &lt;code&gt;PutItem&lt;/code&gt; calls took 5.18 s (964 requests/s). A Python Lambda invoke took 97–199 ms per round trip, with a median of 117 ms. A Node.js invoke took about 8 ms once warm.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Python Lambdas don't run on CPython
&lt;/h2&gt;

&lt;p&gt;Out of curiosity, I deployed a function that returns &lt;code&gt;sys.version&lt;/code&gt;. The answer was &lt;code&gt;3.14.0.alpha ... [RustPython 0.6.0 with rustc 1.99.1]&lt;/code&gt;. The container doesn't have a Python binary at all. The python3.12 runtime is RustPython, a Python interpreter written in Rust and built into the emulator. The &lt;code&gt;boto3&lt;/code&gt; your function imports is a shim that talks to the emulator in-process. The Node.js runtime isn't real Node either: &lt;code&gt;process.versions&lt;/code&gt; contains only &lt;code&gt;{"node":"22.0.0"}&lt;/code&gt;, and &lt;code&gt;process.pid&lt;/code&gt; is undefined.&lt;/p&gt;

&lt;p&gt;For the kind of code I tested, plain handlers that call AWS through the SDK, this was invisible: all three agent builds passed. But it's a real parity gap for anything that depends on CPython internals. &lt;code&gt;import sqlite3&lt;/code&gt; fails with &lt;code&gt;No module named '_sqlite3'&lt;/code&gt;, even though real python3.12 Lambdas include it. I'd expect any C-extension wheel you vendor into the zip to fail as well, though I didn't test that directly. Pure-Python packages in the zip imported fine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gaps I hit, compared with real AWS
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;S3 has hard size caps.&lt;/strong&gt; The 1,001st object in a bucket failed with &lt;code&gt;ValidationException: the bucket ... already holds the maximum of 1000 objects this emulator keeps per bucket&lt;/code&gt;. A single &lt;code&gt;PutObject&lt;/code&gt; of 8 MiB worked, but 16 MiB and above returned &lt;code&gt;413 Content Too Large&lt;/code&gt;. The source code also sets caps of 64 MiB of data per bucket, and 10,000 objects and 128 MiB across all buckets. The comments say these exist to bound memory and are "NOT a fidelity claim". They're fine for unit-sized tests, but a pagination test over 5,000 keys or a large-file upload test will fail for reasons that have nothing to do with your code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;IAM is only half enforced.&lt;/strong&gt; Identity policies on a Lambda role are evaluated, which is how the agents found their missing permissions. But a new IAM user with no policies at all could still list buckets with its own access key. The docs say plainly that it doesn't evaluate IAM and resource policies the way AWS does and shouldn't be used for security testing. Believe them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The fidelity scores are uneven.&lt;/strong&gt; Counting tests that involve only one service: S3 passes 82.7% (1,014 of 1,226), DynamoDB 71.0%, IAM 71.4% and Lambda 50.1% (395 of 788). The docs say it matches AWS most closely for control-plane APIs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One account, one Region.&lt;/strong&gt; Everything runs as account &lt;code&gt;000000000000&lt;/code&gt;, and resources aren't kept separate by Region. To test several Regions you run one emulator per Region.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Some requests never reach it.&lt;/strong&gt; S3 Control requests always put the account ID in the host name, so they never reach the emulator. Operations that use host prefixes need &lt;code&gt;AWS_DISABLE_HOST_PREFIX_INJECTION=true&lt;/code&gt; (or the equivalent setting in your SDK).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Two things looked like gaps but aren't. S3 Select returned &lt;code&gt;MethodNotAllowed&lt;/code&gt;, but AWS stopped offering S3 Select to new customers in 2024. And DynamoDB's &lt;code&gt;ItemCount&lt;/code&gt; stayed at 0 after 5,000 writes, but AWS only refreshes it about every six hours.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I couldn't test
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The native Debian/RPM install (I only ran it in Docker), embedded Rust mode, macOS, Windows and arm64.&lt;/li&gt;
&lt;li&gt;Docker's default bridge networking with &lt;code&gt;-p 127.0.0.1:9297:9297&lt;/code&gt;, because my box has no iptables.&lt;/li&gt;
&lt;li&gt;CloudFormation, CDK or Terraform, container-image Lambdas, calling a function URL, multi-Region setups and long-running use.&lt;/li&gt;
&lt;li&gt;What telemetry actually sends. It was off throughout, so I only have the docs' description.&lt;/li&gt;
&lt;li&gt;Cursor or Claude Code as the agent. I used OpenCode with a free model.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Verdict
&lt;/h2&gt;

&lt;p&gt;For the job it's built for, letting an agent build and test AWS code locally, it works today. A free model plus the skill went from an empty folder to a passing S3 → Lambda → DynamoDB test suite three times out of three, with no help from me, in about three minutes each. The errors the agents tripped over were real AWS behaviours, caught before any deploy. And with a median start of 375 ms and 175 MiB of memory at idle, starting a fresh emulator for every test run is realistic.&lt;/p&gt;

&lt;p&gt;I'd add it to a repo's test loop today, with two rules. Keep test data small: fewer than 1,000 objects per bucket, and single uploads of 8 MiB or less. And don't treat a passing IAM test as proof of anything. Before I'd trust it for Python Lambdas with native dependencies, I'd want CPython instead of RustPython, or at least a clear compatibility list.&lt;/p&gt;

&lt;p&gt;If you want to reproduce this, here's the minimum:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws agent-toolkit add-skill --skill-name aws-emulator --region us-east-1
&lt;span class="c"&gt;# then ask your agent: "Install AWS Emulator in Docker with telemetry off,&lt;/span&gt;
&lt;span class="c"&gt;# point boto3 at it, and build + test &amp;lt;your service&amp;gt; against it."&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All numbers come from my own runs on 13 Nov 2026 (IST) with AWS Emulator 1.0.338. The emulator ships several releases a day, so the gaps above may already be smaller by the time you read this.&lt;/p&gt;

</description>
      <category>aws</category>
      <category>docker</category>
      <category>serverless</category>
      <category>ai</category>
    </item>
    <item>
      <title>claude-mem Hands-On: Persistent Memory for AI Coding Agents Across Sessions</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Sat, 14 Nov 2026 04:52:36 +0000</pubDate>
      <link>https://dev.to/ishank-dev/claude-mem-hands-on-persistent-memory-for-ai-coding-agents-across-sessions-2c0j</link>
      <guid>https://dev.to/ishank-dev/claude-mem-hands-on-persistent-memory-for-ai-coding-agents-across-sessions-2c0j</guid>
      <description>&lt;p&gt;A few weeks ago I tested billion-context, which squeezes a long coding-agent session so it fits a small context window (&lt;a href="https://medium.com/@ishank.iandroid/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8" rel="noopener noreferrer"&gt;my hands-on write-up&lt;/a&gt;). That covers one session. The other half of the problem shows up the next morning: I open a new session and the agent has forgotten everything, including decisions we made together yesterday.&lt;/p&gt;

&lt;p&gt;claude-mem goes after that second half. It has been around since August 2025, has 116k GitHub stars, and Medium already has several explainers on it. So I won't explain it again. I installed it, ran the same task in fresh sessions with and without it, and counted everything I could: context sent per request, wall time, tool calls, and how often the agent had to go looking for things it should already know.&lt;/p&gt;

&lt;h2&gt;
  
  
  What claude-mem is (from the repo, not the hype)
&lt;/h2&gt;

&lt;p&gt;claude-mem (&lt;a href="https://github.com/thedotmack/claude-mem" rel="noopener noreferrer"&gt;github.com/thedotmack/claude-mem&lt;/a&gt;, Apache-2.0, by Alex Newman) describes itself as "persistent memory for AI coding agents: the next session starts where the last one left off." I tested v15.18.1, released on 12 Nov 2026. It has three parts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Hooks&lt;/strong&gt; in your agent send every tool call (reads, edits, commands) to a small local worker.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An observer&lt;/strong&gt;, which is a second model, turns that activity into short "observations" and writes a summary when the session ends.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context injection&lt;/strong&gt;: the next session starts with a compact index of those observations. The agent fetches full details only when it needs them, through a three-step search (&lt;code&gt;search&lt;/code&gt;, then &lt;code&gt;timeline&lt;/code&gt;, then &lt;code&gt;get_observations&lt;/code&gt;) that the project calls mem-search. There is also a &lt;code&gt;save_memory&lt;/code&gt; tool for saving a note on purpose.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everything is stored in SQLite at &lt;code&gt;~/.claude-mem/claude-mem.db&lt;/code&gt;, with Chroma adding semantic search on top of SQLite's keyword search. The worker also serves a web viewer on localhost. The README lists Claude Code, Codex, Cursor, OpenCode and 11 more agents. The observer can run on your Anthropic plan, a hosted claude-mem plan with a 30-day trial, or your own Gemini or OpenRouter key.&lt;/p&gt;

&lt;p&gt;One honest detail from the README deserves credit: the "index rows are about a tenth the size of a full record" claim comes with the note that it "is not a measured recall or end-to-end saving." So I measured it myself.&lt;/p&gt;

&lt;h2&gt;
  
  
  My setup
&lt;/h2&gt;

&lt;p&gt;I don't have a Claude subscription, and I wanted to test it without paying for anything, so I used:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Coding agent:&lt;/strong&gt; OpenCode 1.19, with OpenCode Zen's free &lt;code&gt;deepseek-v4.1-flash-free&lt;/code&gt; model (no login needed)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Observer:&lt;/strong&gt; my free-tier Gemini key, using &lt;code&gt;gemini-flash-lite-latest&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Memory:&lt;/strong&gt; claude-mem 15.18.1, installed with &lt;code&gt;npx claude-mem install --ide opencode --provider gemini&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;CLAUDE_MEM_GEMINI_API_KEY&lt;/span&gt;&lt;span class="o"&gt;=&amp;lt;&lt;/span&gt;your Gemini key&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; npx claude-mem install --ide opencode --provider gemini
npx claude-mem telemetry disable   &lt;span class="c"&gt;# anonymous usage analytics are on by default&lt;/span&gt;
npx claude-mem start               &lt;span class="c"&gt;# worker on 127.0.0.1:37700, viewer at /viewer&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The install wasn't smooth. The plugin installed fine, but even with &lt;code&gt;CLAUDE_MEM_ONLINE_OPTIN=false&lt;/code&gt; the installer then waited for a browser login to cmem.ai. The CLI help explains why: Gemini, Codex and OpenRouter providers "need a cmem.ai sign-in"; only &lt;code&gt;claude&lt;/code&gt; and &lt;code&gt;host&lt;/code&gt; don't. I stopped the installer after the plugin step and set &lt;code&gt;CLAUDE_MEM_PROVIDER&lt;/code&gt; to &lt;code&gt;gemini&lt;/code&gt; in &lt;code&gt;~/.claude-mem/settings.json&lt;/code&gt; myself.&lt;/p&gt;

&lt;p&gt;The free Gemini tier was the bigger problem. My first observer model ran out of its 20-requests-a-day quota during the first session. claude-mem then set a 30-minute cooldown, saved in &lt;code&gt;quota-cooldown.json&lt;/code&gt;, and paused capture. The model id &lt;code&gt;gemini-3.5-flash-lite&lt;/code&gt; was rejected as invalid, so I used the &lt;code&gt;gemini-flash-lite-latest&lt;/code&gt; alias instead. If you try this on a free key, pick a lite model before you start.&lt;/p&gt;

&lt;h2&gt;
  
  
  The test
&lt;/h2&gt;

&lt;p&gt;I wrote a small invoicing repo, &lt;code&gt;ledgerlite&lt;/code&gt;: seven Python files, five unittest tests, money stored as integer paise, and a deprecated float-based &lt;code&gt;legacy_calc.py&lt;/code&gt; that looks tempting to use.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Session 1 (memory on):&lt;/strong&gt; "The GST test is failing. Find the bug, fix it, run the tests." In the same prompt I added a decision that exists nowhere in the repo: finance has decided that any discount is capped at 30%, stored as basis points, and applied to the subtotal before GST. I also said never to use &lt;code&gt;legacy_calc.py&lt;/code&gt;, and that new features need a line under "Unreleased" in &lt;code&gt;CHANGELOG.md&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Session 2 (a brand-new session, same repo, same commit):&lt;/strong&gt; "Add percentage discount support to invoices, following the finance rule we agreed last time. Add tests and run them."&lt;/p&gt;

&lt;p&gt;I ran session 2 under three conditions, three times each:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;With memory:&lt;/strong&gt; restored from the same snapshot of the memory database before each run, so every run saw identical memory&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Without memory:&lt;/strong&gt; the same prompt, with no claude-mem at all&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Without memory, re-explained:&lt;/strong&gt; the same task, with me typing the 40-word rule again, which is what you'd normally do&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The first no-memory baselines taught me something I hadn't planned for. With the plugin off, the agent still went looking for "the rule we agreed last time." It searched git history, then my home directory, and then opened &lt;code&gt;~/.claude-mem/claude-mem.db&lt;/code&gt; with Python's &lt;code&gt;sqlite3&lt;/code&gt; and read the rule straight out of the observations table. The memory is just a SQLite file that any process running as you can read, and an agent with shell access will find it. For a fair baseline I re-ran it in a &lt;code&gt;bwrap&lt;/code&gt; sandbox with no claude-mem plugin or MCP server, and with my home and workspace directories hidden.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;                        correct   wall    tool    explore  context processed
                        feature   time    calls   calls    (input + cached)
With memory  (avg of 3)   3/3     35.0s   13.0     3.3       126,307
No memory    (avg of 3)   0/3     31.2s   16.0    13.7        88,428
Re-explained (avg of 3)   3/3     34.0s   16.3    10.3        65,739
------------------------------------------------------------------
First request, empty project:
  no claude-mem            7,470 tokens
  MCP server only         12,228 tokens   (+4,758)
  plugin + MCP (full)     12,701 tokens   (+5,231)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Recall was perfect, and that is the headline.&lt;/strong&gt; In all three memory runs the agent ran one &lt;code&gt;claude_mem_search&lt;/code&gt;, then &lt;code&gt;get_observations&lt;/code&gt; on the top result, and built the feature correctly: a 3,000-basis-point cap, the discount applied before GST, a CHANGELOG line under Unreleased, no &lt;code&gt;legacy_calc.py&lt;/code&gt;, and 9 to 10 passing tests. Without memory, none of the three runs produced code. One searched the repo, git history and my home directory for the rule and then died on a permission prompt. The other two stopped and asked me to restate the rule, which is the right behaviour but means I'm the memory.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;It did not save exploration of the code itself.&lt;/strong&gt; The memory runs still read the source files before editing, and they should: you can't edit code you haven't seen. The drop in exploration calls (3.3 vs 13.7) came almost entirely from not having to hunt for a decision that only existed in a previous chat. On a 7-file repo there was nothing else for memory to save.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;It cost more tokens than re-typing the rule.&lt;/strong&gt; This surprised me. The memory block injected into the system prompt was small, 1,038 characters or about 250 tokens. The fixed cost is the tool definitions. The MCP server alone adds about 4,800 tokens to every request, and the plugin's own search tools add about 500 more. Across a run of 7 to 9 model calls, that is why the memory runs processed about 126k context tokens versus about 66k when I just re-explained the rule. Most of it was cached by the provider, so the uncached input was closer (16k vs 10k), but it is a fixed overhead you pay on every request. Wall time came out the same (35.0s vs 34.0s).&lt;/p&gt;

&lt;p&gt;So the honest summary of the numbers: claude-mem replaced 40 words I would have had to type and remember to type, and it never blocked. In exchange, it added about 5,200 tokens to every request and ran a second model in the background.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the next session actually saw
&lt;/h2&gt;

&lt;p&gt;The injected block was short: one observation title ("Discovered GST calculation bug in invoice module") and the previous session's summary. The 30% rule wasn't in the index itself. It was in the summary's "Next Steps" line and in a note the agent had saved with &lt;code&gt;save_memory&lt;/code&gt; during session 1.&lt;/p&gt;

&lt;p&gt;There is a catch I only found by reading the database. The &lt;code&gt;save_memory&lt;/code&gt; call went through the MCP server, which tagged the note with source &lt;code&gt;claude&lt;/code&gt;, not &lt;code&gt;opencode&lt;/code&gt;. Session-start context is limited to the current agent by default (&lt;code&gt;CLAUDE_MEM_SESSION_START_INCLUDE_ALL_SOURCES&lt;/code&gt; is &lt;code&gt;false&lt;/code&gt;, per the README). So that note never appeared in the injected index; the agent found it through search. The viewer's settings preview shows the same thing: switch the source to &lt;code&gt;claude&lt;/code&gt; and it says "No previous sessions found for this project yet."&lt;/p&gt;

&lt;h2&gt;
  
  
  The reliability caveat: does "remember X" land?
&lt;/h2&gt;

&lt;p&gt;Capture is done by a model, so it is probabilistic. I tested it three times from the same seeded state with a throwaway task (add a docstring) plus two casual facts, "remember for later that our release branch is named ship/2026-q4" and "Globex invoices always use 12% GST," and then checked the database 150 seconds after each session ended.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Run 1:&lt;/strong&gt; the agent saved both facts itself with &lt;code&gt;save_memory&lt;/code&gt;. The observer wrote nothing of its own, and no new session summary appeared.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run 2:&lt;/strong&gt; the agent saved one note. The observer wrote three observations, including a "decision" with both facts. Again no new summary.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run 3:&lt;/strong&gt; the agent saved nothing. The observer caught both facts, and the session summary has them in its notes.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both facts landed 3 out of 3 times, but through a different path each time, and the end-of-session summary was written only once in three runs. The seed session was worse. The free Gemini endpoint returned transient errors, and the worker log reads &lt;code&gt;Skipped expired live observer inputs; no replay scheduled {skipped=7}&lt;/code&gt;. Seven tool calls were never turned into observations, and the summary only appeared after I re-sent the summarize request. If something really matters, ask the agent to call &lt;code&gt;save_memory&lt;/code&gt; explicitly and check the viewer. Don't assume the observer caught it.&lt;/p&gt;

&lt;p&gt;The observer isn't free either. For the docstring session, which made 10 tool calls, it used 31,500 input tokens on Gemini and took 172 seconds to finish in the background.&lt;/p&gt;

&lt;h2&gt;
  
  
  Privacy: what I verified
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;&amp;lt;private&amp;gt;&lt;/code&gt; tags work as documented.&lt;/strong&gt; I put a fake staging password inside &lt;code&gt;&amp;lt;private&amp;gt;&lt;/code&gt; tags in all three reliability prompts. It appeared zero times in observations, summaries, stored prompts, raw tool records, the Chroma index or the worker log. The stored prompt shows the block removed. The docs also say tags are stripped at storage time, not from the live conversation, and I confirmed that: OpenCode's own session database had the string in 8 rows. &lt;code&gt;&amp;lt;private&amp;gt;&lt;/code&gt; protects claude-mem's memory, not your agent's own history.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Local by default.&lt;/strong&gt; Everything lives in &lt;code&gt;~/.claude-mem/&lt;/code&gt; on your machine. The README says Cloud Sync is opt-in, and my viewer header said "Cloud sync off".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Telemetry is on by default.&lt;/strong&gt; Anonymous usage analytics are enabled until you run &lt;code&gt;npx claude-mem telemetry disable&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Screenshots are kept by default.&lt;/strong&gt; &lt;code&gt;CLAUDE_MEM_MEDIA_CAPTURE_ENABLED&lt;/code&gt; defaults to &lt;code&gt;true&lt;/code&gt;. The "Keep images" switch stores converted copies of tool screenshots under &lt;code&gt;~/.claude-mem/media-v1/&lt;/code&gt;, with no size cap or age limit until the event is deleted. They upload to cmem.ai only if Cloud Sync is on, and go to the observer model only if a separate switch (off by default) is on.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Auto-redaction is opt-in.&lt;/strong&gt; Regex redaction of common key formats is off until you set &lt;code&gt;CLAUDE_MEM_REDACT_ENABLED&lt;/code&gt; to &lt;code&gt;true&lt;/code&gt;. Unless you turn it on, anything a tool prints is stored as-is.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The database is a plain file.&lt;/strong&gt; As the baseline agent showed me, anything running as your user can read it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  My take
&lt;/h2&gt;

&lt;p&gt;claude-mem does the one thing it promises: a decision from yesterday's session reached today's session without me repeating it, every time I tried. For long-running projects where the context is mostly decisions, conventions and "we tried X and it broke," that's real value. It also pairs well with the billion-context approach: one keeps a single session small, the other carries the important parts into the next one.&lt;/p&gt;

&lt;p&gt;What I'd tell my team before they install it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Budget for the overhead. About 5,200 extra tokens per request on a small task is more than re-typing a short rule. The trade only pays off when the context you'd otherwise re-explain is large, or when you'd forget to re-explain it.&lt;/li&gt;
&lt;li&gt;Don't run the observer on a free-tier key with a 20-requests-a-day limit. Capture silently drops work when the provider has problems.&lt;/li&gt;
&lt;li&gt;Use &lt;code&gt;save_memory&lt;/code&gt; for anything you can't afford to lose. Automatic capture is a model's judgement.&lt;/li&gt;
&lt;li&gt;Turn off telemetry, decide on "Keep images" on purpose, turn on redaction, and use &lt;code&gt;&amp;lt;private&amp;gt;&lt;/code&gt; for anything sensitive.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Reproduce it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx claude-mem install --ide opencode      &lt;span class="c"&gt;# or --ide cursor, claude-code, codex-cli, ...&lt;/span&gt;
npx claude-mem telemetry disable
npx claude-mem start
&lt;span class="c"&gt;# session 1: do some work and state a decision; session 2: ask for work that depends on it&lt;/span&gt;
npx claude-mem search &lt;span class="s2"&gt;"what discount rule did finance agree"&lt;/span&gt;
open http://127.0.0.1:37700/viewer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All numbers come from OpenCode's JSON event stream (&lt;code&gt;opencode run --format json&lt;/code&gt;), on 12 Nov 2026 IST, with claude-mem 15.18.1 and a deliberately tiny repo. On a large codebase the balance between fixed overhead and saved exploration will look different. That's the next test I want to run.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>agents</category>
      <category>devtools</category>
    </item>
    <item>
      <title>billion-context Hands-On: Compressing Coding-Agent History to Fit a Small Context Window</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Mon, 09 Nov 2026 12:13:31 +0000</pubDate>
      <link>https://dev.to/ishank-dev/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-9pb</link>
      <guid>https://dev.to/ishank-dev/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-9pb</guid>
      <description>&lt;p&gt;Long agent sessions keep failing the same way for me. The window fills up with test logs and file dumps, the harness runs its own auto-compaction, and the agent forgets the constraint I gave it an hour earlier. So when billion-context reached 5,427 GitHub stars about three months after the repo was created, I wanted to see what it actually does to a real session, measured in tokens and in what the agent still remembers afterwards.&lt;/p&gt;

&lt;p&gt;This post covers what I installed, the one bug I ran into, the numbers from a real tool-calling session, and where I think the project's claims hold up and where they don't.&lt;/p&gt;

&lt;h2&gt;
  
  
  What billion-context is
&lt;/h2&gt;

&lt;p&gt;billion-context (&lt;a href="https://github.com/ranxianglei/billion-context" rel="noopener noreferrer"&gt;github.com/ranxianglei/billion-context&lt;/a&gt;, MIT plus an attribution clause) is a local proxy, &lt;code&gt;bili&lt;/code&gt;, that sits between a coding agent and its model API. It works with Anthropic, OpenAI and Google-native requests, and it adds four tools to every request:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;compress&lt;/code&gt;: fold a range of messages into a summary&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;decompress&lt;/code&gt;: restore a folded range&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;search_context&lt;/code&gt;: keyword search over folded summaries&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;acp_status&lt;/code&gt;: report how full the context is&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The main difference from Claude Code-style auto-compaction is that &lt;strong&gt;the model decides what to fold&lt;/strong&gt;, nudged by the proxy as the window fills. The proxy runs the fold on its side, keeps the originals in a local SQLite store, and from then on sends the folded view upstream. Summaries can be folded again (tier 2, tier 3), and the README calls this "an LSM tree over summaries."&lt;/p&gt;

&lt;p&gt;The README makes three headline claims: "100K context is enough", "5× fewer tokens", and "month-long single sessions." The author's preprint in the repo reports 174,327 model calls over 4.5 months. A benchmark folder labeled "independent third-party" reports 79.2% vs 83.2% for full context on LongMemEval-S at 71% lower cost. Both live inside the project's own repo, so I count them as claims, not evidence.&lt;/p&gt;

&lt;p&gt;The repo is also changing very fast: 565 commits between 2 Nov and today, and npm &lt;code&gt;latest&lt;/code&gt; is 0.1.213. I tested that version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setup
&lt;/h2&gt;

&lt;p&gt;Installing took about five seconds:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-g&lt;/span&gt; billion-context &lt;span class="nt"&gt;--prefix&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;~/.local
bili &lt;span class="nt"&gt;--version&lt;/span&gt;   &lt;span class="c"&gt;# 0.1.213 (133f3525c)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I had a Free Tier Gemini key and no other credentials, so I skipped the &lt;code&gt;bili claude&lt;/code&gt; / &lt;code&gt;bili codex&lt;/code&gt; launchers. Instead I used the universal path: start the proxy and prefix the upstream URL with &lt;code&gt;/bili/&lt;/code&gt;. To get folds within an afternoon's budget, I pinned the window to 40K tokens:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"port"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;8787&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"compress"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"modelContextLimit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;40000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"maxContextLimit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"60%"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"nudgeGrowthTokens"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;8000&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;ACP_AUTO_UPDATE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0 bili start
curl &lt;span class="nt"&gt;-s&lt;/span&gt; http://127.0.0.1:8787/__bili/health
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I set &lt;code&gt;ACP_AUTO_UPDATE=0&lt;/code&gt; on purpose. By default &lt;code&gt;bili&lt;/code&gt; checks npm every three minutes and upgrades itself after a staggered delay. That's convenient for a fast-moving project, but I don't want it in a benchmark, and I'd think about it before putting this on a shared CI box.&lt;/p&gt;

&lt;h2&gt;
  
  
  The first bug: Gemini's OpenAI-compatible endpoint
&lt;/h2&gt;

&lt;p&gt;My first harness used Gemini's OpenAI-compatible endpoint. The first request went through fine. Every request after a tool call failed with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;HTTP 400: Function call is missing a thought_signature in functionCall parts.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Gemini 3.x returns an opaque &lt;code&gt;thought_signature&lt;/code&gt; with each tool call, and it has to come back in the next request. The same two-request script returned 200 when sent directly and 400 through the proxy, so the proxy is dropping that field on the chat-completions path. Switching to Gemini's native &lt;code&gt;generateContent&lt;/code&gt; wire fixed it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;BASE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;http://127.0.0.1:8787/bili/https://generativelanguage.googleapis.com/v1beta&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;BASE&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/models/gemini-3.5-flash-lite:generateContent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                  &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                  &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x-goog-api-key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x-acp-session&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you run Gemini 3 behind an OpenAI-style client, test multi-turn tool calls before anything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  The test session
&lt;/h2&gt;

&lt;p&gt;I wrote a small agent loop with a single &lt;code&gt;bash&lt;/code&gt; tool and pointed it at a fresh clone of &lt;code&gt;python-slugify&lt;/code&gt;. I scripted 11 user turns, the kind I'd actually type:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;ticket SLUG-482 for a client, Python 3.8 compatibility, no new dependencies&lt;/li&gt;
&lt;li&gt;run the tests, read &lt;code&gt;slugify.py&lt;/code&gt;, explain &lt;code&gt;max_length&lt;/code&gt; vs &lt;code&gt;word_boundary&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;a standup decision: keep the &lt;code&gt;-&lt;/code&gt; separator and add &lt;code&gt;max_words: int = 0&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;a CLI flag, a CHANGELOG entry, a README update&lt;/li&gt;
&lt;li&gt;mid-session: staging port 7431, reviewer Priya Raman, Unicode refactor pushed to Q1 because of a Django adapter risk&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The model was &lt;code&gt;gemini-3.5-flash-lite&lt;/code&gt;. For the baseline, I ran Gemini's &lt;code&gt;countTokens&lt;/code&gt; on every uncompressed request prefix, using the same system prompt and tool schema without the proxy. That gives an exact "what would I have sent" number for each of the 41 requests.&lt;/p&gt;

&lt;p&gt;One thing about the agent itself: Flash-Lite never edited a file. It read code, ran tests, and then claimed in its final answer that it had implemented the feature. The git tree was clean. That's a model problem, not a compression problem. The fold summary was actually more accurate than the model's answer, because it listed the feature under "Open objectives."&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;p&gt;The session took 41 client requests and 184 seconds of wall-clock time. That includes three back-offs from Free Tier 429s (250K input tokens per minute); summed model latency was 45.9 seconds. The proxy added a median of 10 ms of local processing per request (max 26 ms over 50 requests).&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Folds:&lt;/strong&gt; 2 events, 7 summary blocks of 111–284 characters each&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fold 1&lt;/strong&gt; (turn 6): next prompt went from 27,853 to 9,396 tokens; ~14,095 tokens reclaimed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fold 2&lt;/strong&gt; (turn 10): 26,778 to 12,046 tokens; ~9,515 reclaimed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Final request:&lt;/strong&gt; 13,431 tokens via bili vs 38,816 uncompressed (&lt;strong&gt;2.9× smaller&lt;/strong&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Peak prompt:&lt;/strong&gt; 27,853 vs 38,816&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Total input over the session:&lt;/strong&gt; 663,321 vs 758,108 tokens (&lt;strong&gt;12.5% less&lt;/strong&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fixed overhead:&lt;/strong&gt; the first request was 4,243 tokens upstream against a 180-token uncompressed prompt, so the injected tools and instructions cost about 4,060 tokens on every request&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is the first block the model wrote:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Compressed conversation section] — Repo Exploration
- User request (m00001): ticket SLUG-482 for client Northwind.
  Constraints: Python 3.8 compatibility, no new dependencies.
- Explored repo layout and located main `slugify()` in
  `slugify/slugify.py:126` taking 16 parameters.
- Ran pytest via `python -m pytest`: 133 passed, 0 skipped.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's 284 characters standing in for a few thousand tokens of &lt;code&gt;find&lt;/code&gt; output, a full pytest log, and an &lt;code&gt;inspect.signature&lt;/code&gt; dump. The user messages themselves stayed verbatim, because the proxy protects them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Did the agent still remember?
&lt;/h2&gt;

&lt;p&gt;I asked two quizzes at the end of the session, once through the proxy (folded view) and once directly with the full history.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Easy set (10 questions about things I told it):&lt;/strong&gt; ticket ID, client, Python version, separator decision, new kwarg, test file, first test result, port, reviewer, postponed refactor. Through bili: &lt;strong&gt;10/10&lt;/strong&gt; on 13,821 prompt tokens. Full history: &lt;strong&gt;10/10&lt;/strong&gt; on 39,114. Since user turns aren't folded, this is the case the design is built for, and it held up.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hard set (8 questions about tool output):&lt;/strong&gt; the line number of &lt;code&gt;slugify()&lt;/code&gt;, the module marked "DO NOT MODIFY", the CLI &lt;code&gt;--algorithm&lt;/code&gt; choices, the last commit hash, the top CHANGELOG version, the pytest/Python versions, the parameter count, and the CLI test files.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Through bili with tools disabled: &lt;strong&gt;4/8&lt;/strong&gt;. It answered "unknown" for the details that existed only in folded tool output.&lt;/li&gt;
&lt;li&gt;Through bili with &lt;code&gt;search_context&lt;/code&gt;/&lt;code&gt;decompress&lt;/code&gt; allowed (streaming): &lt;strong&gt;6/8&lt;/strong&gt;. The model ran five &lt;code&gt;search_context&lt;/code&gt; queries ("slugify.py", "CLI", "algorithm", "CHANGELOG", "git log") and then answered. The "algorithm" search matched zero summaries but five folded originals, so the search covers raw history too, not only the summaries.&lt;/li&gt;
&lt;li&gt;Full history, direct: &lt;strong&gt;6/8&lt;/strong&gt;, with different misses: it got the line number wrong (195 instead of 126) and named the wrong CLI test file.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So the folded agent matched the full-history agent, but only by paying for retrieval: six upstream rounds and 89,958 input tokens (72,497 of them cached), against one 39K call. Three practical notes. First, on the non-streaming lane the &lt;code&gt;search_context&lt;/code&gt; output came back to my client as the response text, with no answer after it, so plan on streaming. Second, the search receipts ("Found 2 block(s) for ...") show up in the client-visible text stream, as do fold receipts. Third, in an earlier attempt the model decompressed a block to &lt;code&gt;/tmp/acp-decompress-b6-*.txt&lt;/code&gt; and then tried to &lt;code&gt;grep&lt;/code&gt; it with its own bash tool, so decompress-to-file assumes the agent can read local files.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it cost
&lt;/h2&gt;

&lt;p&gt;At Gemini 3.5 Flash-Lite list prices ($0.30/M input, $0.03/M cached, $2.50/M output):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;bili:&lt;/strong&gt; 77% of input tokens were cache hits, about &lt;strong&gt;$0.067&lt;/strong&gt; for the session&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Uncompressed, assuming perfect prefix caching:&lt;/strong&gt; about &lt;strong&gt;$0.039&lt;/strong&gt; (estimate)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;With no caching on either side:&lt;/strong&gt; bili $0.205 vs $0.234&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;On a 39K-token session, compression &lt;strong&gt;lost&lt;/strong&gt; on cost once caching is counted. Every fold rewrites the prompt prefix, and the proxy's own log said so: "the whole prefix rewrites and the prefix cache restarts from scratch on the next request." On top of that, the ~4K-token tool overhead is charged on every call. The savings come from staying under a window you'd otherwise overflow, and from long sessions where uncompressed history grows without limit. My session was neither, so treat 12.5% as a lower bound for a short session, not as evidence for the "5×" claim.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd tell my team
&lt;/h2&gt;

&lt;p&gt;Where it's useful:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Long-running agent sessions on small-window or cheap models, where hitting the window is the real failure&lt;/li&gt;
&lt;li&gt;Teams that want folds they can audit and undo; &lt;code&gt;bili export &amp;lt;session&amp;gt;&lt;/code&gt; dumps the folded view and every block summary&lt;/li&gt;
&lt;li&gt;Setups that already run a local proxy; median overhead was 10 ms&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Where I'd hold off:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Short tasks under ~50K tokens. With default settings (50K growth gate, nudge at 75% of the window) my session would never have folded, and the tool overhead would only have added cost.&lt;/li&gt;
&lt;li&gt;Gemini behind an OpenAI-compatible client until the &lt;code&gt;thought_signature&lt;/code&gt; drop is fixed&lt;/li&gt;
&lt;li&gt;Anything user-facing without reading the license. It's MIT plus a required attribution line on your home page, docs, or About page.&lt;/li&gt;
&lt;li&gt;Pinned production environments. Hundreds of commits a week plus self-update on by default means you need &lt;code&gt;ACP_AUTO_UPDATE=0&lt;/code&gt; and a pinned version.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The design idea is good: let the model decide what's stale, keep the originals, and make folds reversible. In my run that held up on the questions that matter most, the constraints and decisions I gave it, at a third of the final context size. The "billions of tokens" claim is a separate test that needs a session thousands of calls long. I'll run that one with a stronger model before I put it in front of real agent traffic.&lt;/p&gt;

&lt;p&gt;Originally published on Medium: &lt;a href="https://medium.com/@ishank.iandroid/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8" rel="noopener noreferrer"&gt;https://medium.com/@ishank.iandroid/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>agents</category>
      <category>gemini</category>
    </item>
    <item>
      <title>Gemini Omni 1.2 Flash Generates and Edits 15-Second Videos in the Gemini API for $0.07 a Second</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Sat, 07 Nov 2026 04:54:28 +0000</pubDate>
      <link>https://dev.to/ishank-dev/gemini-omni-12-flash-generates-and-edits-15-second-videos-in-the-gemini-api-for-007-a-second-3p1p</link>
      <guid>https://dev.to/ishank-dev/gemini-omni-12-flash-generates-and-edits-15-second-videos-in-the-gemini-api-for-007-a-second-3p1p</guid>
      <description>&lt;p&gt;On November 5, 2026, Google made &lt;strong&gt;Gemini Omni 1.2 Flash&lt;/strong&gt; (&lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt;) generally available across the Gemini API and Google Cloud. Two weeks after shutting down Veo 3.1 Preview in the developer API on October 22, Google is positioning Omni Flash as its default model for generating, editing and extending video with synchronized audio.&lt;/p&gt;

&lt;p&gt;Three headline changes stand out:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Native 15-second clips&lt;/strong&gt;, up from 10 seconds in Omni 1.1 Flash, selectable in one-second steps from 3 to 15 seconds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A 30% price cut&lt;/strong&gt; on video output tokens, dropping 720p generation from about $0.10 per second to roughly &lt;strong&gt;$0.07 per second&lt;/strong&gt; ($12.00 per million video output tokens, down from $17.50).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multi-turn conversational editing and stronger reference adherence&lt;/strong&gt;, accepting up to 6 reference images and refining clips across turns without starting over.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One date in the release notes matters if you already ship on Omni 1.1: &lt;strong&gt;on December 3, 2026, every API call to &lt;code&gt;gemini-omni-1.1-flash&lt;/code&gt; will automatically route to &lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I spent Friday evening wiring &lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt; up on Linux with Python 3.12, &lt;code&gt;google-genai&lt;/code&gt; 3.0.0, raw REST calls and &lt;code&gt;ffprobe&lt;/code&gt;. Because my AI Studio key is on the Free Tier, where Omni 1.2 enforces a hard zero-quota wall, I couldn't render paid clips from my own prompts. Everything else on the wire was testable right now, and the first bug hit in 90 milliseconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I tested and where the Free Tier stops
&lt;/h2&gt;

&lt;p&gt;My setup and test boundary:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Environment:&lt;/strong&gt; Linux x86_64, Python 3.12.14, &lt;code&gt;google-genai&lt;/code&gt; 3.0.0, &lt;code&gt;ffprobe&lt;/code&gt; and a Free Tier AI Studio &lt;code&gt;GEMINI_API_KEY&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What ran live ($0.00 spend):&lt;/strong&gt; model discovery (&lt;code&gt;models.list&lt;/code&gt;, &lt;code&gt;models.get&lt;/code&gt;), input token accounting (&lt;code&gt;countTokens&lt;/code&gt;) across text, reference images, uploaded 720p video and WAV audio, &lt;code&gt;Files API&lt;/code&gt; upload latency, pre-quota schema validation on &lt;code&gt;/v1beta/interactions&lt;/code&gt;, model-routing checks, stateful chaining and container inspection of Google's raw &lt;code&gt;omni_*.mp4&lt;/code&gt; API samples.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What blocked (&lt;code&gt;HTTP 429&lt;/code&gt;):&lt;/strong&gt; all four video generation calls I sent (5s 360p draft, 6s 720p alley tracking shot, 15s 720p timecoded clip via URI delivery, and 6s 2-image reference shot) returned &lt;code&gt;Rate limit exceeded for model gemini-omni-1.2-flash (limit: 0 input tokens per minute on Free Tier)&lt;/code&gt; in &lt;strong&gt;0.41 to 0.48 seconds&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Omni 1.2 Flash has &lt;strong&gt;zero Free Tier quota&lt;/strong&gt;: 0 RPM and 0 RPD. Even a 3-second 360p draft (~$0.07) requires Paid Tier 1 billing enabled (8 RPM, 40 RPD, doubled from Omni 1.1). If you want to try 15-second clips manually without API billing, Google Vids includes them at no extra cost on Google and Workspace accounts.&lt;/p&gt;

&lt;p&gt;Because I didn't run on a paid project, I haven't measured live video render times or visual drift on my own prompts. Google also published zero benchmarks comparing 1.2 against 1.1, so treat quality claims as untested until you run your own prompts.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bug #1: Google's Python quickstart 404s out of the box
&lt;/h2&gt;

&lt;p&gt;Gemini Omni models run on Google's stateful &lt;strong&gt;Interactions API&lt;/strong&gt; (&lt;code&gt;/v1beta/interactions&lt;/code&gt;), not &lt;code&gt;generateContent&lt;/code&gt;, even though &lt;code&gt;GET /v1beta/models/gemini-omni-1.2-flash&lt;/code&gt; (0.17s) still lists &lt;code&gt;["generateContent", "countTokens"]&lt;/code&gt; under &lt;code&gt;supportedGenerationMethods&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;If you install &lt;code&gt;google-genai&lt;/code&gt; 3.0.0 and run the quickstart from Google's Omni docs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;google&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;

&lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;genai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Client&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;interaction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;interactions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-omni-1.2-flash&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nb"&gt;input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;A marble rolling fast on a chain reaction style track, continuous smooth shot.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;response_format&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;video&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;duration&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;5s&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resolution&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;360p&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It fails in &lt;strong&gt;0.09 to 0.46 seconds&lt;/strong&gt; with a 404:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NotFoundError: Error code: 404 - {'error': {'message':
"Model 'gemini-omni-1.2-flash' not found. Did you mean 'gemini-omni-1.1-flash'?",
'code': 'not_found'}}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;"Did you mean 'gemini-omni-1.1-flash'?"&lt;/code&gt; is a red herring: &lt;code&gt;gemini-omni-1.1-flash&lt;/code&gt; 404s on that path too (0.20s).&lt;/p&gt;

&lt;p&gt;Inspecting &lt;code&gt;client.interactions.sdk_configuration&lt;/code&gt; shows why. In &lt;code&gt;google-genai&lt;/code&gt; 3.0.0, the Interactions sub-client defaults to &lt;code&gt;Globals(api_version='v1')&lt;/code&gt; (&lt;code&gt;POST /v1/interactions&lt;/code&gt;), where neither Omni 1.1 nor 1.2 exists. Both live only on &lt;code&gt;v1beta&lt;/code&gt;. (Importing &lt;code&gt;google.genai&lt;/code&gt; on Python 3.12 also emits two &lt;code&gt;SyntaxWarning&lt;/code&gt; escape warnings in &lt;code&gt;types.py&lt;/code&gt;.)&lt;/p&gt;

&lt;p&gt;Passing &lt;code&gt;api_version="v1beta"&lt;/code&gt; fixes the 404:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;interaction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;interactions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;api_version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;v1beta&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-omni-1.2-flash&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nb"&gt;input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;A marble rolling fast on a chain reaction style track, continuous smooth shot.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;response_format&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;video&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;duration&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;5s&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resolution&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;360p&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;delivery&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;uri&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Right behind that 404 sits a second SDK catch. When &lt;code&gt;client.interactions.create(api_version="v1beta", ...)&lt;/code&gt; hits a &lt;code&gt;429&lt;/code&gt;, even a permanent &lt;code&gt;limit: 0&lt;/code&gt; Free Tier error, its internal retry loop retries silently for &lt;strong&gt;over 90 seconds per call&lt;/strong&gt; before raising &lt;code&gt;RateLimitError&lt;/code&gt;. Two calls hung my script for over three minutes before I killed it. Cap retries in CI so bad keys fail fast.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I measured on the live API
&lt;/h2&gt;

&lt;p&gt;Even without paid video output quota, several parts of the &lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt; stack respond on a Free Tier key.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Real input token counts (&lt;code&gt;countTokens&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Omni 1.2 Flash bills inputs at &lt;strong&gt;$1.50 per million tokens&lt;/strong&gt; across text, images and video. Calling &lt;code&gt;POST /v1beta/models/gemini-omni-1.2-flash:countTokens&lt;/code&gt; returned &lt;code&gt;HTTP 200&lt;/code&gt; in &lt;strong&gt;0.18 to 0.52 seconds&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;14-word text prompt:&lt;/strong&gt; &lt;code&gt;16&lt;/code&gt; TEXT tokens (0.19s), costing &lt;strong&gt;$0.000024&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two reference PNGs&lt;/strong&gt; (1280x720 mug card and 800x800 robot sheet) plus 11-token prompt: &lt;code&gt;527&lt;/code&gt; tokens (&lt;code&gt;11&lt;/code&gt; TEXT + &lt;code&gt;516&lt;/code&gt; IMAGE, or &lt;strong&gt;258 tokens per image&lt;/strong&gt;, 0.18s), costing &lt;strong&gt;$0.00079&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Six reference PNGs&lt;/strong&gt; (the maximum allowed) plus 16-token prompt: &lt;code&gt;1,564&lt;/code&gt; tokens (&lt;code&gt;16&lt;/code&gt; TEXT + &lt;code&gt;1,548&lt;/code&gt; IMAGE, 0.20s), costing &lt;strong&gt;$0.00235&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Uploaded 5.0-second 720p 24fps MP4&lt;/strong&gt; with AAC audio plus edit prompt: &lt;code&gt;Files API&lt;/code&gt; upload took &lt;strong&gt;1.13s&lt;/strong&gt; and reached &lt;code&gt;ACTIVE&lt;/code&gt; in &lt;strong&gt;3.49s&lt;/strong&gt; (2 polls). &lt;code&gt;countTokens&lt;/code&gt; returned &lt;code&gt;1,488&lt;/code&gt; tokens in 0.30s: &lt;code&gt;13&lt;/code&gt; TEXT + &lt;code&gt;1,315&lt;/code&gt; VIDEO (&lt;strong&gt;263 video tokens/s&lt;/strong&gt;) + &lt;code&gt;160&lt;/code&gt; AUDIO (&lt;strong&gt;32 audio tokens/s&lt;/strong&gt;), or &lt;strong&gt;295 media tokens/s&lt;/strong&gt;. That is &lt;strong&gt;$0.00223&lt;/strong&gt; for 5s, or about &lt;strong&gt;$0.0066&lt;/strong&gt; for a 15s input clip.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Input tokens are rounding error. Six reference images or a 15-second source video cost under two-thirds of a cent; over 99.5% of your bill is video output.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Schema checks that fail fast (&lt;code&gt;HTTP 400&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;POST /v1beta/interactions&lt;/code&gt; validates its request schema &lt;em&gt;before&lt;/em&gt; checking quota, returning &lt;code&gt;HTTP 400&lt;/code&gt; in &lt;strong&gt;0.18 to 0.21 seconds&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duration needs &lt;code&gt;"s"&lt;/code&gt;:&lt;/strong&gt; &lt;code&gt;"duration": 5&lt;/code&gt; fails; pass &lt;code&gt;"3s"&lt;/code&gt; through &lt;code&gt;"15s"&lt;/code&gt; (defaults to &lt;code&gt;"10s"&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Four resolutions:&lt;/strong&gt; &lt;code&gt;"480p"&lt;/code&gt; fails, listing &lt;code&gt;'360p'&lt;/code&gt;, &lt;code&gt;'720p'&lt;/code&gt;, &lt;code&gt;'1080p'&lt;/code&gt;, &lt;code&gt;'4k'&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two aspect ratios:&lt;/strong&gt; &lt;code&gt;"1:1"&lt;/code&gt; fails, listing &lt;code&gt;'16:9'&lt;/code&gt; (default) and &lt;code&gt;'9:16'&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Five &lt;code&gt;task&lt;/code&gt; enums:&lt;/strong&gt; &lt;code&gt;"video_upscale"&lt;/code&gt; fails, listing &lt;code&gt;'text_to_video'&lt;/code&gt;, &lt;code&gt;'image_to_video'&lt;/code&gt;, &lt;code&gt;'reference_to_video'&lt;/code&gt;, &lt;code&gt;'edit'&lt;/code&gt;, &lt;code&gt;'extend'&lt;/code&gt;. Google advises omitting &lt;code&gt;task&lt;/code&gt; unless prompting picks the wrong mode, and deprecates &lt;code&gt;&amp;lt;FIRST_FRAME&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;IMAGE_REF_0&amp;gt;&lt;/code&gt; control tags.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  3. Stateful chaining and silent model routing
&lt;/h3&gt;

&lt;p&gt;To test multi-turn state lookup, I created a Turn 1 interaction on &lt;code&gt;/v1beta/interactions&lt;/code&gt; with &lt;code&gt;gemini-3-flash-preview&lt;/code&gt; (&lt;code&gt;HTTP 200&lt;/code&gt; in &lt;strong&gt;3.22s&lt;/strong&gt;, returning &lt;code&gt;v1_Chds...&lt;/code&gt;), then passed that &lt;code&gt;previous_interaction_id&lt;/code&gt; into &lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt; on Turn 2. A fake ID failed pre-quota with &lt;code&gt;HTTP 400&lt;/code&gt; in 0.32s, while the real &lt;code&gt;v1_...&lt;/code&gt; ID passed state lookup across models and hit the Omni 1.2 quota gate in 0.88s. Avoid setting &lt;code&gt;store=false&lt;/code&gt; on Turn 1 if you plan to edit later, since disabling storage breaks &lt;code&gt;previous_interaction_id&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Sending &lt;code&gt;"model": "gemini-omni-flash-preview"&lt;/code&gt; (shut down earlier) returned a 429 naming &lt;code&gt;gemini-omni-1.1-flash&lt;/code&gt; (0.52s), confirming the gateway already aliases retired Omni IDs server-side ahead of the December 3 &lt;code&gt;1.1&lt;/code&gt; to &lt;code&gt;1.2&lt;/code&gt; switch.&lt;/p&gt;

&lt;h2&gt;
  
  
  Output cost math and the 4 MB payload trap
&lt;/h2&gt;

&lt;p&gt;Google bills Omni 1.2 Flash video output at &lt;strong&gt;$12.00 per million tokens&lt;/strong&gt; (text and thinking tokens at $9.00 per million):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;360p:&lt;/strong&gt; &lt;code&gt;1,931&lt;/code&gt; tokens/s, or &lt;strong&gt;$0.023/s&lt;/strong&gt; ($0.12 for 5s, $0.35 for 15s)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;720p (default):&lt;/strong&gt; &lt;code&gt;5,792&lt;/code&gt; tokens/s, or &lt;strong&gt;$0.070/s&lt;/strong&gt; ($0.42 for 6s, $0.70 for 10s, $1.04 for 15s)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;1080p (upsampled):&lt;/strong&gt; &lt;code&gt;8,688&lt;/code&gt; tokens/s, or &lt;strong&gt;$0.104/s&lt;/strong&gt; ($0.52 for 5s, $1.56 for 15s)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;4K (upsampled):&lt;/strong&gt; &lt;code&gt;17,376&lt;/code&gt; tokens/s, or &lt;strong&gt;$0.209/s&lt;/strong&gt; ($1.04 for 5s, $3.13 for 15s)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Nine 15-second drafts at 360p ($3.13) cost the same as one 15-second render at 4K ($3.13).&lt;/strong&gt; Draft at 360p or 720p, refine via &lt;code&gt;previous_interaction_id&lt;/code&gt;, and render 1080p or 4K last. Batch API is unsupported.&lt;/p&gt;

&lt;p&gt;Inspecting four raw &lt;code&gt;omni_*.mp4&lt;/code&gt; samples from Google's API docs (&lt;code&gt;encoder: Google&lt;/code&gt;) and three launch-blog demos with &lt;code&gt;ffprobe&lt;/code&gt; showed why delivery settings matter:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Container specs:&lt;/strong&gt; raw outputs are H.264 (&lt;code&gt;yuv420p&lt;/code&gt;) at &lt;strong&gt;24 fps&lt;/strong&gt; with &lt;strong&gt;48 kHz stereo AAC&lt;/strong&gt; at &lt;strong&gt;2.37 to 4.54 Mbps&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The 4 MB inline limit:&lt;/strong&gt; inline base64 responses cap out at &lt;strong&gt;4 MB&lt;/strong&gt;. Three of Google's four raw 720p doc samples (&lt;code&gt;5.42 MB&lt;/code&gt; at 10s, &lt;code&gt;5.65 MB&lt;/code&gt; at 20s, &lt;code&gt;5.78 MB&lt;/code&gt; at 20s) exceed 4 MB; only one 10s clip (&lt;code&gt;3.08 MB&lt;/code&gt;) fits. Always pass &lt;code&gt;"delivery": "uri"&lt;/code&gt; for 10-to-15-second 720p+ clips and poll &lt;code&gt;GET /v1beta/files/{id}&lt;/code&gt; until &lt;code&gt;ACTIVE&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Provenance (&lt;code&gt;SynthID&lt;/code&gt; + &lt;code&gt;C2PA&lt;/code&gt;):&lt;/strong&gt; alongside invisible SynthID watermarking, all four raw API MP4s carry a &lt;strong&gt;C2PA JUMBF manifest&lt;/strong&gt; (&lt;code&gt;jumdc2pa&lt;/code&gt;) signed by &lt;code&gt;Google Media Processing Services&lt;/code&gt; under &lt;code&gt;Google C2PA Root CA G3&lt;/code&gt;. The FFmpeg-reencoded blog demos (&lt;code&gt;Lavf62.12.102&lt;/code&gt;) lost their C2PA box, while SynthID survives in the frames.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One doc slip:&lt;/strong&gt; &lt;code&gt;omni_misty_mountains_1080p.mp4&lt;/code&gt;, embedded right under Google's &lt;code&gt;resolution: "1080p"&lt;/code&gt; example, is actually &lt;code&gt;1280x720&lt;/code&gt; in &lt;code&gt;ffprobe&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Four doc contradictions and what to fix before December 3
&lt;/h2&gt;

&lt;p&gt;Reading Google's Keyword post, Cloud blog, Gemini API docs and Cloud Agent Platform model card side by side surfaced four mismatches:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;15 seconds does not apply to every mode:&lt;/strong&gt; text, single-image, reference-to-video and edits reach &lt;strong&gt;15 seconds&lt;/strong&gt; (with 11-to-15s clips limited to &lt;strong&gt;1&lt;/strong&gt; 3s reference video instead of 3), but &lt;strong&gt;first-and-last-frame interpolation stays capped at 3 to 10 seconds&lt;/strong&gt;. Passing &lt;code&gt;"15s"&lt;/code&gt; with two keyframes returns an invalid argument error.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audio references are gated:&lt;/strong&gt; the DeepMind blog pitches 1-second voice cues, the Cloud blog says audio references need &lt;strong&gt;allowlisting&lt;/strong&gt;, and the Gemini API docs mark audio uploads &lt;strong&gt;unsupported&lt;/strong&gt;—even though &lt;code&gt;countTokens&lt;/code&gt; already tokenizes WAV input at &lt;code&gt;32.5&lt;/code&gt; AUDIO tokens/s.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Extended length (&lt;code&gt;30s&lt;/code&gt; vs &lt;code&gt;40s&lt;/code&gt;):&lt;/strong&gt; Gemini API docs cap multi-turn extension at &lt;strong&gt;30 seconds total&lt;/strong&gt;, while the Cloud model card implies up to &lt;strong&gt;40 seconds&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Output token ceiling (&lt;code&gt;65,536&lt;/code&gt; vs &lt;code&gt;260,640&lt;/code&gt;):&lt;/strong&gt; &lt;code&gt;models.get&lt;/code&gt; lists &lt;code&gt;outputTokenLimit: 65,536&lt;/code&gt;, yet 15s at 720p needs &lt;code&gt;86,880&lt;/code&gt; tokens and 15s at 4K needs &lt;code&gt;260,640&lt;/code&gt;, matching the Cloud card's &lt;code&gt;260,640&lt;/code&gt; limit.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Google also notes rough spots on &lt;strong&gt;fast sports action, rapid script changes and complex physics&lt;/strong&gt;, no support for system instructions, &lt;code&gt;temperature&lt;/code&gt;, &lt;code&gt;top_p&lt;/code&gt; or negative prompt fields, and EEA/UK/Switzerland blocks on editing uploaded video.&lt;/p&gt;

&lt;p&gt;If you run &lt;code&gt;gemini-omni-1.1-flash&lt;/code&gt; today, four updates get you ahead of the &lt;strong&gt;December 3, 2026&lt;/strong&gt; automatic reroute:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Pass &lt;strong&gt;&lt;code&gt;api_version="v1beta"&lt;/code&gt;&lt;/strong&gt; in &lt;code&gt;google-genai&lt;/code&gt; 3.0.0 and cap retries so quota errors fail fast.&lt;/li&gt;
&lt;li&gt;Default to &lt;strong&gt;&lt;code&gt;"delivery": "uri"&lt;/code&gt;&lt;/strong&gt; on 10-to-15-second clips at 720p+ to avoid the 4 MB inline cap.&lt;/li&gt;
&lt;li&gt;Cap UI durations at &lt;strong&gt;3–15s&lt;/strong&gt; for standard generation and &lt;strong&gt;3–10s&lt;/strong&gt; for keyframe interpolation and extensions.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  4. Strip deprecated &lt;code&gt;&amp;lt;FIRST_FRAME&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;IMAGE_REF_0&amp;gt;&lt;/code&gt; tags and test &lt;code&gt;gemini-omni-1.2-flash&lt;/code&gt; on staging now, banking the 30% price drop on your own schedule.
&lt;/h2&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/gemini-omni-1-2-flash-generates-and-edits-15-second-videos-in-the-gemini-api-for-0-07-a-second-5172a9bbea96" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>gemini</category>
      <category>generativeai</category>
      <category>llm</category>
      <category>python</category>
    </item>
    <item>
      <title>Aura Autopilot Review: An Open-Source AI Agent That Playtests Your Godot, Unity and Unreal Games</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Thu, 05 Nov 2026 12:55:31 +0000</pubDate>
      <link>https://dev.to/ishank-dev/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-1omf</link>
      <guid>https://dev.to/ishank-dev/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-1omf</guid>
      <description>&lt;p&gt;&lt;em&gt;I hid 4 bugs in a Godot game. It played with real keys and found all 4 in 4 minutes, plus one I didn't know about. Then I compared it to a plain LLM.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-081fb9246738" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Every game dev I know has the same quiet dread: the bug a player finds in the first five minutes that nobody on the team ever hit.&lt;/p&gt;

&lt;p&gt;So when Aura open-sourced &lt;strong&gt;Autopilot&lt;/strong&gt;, an Apache-2.0 agent that plays your game to find bugs, I cleared an evening. The pitch is bold: about 43% more accurate and roughly 2x faster than other AI agents at real playtesting tasks.&lt;/p&gt;

&lt;p&gt;I wanted to see it with my own eyes. Here's what I ran, what it found, and what it cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Aura Autopilot actually is
&lt;/h2&gt;

&lt;p&gt;Autopilot is not one big model clicking around. It's four parts working together:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;An engine plugin&lt;/strong&gt; for Unreal Engine 5 (5.4â€“5.8), Unity 6 or Godot 4.7+. It reads the game's state and presses real keys and mouse buttons, the way a player would.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An agent service&lt;/strong&gt;, a Python program on your machine. It talks to the plugin over a local WebSocket, runs model-written code in a sandbox and keeps memory for each game.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A coding agent&lt;/strong&gt; (OpenCode by default, or Claude Code or Codex). It reads your source and writes a small "harness": the list of moves that make sense for &lt;em&gt;your&lt;/em&gt; game, like "dodge left" or "open inventory".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A decision model&lt;/strong&gt; called Jev, from TypeSafe. During play it picks one of those moves roughly every 0.25 seconds. Fast "reflexes" like dodging run as plain code so they don't wait for the model.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The clever bit is the harness. Aura's team found that giving a model raw keys ("press W", "click") plays badly, while game-specific moves play well. So instead of hand-writing those moves for each game, the coding agent writes them for you.&lt;/p&gt;

&lt;p&gt;Then it remembers. Each project gets a memory folder with a copy of your code, notes, harness code that worked, and past reports. Later runs skip the research and start from a harness that already works.&lt;/p&gt;

&lt;p&gt;You give it one of four kinds of prompts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Explore:&lt;/strong&gt; "Explore the crafting system in Lvl_Main."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test:&lt;/strong&gt; "Go through this level and tell me everything that looks wrong."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Verify:&lt;/strong&gt; "Verify that the tree shakes when you hit it with an axe."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Goal:&lt;/strong&gt; "Kill the boss while keeping your health above 20%."&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where the 43% and 2x numbers come from
&lt;/h2&gt;

&lt;p&gt;I always read the method before I quote a number. Here it is, from Aura's own blog:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;They tested on &lt;strong&gt;Easy Survival RPG V5&lt;/strong&gt; in Unreal Engine 5.8.&lt;/li&gt;
&lt;li&gt;That was &lt;strong&gt;7 tasks&lt;/strong&gt;: 5 verify tasks and 2 goal tasks, like "craft a stone hatchet and use it to kill a zombie".&lt;/li&gt;
&lt;li&gt;Each tool got &lt;strong&gt;one cold attempt&lt;/strong&gt; with empty memory, on the same Windows machine, all using Claude Opus 5.5 on medium effort.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Accuracy:&lt;/strong&gt; Autopilot 7/7, Claude Code 5/7, and an unnamed "leading Unreal/Unity AI agent" 4/7.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mean time:&lt;/strong&gt; Autopilot 262s, Claude Code 437s, competitor 508s.&lt;/li&gt;
&lt;li&gt;Cheating (teleporting, spawning items, god mode) counted as a fail. Both of Claude Code's goal runs cheated.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So the "43% more accurate, ~2x faster" headline is against that unnamed competitor: 100% vs 57%, and 262s vs 508s. Against Claude Code it's 100% vs 71%, and about 1.7x faster.&lt;/p&gt;

&lt;p&gt;That's a real result, but a small one: seven tasks, one game, one attempt each, and the vendor ran it. I'd treat it as a promising signal, not a law.&lt;/p&gt;

&lt;h2&gt;
  
  
  My test setup
&lt;/h2&gt;

&lt;p&gt;My setup:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Machine:&lt;/strong&gt; a Linux box with a real X display. Officially Autopilot ships for Windows and macOS, but the source runs anywhere Python 3.12 does, and the Godot addon is pure GDScript.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Engine:&lt;/strong&gt; Godot 4.7.2, the Linux build.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Game:&lt;/strong&gt; "Dodge the Creeps", the classic open-source Godot demo.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The trap:&lt;/strong&gt; I planted 4 bugs a player would notice:

&lt;ul&gt;
&lt;li&gt;The score keeps ticking after you die.&lt;/li&gt;
&lt;li&gt;The Start button never comes back after Game Over, so you're stuck.&lt;/li&gt;
&lt;li&gt;The player can walk 240px off the right edge of the screen and hide there.&lt;/li&gt;
&lt;li&gt;Moving diagonally is about 1.4x faster than moving straight.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;I removed the git history so no tool could just read my diff.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Install from source was smooth:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Cloned the repo, then &lt;code&gt;uv venv&lt;/code&gt; and &lt;code&gt;uv pip install -e ".[dev]"&lt;/code&gt;. Under a minute.&lt;/li&gt;
&lt;li&gt;Ran the test suite, which uses a simulated game with no keys and no network: &lt;strong&gt;492 passed, 5 skipped in 4m50s.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Copied &lt;code&gt;addons/autopilot&lt;/code&gt; into the game and enabled the plugin.&lt;/li&gt;
&lt;li&gt;Signed in to Aura, started the agent service with &lt;code&gt;autopilot serve --aura&lt;/code&gt;, then launched the game with &lt;code&gt;--playtest=explore --playtest-minutes=5&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What happened when I ran it
&lt;/h2&gt;

&lt;p&gt;I gave it the laziest prompt possible: explore the game for bugs, five minutes of play. No hints about what I'd planted.&lt;/p&gt;

&lt;p&gt;Here's how the run went, start to finish:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The game window opened and the plugin connected to the agent service.&lt;/li&gt;
&lt;li&gt;The coding agent read my 10 project files and wrote a harness: moves like "start game", "hold right against the edge", "move diagonally", "steer away from the nearest creeps" and "wait on the game-over screen".&lt;/li&gt;
&lt;li&gt;Before playing, it ran a quick probe to check that each move actually worked.&lt;/li&gt;
&lt;li&gt;Then Jev took over and played one 150-second episode with real key presses. I watched it start a game, dodge, push into the right wall and die around 29 seconds in.&lt;/li&gt;
&lt;li&gt;After the death it kept trying to start a new game. It tried 10 times, and nothing happened.&lt;/li&gt;
&lt;li&gt;It wrote a report, saved notes to the game's memory and exited cleanly.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Total wall time: &lt;strong&gt;about 4 minutes&lt;/strong&gt; from launch to exit (222 seconds of agent time). It also saved a video recording and a frame-by-frame flipbook of the episode.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it found
&lt;/h2&gt;

&lt;p&gt;All &lt;strong&gt;4 of 4&lt;/strong&gt; planted bugs, each one backed by numbers from the run:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Walking off screen:&lt;/strong&gt; the player reached x=720 on a 480px-wide screen. It saw this five separate times while holding right.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Score after death:&lt;/strong&gt; the score went from 16 at death to 146 two minutes later, with no game running.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Stuck after Game Over:&lt;/strong&gt; the Start button stayed hidden for 120 seconds, and Enter did nothing in 10 tries. It called it a softlock, which is exactly right.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fast diagonal:&lt;/strong&gt; it measured about 580 px/s diagonally against 400 px/s straight, so roughly 41% faster.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It also found a fifth issue I hadn't planted: the hidden player moves when you hold a key on the title screen, before any game starts. That's a real bug in the demo. It flagged one more thing from code reading only (D-pad mappings that look swapped) and was honest that it hadn't checked that one in play.&lt;/p&gt;

&lt;p&gt;No false alarms. Every claim pointed to a moment in the trace.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it cost
&lt;/h2&gt;

&lt;p&gt;The run went through my Aura plan, and the report logged &lt;strong&gt;about $1.11&lt;/strong&gt; of model usage.&lt;/p&gt;

&lt;p&gt;One surprise: I'd set the coding agent to OpenCode with Gemini, but in Aura mode the report shows Aura ran it on Claude Opus 5.5 for me. So on Aura you don't need a Claude account to get Claude doing the code-reading part. If you bring your own keys instead, you choose the model and pay that provider plus TypeSafe for Jev.&lt;/p&gt;

&lt;p&gt;A few things to know before you try it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I couldn't test the &lt;code&gt;autopilot connect claude&lt;/code&gt; or &lt;code&gt;connect codex&lt;/code&gt; path. That hooks Autopilot into your own Claude Code or Codex, and I don't use either plan.&lt;/li&gt;
&lt;li&gt;Run sharing is &lt;strong&gt;on by default&lt;/strong&gt;. I turned it off with &lt;code&gt;AUTOPILOT_SHARE=0&lt;/code&gt; and &lt;code&gt;--playtest-share=0&lt;/code&gt;. Do the same if your game is private.&lt;/li&gt;
&lt;li&gt;Without an Aura account or a TypeSafe key, it stops about 20 seconds in. Jev is the player, and there's no swapping it out.&lt;/li&gt;
&lt;li&gt;A small Linux gotcha: if something already uses port 8765, start the service with &lt;code&gt;--port&lt;/code&gt; and pass &lt;code&gt;--playtest-url&lt;/code&gt; to the game. That cost me two 2-minute timeouts.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The baseline: just ask an LLM to read the code
&lt;/h2&gt;

&lt;p&gt;For comparison, I gave Gemini the full source of the bugged game in one prompt: "play it in your head and list every bug a player would hit."&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;gemini-3-flash-preview&lt;/strong&gt; (31.6s, about 6.7K tokens in):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Found &lt;strong&gt;all 4&lt;/strong&gt; planted bugs, with the right file and line for each.&lt;/li&gt;
&lt;li&gt;Listed 5 extra issues. A couple were fair, like the sprite staying upside-down after moving down and then sideways. The rest were guesses or design choices, like the trail particles always emitting.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;gemini-3.1-flash-lite&lt;/strong&gt; (25.6s):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Found &lt;strong&gt;1 of 4&lt;/strong&gt;: the score that keeps counting.&lt;/li&gt;
&lt;li&gt;Missed the stuck Start button, the off-screen hiding spot and the fast diagonal.&lt;/li&gt;
&lt;li&gt;Padded the list with timing "races" that don't really happen in play.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both runs were free on the Gemini free tier.&lt;/p&gt;

&lt;h2&gt;
  
  
  Autopilot vs a plain LLM
&lt;/h2&gt;

&lt;p&gt;On my little test, here's how they compare:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Bugs found:&lt;/strong&gt; Autopilot 4/4 plus one real extra. Gemini Flash 4/4 plus five extras, mostly noise. Flash-Lite 1/4.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Time:&lt;/strong&gt; Autopilot about 4 minutes. Gemini about 30 seconds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost:&lt;/strong&gt; Autopilot about $1.11. Gemini $0 on the free tier.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Evidence:&lt;/strong&gt; Autopilot gave me measured numbers, a trace and a video. Gemini gave me line numbers and a confident guess.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So on speed and cost, the plain LLM wins easily, and that's fair: my bugs were all visible in the code. If that's your kind of bug, a good model reading your source is hard to beat.&lt;/p&gt;

&lt;p&gt;But the difference I felt was trust. Gemini &lt;em&gt;said&lt;/em&gt; I could hide off-screen. Autopilot walked there and showed me x=720. Gemini mixed real bugs with design choices, and I had to sort them myself. Autopilot's list had nothing to throw away, and it found a real bug I didn't know the demo had.&lt;/p&gt;

&lt;p&gt;And code-reading has a hard ceiling. It can't catch bugs that only show up in play, like physics, timing, level layout or a door you can clip through. That's the space Autopilot is built for, and my test didn't even reach it.&lt;/p&gt;

&lt;p&gt;My result doesn't confirm Aura's "43% more accurate, 2x faster" claim, and it doesn't refute it. That was against other playing agents on 7 Unreal tasks. What I can say is that on a cold run, with a lazy prompt, on Linux, which isn't even officially supported, it caught everything I hid.&lt;/p&gt;

&lt;h2&gt;
  
  
  Should you try it?
&lt;/h2&gt;

&lt;p&gt;My take:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Yes, if you ship on Unreal, Unity or Godot&lt;/strong&gt; and have repeatable QA checks ("can I still open the inventory after a new game?"). Verify and goal prompts with pass/fail exit codes fit straight into CI.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use a plain LLM first for cheap code review.&lt;/strong&gt; It's faster and free. Bring Autopilot in for what code can't prove.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Expect it to get better per game.&lt;/strong&gt; The memory design is the real idea here. My next run starts from a harness that already works.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Read the benchmark as a start.&lt;/strong&gt; Seven tasks is a demo, not a verdict. Run it on your own game.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The repo is at github.com/ramenvr/autopilot. If you try it on your own project, tell me what it finds. I'd love to hear about a bug that only shows up in play.&lt;/p&gt;




&lt;p&gt;Originally published on Medium: &lt;a href="https://medium.com/@ishank.iandroid/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-081fb9246738" rel="noopener noreferrer"&gt;https://medium.com/@ishank.iandroid/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-081fb9246738&lt;/a&gt;&lt;/p&gt;

</description>
      <category>godot</category>
      <category>gamedev</category>
      <category>ai</category>
      <category>playtesting</category>
    </item>
    <item>
      <title>HackerShelf: The Books Hacker News Recommends, Rebuilt from Karpathy's HackerTube with Cursor</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Tue, 03 Nov 2026 04:56:38 +0000</pubDate>
      <link>https://dev.to/ishank-dev/hackershelf-the-books-hacker-news-recommends-rebuilt-from-karpathys-hackertube-with-cursor-4ggd</link>
      <guid>https://dev.to/ishank-dev/hackershelf-the-books-hacker-news-recommends-rebuilt-from-karpathys-hackertube-with-cursor-4ggd</guid>
      <description>&lt;p&gt;On 1 Nov, Andrej Karpathy shared HackerTube (&lt;a href="https://hackertube.org/" rel="noopener noreferrer"&gt;https://hackertube.org/&lt;/a&gt;). He had Claude filter 43M Hacker News comments from 20 years down to 6,454 recommended videos, and he said the site took about 2 hours with Opus 5.5 in Claude Code. He also published the data as a public zip.&lt;/p&gt;

&lt;p&gt;As a Lead SWE, my first question was whether I could pull off the same trick with a different topic and a different tool. I picked books, because HN comment threads are basically one long reading list. I picked Cursor, because that's the tool I use every day.&lt;/p&gt;

&lt;p&gt;The result is HackerShelf: &lt;a href="https://ishankdev.github.io/hackershelf/" rel="noopener noreferrer"&gt;https://ishankdev.github.io/hackershelf/&lt;/a&gt; (code: &lt;a href="https://github.com/IshankDev/hackershelf" rel="noopener noreferrer"&gt;https://github.com/IshankDev/hackershelf&lt;/a&gt;).&lt;/p&gt;

&lt;h2&gt;
  
  
  How Karpathy's pipeline works
&lt;/h2&gt;

&lt;p&gt;His DATA.md lays out the method, and I copied it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Scan every HN comment from 2007 to 2026 in the open-index/hacker-news dataset for YouTube links. The comment needs at least 100 characters of its own text.&lt;/li&gt;
&lt;li&gt;Keep a video only if at least 3 different people mentioned it in at least 3 different months.&lt;/li&gt;
&lt;li&gt;Claude Sonnet 5.5 labels each mention with a stance, topics, a reason and a quote.&lt;/li&gt;
&lt;li&gt;Sonnet writes a card for each video, then puts it into 1 to 3 categories.&lt;/li&gt;
&lt;li&gt;Opus 5.5 in Claude Code runs the whole thing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;He did a few steps by hand: a Google Cloud API key for YouTube metadata, a VPS and a domain.&lt;/p&gt;

&lt;h2&gt;
  
  
  My version: HackerShelf
&lt;/h2&gt;

&lt;p&gt;Same idea, but for books:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Source:&lt;/strong&gt; HN comments from Jan 2024 to Aug 2026 from open-index/hacker-news, 9,437,609 comments in total&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pre-filter:&lt;/strong&gt; DuckDB regex for book links (Amazon, Goodreads, Open Library, Bookshop, ISBNs, O'Reilly, Manning, PragProg, No Starch) and phrases like "must read", "worth a read" and "book called". This took about 9 seconds and left 11,519 candidates.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Extraction:&lt;/strong&gt; gemini-3.5-flash-lite in 288 batches of 40 found 6,076 book mentions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Resolve:&lt;/strong&gt; Open Library matched 2,212 works. Using Karpathy's rule (at least 3 people, at least 3 months), 151 books were left.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cards:&lt;/strong&gt; one Gemini call per book, so 151 calls&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Site:&lt;/strong&gt; a static site on GitHub Pages&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The final shelf has &lt;strong&gt;151 books, 723 mentions and 601 unique recommenders&lt;/strong&gt;, sorted into 9 categories.&lt;/p&gt;

&lt;h2&gt;
  
  
  Time: Karpathy's 2 hours vs. mine
&lt;/h2&gt;

&lt;p&gt;Being honest, my total was longer than his.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Planning and data checks:&lt;/strong&gt; 7:47 to 7:50 PM IST&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cursor build&lt;/strong&gt; (pipeline code, the full site and deploy setup): about 40 minutes, 7:56 to 8:37 PM IST&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemini extraction:&lt;/strong&gt; about 20 minutes, about 8:50 to 9:11 PM IST&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Open Library resolve and filter:&lt;/strong&gt; about 44 minutes, 9:24 to 10:08 PM IST, with no LLM. This was the slowest step.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cards:&lt;/strong&gt; 509 seconds, 10:09 to 10:17 PM IST&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Site data:&lt;/strong&gt; under 1 second, at 10:17 PM IST&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From start to finish that's about &lt;strong&gt;2.5 hours of wall clock&lt;/strong&gt; (7:47 to about 10:17 PM IST). That includes my own manual steps: creating the repo, giving Cursor GitHub access and turning on GitHub Pages.&lt;/p&gt;

&lt;p&gt;Cursor's part, writing the actual code, took about 40 minutes. Most of the remaining time went to calling APIs and waiting on rate limits.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cost
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Tokens:&lt;/strong&gt; 1,968,177 in and 623,943 out&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What I paid:&lt;/strong&gt; $0, on Gemini's free tier. I hit 429s and handled them with backoff and retry.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Paid-rate equivalent:&lt;/strong&gt; at $0.30 in and $2.50 out per 1M tokens, about $0.59 + $1.56 = &lt;strong&gt;about $2.15&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cursor usage cost:&lt;/strong&gt; I don't know this number yet, so I won't guess.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What went wrong
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The first Cursor run shipped sample data.&lt;/strong&gt; The site worked, but it only had 9 placeholder books. The real data had to come from a separate run of the LLM stage.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Too many candidates.&lt;/strong&gt; The first broad filter matched about 392k comments that might mention a book, which was far too many for a free tier. Tighter link and phrase rules cut that down to 11,519.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The model paraphrased.&lt;/strong&gt; 470 of the extracted "quotes" weren't actually word for word in the source comment. I didn't trust them, so I blanked them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Open Library isn't perfect.&lt;/strong&gt; One top-10 entry resolved as "1984 (adaptation)" by Michael Dean instead of Orwell's novel. Fuzzy title matching is the weak link here.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Does it hold up?
&lt;/h2&gt;

&lt;p&gt;I spot-checked 10 books and 83 mentions against the original HN comments. &lt;strong&gt;83 out of 83&lt;/strong&gt; had the right author, and every quote was verbatim in its source comment. For example:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"I recommend reading The Mom Test, it's a great book about how to navigate this early validation stage"&lt;/li&gt;
&lt;li&gt;"Robert Nystrom's Crafting Interpreters [0] is a really fantastic book on the topic."&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The top 10 books Hacker News recommends (2024 to 2026)
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Designing Data-Intensive Applications&lt;/strong&gt;, Martin Kleppmann: 13 recommenders, 14 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Amusing Ourselves to Death&lt;/strong&gt;, Neil Postman: 11 recommenders, 11 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Making of the Atomic Bomb&lt;/strong&gt;, Richard Rhodes: 11 recommenders, 12 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Permutation City&lt;/strong&gt;, Greg Egan: 11 recommenders, 11 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Mom Test&lt;/strong&gt;, Rob Fitzpatrick: 10 recommenders, 10 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A Philosophy of Software Design&lt;/strong&gt;, John Ousterhout: 9 recommenders, 11 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Blindsight&lt;/strong&gt;, Peter Watts: 11 recommenders, 11 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Chip War&lt;/strong&gt;, Chris Miller: 9 recommenders, 9 mentions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;1984&lt;/strong&gt;: 10 recommenders, 10 mentions (Open Library mismatched this one, see above)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Children of Time&lt;/strong&gt;, Adrian Tchaikovsky: 11 recommenders, 11 mentions&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The ranking uses a score that weighs strong recommendations above passing mentions, which is why a book with 9 recommenders can rank above one with 11.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I took away
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Karpathy's recipe works beyond videos. The "3 people, 3 months" rule is a surprisingly good filter for noise.&lt;/li&gt;
&lt;li&gt;An AI coding tool can write the code in under an hour. The time goes into data, rate limits and the manual steps.&lt;/li&gt;
&lt;li&gt;Don't trust LLM-extracted quotes until you check them against the source. 470 were quietly wrong.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Credits
&lt;/h2&gt;

&lt;p&gt;All credit for the idea and method goes to Andrej Karpathy and HackerTube. The comment data comes from open-index/hacker-news under ODC-By 1.0. The comments themselves are still their authors' work, and every mention on HackerShelf links back to the original comment.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/hackershelf-the-books-hacker-news-recommends-rebuilt-from-karpathys-hackertube-with-cursor-7a9cff210bc5" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>cursor</category>
      <category>hackernews</category>
      <category>llm</category>
    </item>
    <item>
      <title>NVIDIA Nemotron 3.5 Bolt E4B: Running an Offline Voice Agent With Tool Calling on a CPU</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Wed, 28 Oct 2026 04:46:48 +0000</pubDate>
      <link>https://dev.to/ishank-dev/nvidia-nemotron-35-bolt-e4b-running-an-offline-voice-agent-with-tool-calling-on-a-cpu-3m5g</link>
      <guid>https://dev.to/ishank-dev/nvidia-nemotron-35-bolt-e4b-running-an-offline-voice-agent-with-tool-calling-on-a-cpu-3m5g</guid>
      <description>&lt;p&gt;NVIDIA released Nemotron 3.5 Bolt E4B on October 23. The pitch is a small model tuned for agents: it calls tools, follows instructions, and is small enough for laptops, Jetsons and other edge hardware. The model card's benchmarks are tested on H100s, RTX 5090s and DGX Spark. I wanted to see what happens without any of those.&lt;/p&gt;

&lt;p&gt;So I took the official 4-bit GGUF, ran it on a plain 8-vCPU Xeon VM with 16 GB of RAM and no GPU, and built a small voice assistant around it. It listens to a WAV file, picks a tool, calls it, and speaks the answer back. Everything below is from my own runs. I've kept the raw CSVs for every number.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Bolt E4B actually is
&lt;/h2&gt;

&lt;p&gt;A few facts from the model card and the files themselves, because the name hides some of them:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Architecture:&lt;/strong&gt; Gemma 4 E4B, post-trained by NVIDIA with reinforcement learning (GRPO via NeMo RL) for tool use, instruction following and multi-turn chat. About 8B total parameters and about 4B "effective" ones.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;License:&lt;/strong&gt; OpenMDW-1.1 on NVIDIA's release. The base Gemma 4 weights are Apache 2.0.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context:&lt;/strong&gt; 64K tokens. &lt;strong&gt;Text in, text out.&lt;/strong&gt; There's no audio in the model, so a voice agent still needs separate speech-to-text and text-to-speech.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Weights:&lt;/strong&gt; the official repo is &lt;code&gt;nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB&lt;/code&gt;, with a single &lt;code&gt;Q4_K_M&lt;/code&gt; GGUF. NVIDIA also ships a BF16 checkpoint for vLLM and a DFlash speculative-decoding drafter aimed at GPUs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Recommended serving:&lt;/strong&gt; llama.cpp with &lt;code&gt;--jinja&lt;/code&gt;, sampling at temperature 1.0 and top_p 0.95, and a chat-template switch for "thinking" on or off.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The first surprise: the "4GB" file is &lt;strong&gt;6.18 GB on disk&lt;/strong&gt;. I opened the GGUF and summed the tensors:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tensor group&lt;/th&gt;
&lt;th&gt;Size&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Per-layer embedding tables (mostly Q8_0)&lt;/td&gt;
&lt;td&gt;3.05 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transformer blocks (Q4_K + Q6_K)&lt;/td&gt;
&lt;td&gt;2.40 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Token embeddings&lt;/td&gt;
&lt;td&gt;0.71 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The per-layer embeddings are lookup tables: each token only reads a few rows. That's what makes the "4GB" label fair at run time, as you'll see in the memory section. You still need about 6.2 GB of disk space and download time, though.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Hardware:&lt;/strong&gt; 8 vCPU Intel Xeon (AVX-512, VNNI), 16 GB RAM, no swap, no GPU.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Runtime:&lt;/strong&gt; llama.cpp CPU build b11913, using &lt;code&gt;llama-server&lt;/code&gt; as an OpenAI-compatible endpoint.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Speech-to-text:&lt;/strong&gt; faster-whisper, &lt;code&gt;base.en&lt;/code&gt; and &lt;code&gt;small.en&lt;/code&gt;, int8 on CPU.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Text-to-speech:&lt;/strong&gt; Piper with the &lt;code&gt;en_US-lessac-medium&lt;/code&gt; voice.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The server command was nothing special:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;llama-server &lt;span class="nt"&gt;-m&lt;/span&gt; Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf &lt;span class="nt"&gt;--alias&lt;/span&gt; nemotron-bolt &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--host&lt;/span&gt; 127.0.0.1 &lt;span class="nt"&gt;--port&lt;/span&gt; 8000 &lt;span class="nt"&gt;-c&lt;/span&gt; 8192 &lt;span class="nt"&gt;-np&lt;/span&gt; 1 &lt;span class="nt"&gt;-t&lt;/span&gt; 8 &lt;span class="nt"&gt;--jinja&lt;/span&gt; &lt;span class="nt"&gt;--metrics&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The model loaded in 2.5 to 6 seconds. My first sanity check was 17 Ã— 23, and it answered 391.&lt;/p&gt;

&lt;h2&gt;
  
  
  Raw speed on 8 CPU cores
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;llama-bench&lt;/code&gt;, 8 threads, 3 repetitions:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Test&lt;/th&gt;
&lt;th&gt;Tokens/s&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Prompt processing (512 tokens)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;194.6 Â± 32.8&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Generation (128 tokens)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;10.9 Â± 2.8&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;About 11 tokens per second of generation is slow next to a GPU, but it's enough for an assistant whose replies are one or two sentences. Prefill speed matters more than I expected. My system prompt plus five tool schemas came to about 500 tokens, and the very first request took 6.4 seconds before anything came back. After that, llama.cpp's prompt cache kept the prefix warm, and only the new user message needed processing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Test 1: tool calling, 20 prompts Ã— 3 runs
&lt;/h2&gt;

&lt;p&gt;I defined five mock tools, the kind a voice assistant would have:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;get_weather(city)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;calculate(expression)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;create_event(title, date, time, duration_min)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;set_timer(minutes, label)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;control_light(room, state, brightness)&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The system prompt gave the date (Tuesday 2026-10-27, IST) and asked for short spoken replies. I wrote 20 prompts: three per tool, plus five where the right move is not to call any tool. Those were a joke, a definition, a "thanks", a trivia question, and "order me a pizza", which no tool can do.&lt;/p&gt;

&lt;p&gt;Each response was scored on two things: did it pick the right tool (or correctly pick none), and were the arguments exactly right?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Thinking off&lt;/strong&gt; (the mode I'd use for voice), three full runs:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Right tool, or correctly no tool&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;60 / 60&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exactly correct arguments&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;57 / 60 (95%)&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median time to first token&lt;/td&gt;
&lt;td&gt;0.87â€“1.15 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median end-to-end per prompt&lt;/td&gt;
&lt;td&gt;2.1â€“3.6 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median generated tokens&lt;/td&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Some results I liked:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"Set a timer for 45 seconds" became &lt;code&gt;minutes: 0.75&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;"An hour and a half" became &lt;code&gt;duration_min: 90&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;"Dim the bedroom to 30%" became &lt;code&gt;state: "on", brightness: 30&lt;/code&gt;. It didn't treat "dim" as "off".&lt;/li&gt;
&lt;li&gt;"18% GST on 4999" became &lt;code&gt;4999 * 0.18&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;None of the five no-tool prompts triggered a tool. For the pizza request it said plainly that it couldn't order food, rather than forcing the request into &lt;code&gt;create_event&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The miss was the same in all three runs. "Put sprint planning on my calendar for Friday at 11 for an hour" got the right title, time and duration, but the date was &lt;strong&gt;2026-11-06, a week too late&lt;/strong&gt;. With today being Tuesday the 27th, "Friday" should be the 30th. Relative-date math is where a 4B model shows its size.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Thinking on&lt;/strong&gt;, one run of the same 20 prompts:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Right tool&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;20 / 20&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exactly correct arguments&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;20 / 20&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median end-to-end per prompt&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;13.1 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median generated tokens&lt;/td&gt;
&lt;td&gt;150&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;With reasoning on, it got Friday right (2026-10-30). That cost about 350 characters of reasoning and 17.6 seconds for that one prompt. The first token still arrives fast, but it's a reasoning token rather than an answer. Overall that's roughly &lt;strong&gt;5x slower per turn&lt;/strong&gt; for one extra correct argument out of 20. For a voice assistant on CPU, I'd keep thinking off and handle dates in code: pass the model a resolved calendar ("Friday = 2026-10-30") in the prompt, or let the tool parse "Friday" itself.&lt;/p&gt;

&lt;p&gt;One llama.cpp detail if you're measuring this yourself: tool-call output streams in late, once the call has been parsed. So TTFT for a tool call is close to the time the whole call is ready. For plain-text replies, the first token showed up in 0.36 to 0.6 seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Test 2: a full offline voice turn
&lt;/h2&gt;

&lt;p&gt;Next I wired up the whole loop:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Piper speaks the user's request into a WAV file, as a stand-in for a microphone.&lt;/li&gt;
&lt;li&gt;faster-whisper transcribes it.&lt;/li&gt;
&lt;li&gt;Bolt picks a tool (thinking off).&lt;/li&gt;
&lt;li&gt;The mock tool runs.&lt;/li&gt;
&lt;li&gt;Bolt turns the tool result into a one-sentence reply.&lt;/li&gt;
&lt;li&gt;Piper speaks the reply into another WAV.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Six utterances, timed stage by stage. Medians:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stage&lt;/th&gt;
&lt;th&gt;Whisper base.en&lt;/th&gt;
&lt;th&gt;Whisper small.en&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Speech-to-text&lt;/td&gt;
&lt;td&gt;0.35 s&lt;/td&gt;
&lt;td&gt;0.85 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;LLM: choose tool + args&lt;/td&gt;
&lt;td&gt;2.39 s&lt;/td&gt;
&lt;td&gt;1.85 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;LLM: spoken reply from tool result&lt;/td&gt;
&lt;td&gt;1.48 s&lt;/td&gt;
&lt;td&gt;1.51 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Text-to-speech (Piper)&lt;/td&gt;
&lt;td&gt;0.08 s&lt;/td&gt;
&lt;td&gt;0.07 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Whole turn&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;4.08 s&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;4.58 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The LLM is nearly all of the turn time. Piper is basically free: it made a 4-second reply in 70 to 100 ms. Whisper is cheap too. That means most of the latency budget goes to two model calls: one to pick the tool and one to phrase the answer.&lt;/p&gt;

&lt;p&gt;What the transcripts showed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;base.en heard "Pune" as "Pooner". Bolt still called &lt;code&gt;get_weather(city="Pune")&lt;/code&gt; correctly.&lt;/li&gt;
&lt;li&gt;base.en turned "Tell me a one-line joke about compilers" into "Tell me what you want to do." Bolt sensibly listed its abilities, but the joke never happened. small.en got the sentence right, and Bolt told the joke.&lt;/li&gt;
&lt;li&gt;"Book a dentist appointment tomorrow at four thirty p m" became &lt;code&gt;date: 2026-10-28, time: 16:30&lt;/code&gt;, which is correct.&lt;/li&gt;
&lt;li&gt;The 18%-of-4,999 answer was "about 900" in one run and "899.82" in another. At temperature 1.0 the wording varies, so if exact numbers matter, put them in your reply template.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;My takeaway: &lt;strong&gt;small.en is worth its extra half second&lt;/strong&gt;, because recognition errors cost far more than ASR latency. Since the model handles fuzzy input well, I'd spend the speed budget on the ASR side.&lt;/p&gt;

&lt;h2&gt;
  
  
  Memory: where the "4GB" name comes from
&lt;/h2&gt;

&lt;p&gt;On Linux, llama.cpp memory-maps the GGUF, so &lt;code&gt;top&lt;/code&gt; can show a scary number. Right after load, RSS was about 9.6 GB, because the file pages had just been read. Once the system had been running for a while, the server process looked like this:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Anonymous (actually allocated) memory, 8K context, 1 slot&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;3.8 GB&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RSS after the page cache settled&lt;/td&gt;
&lt;td&gt;4.1 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same server with 16K context and 4 slots (anonymous)&lt;/td&gt;
&lt;td&gt;3.7 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Voice stack (Whisper small.en + Piper) in a separate process&lt;/td&gt;
&lt;td&gt;0.66 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;So about 4 GB of RAM for the model, plus about 0.7 GB for speech. The large embedding tables mostly stay on disk until a token needs a row. That matches the name, and it's a key reason this model fits small devices. Just plan for the 6.2 GB download and the disk space.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limitations, honestly
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;CPU only.&lt;/strong&gt; I didn't test the DFlash speculative drafter or any GPU, so these numbers are a floor, not a ceiling. NVIDIA's GPU numbers will look very different.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Small test set.&lt;/strong&gt; 20 prompts Ã— 3 runs is a smoke test, not BFCL. For a fuller picture, NVIDIA reports 67.3 on BFCL-v3 and 74.1 on ToolSandbox for this Q4_K_M build.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Single tool call per turn.&lt;/strong&gt; I didn't test parallel or chained tool calls.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Synthetic audio.&lt;/strong&gt; My "microphone" was Piper's own voice, which is clean speech. Real rooms, accents and background noise will make Whisper's job harder.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dates.&lt;/strong&gt; With thinking off, it misread "Friday" in every run. Resolve dates in code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Variance.&lt;/strong&gt; My first run happened while other jobs were finishing, and its median turn was 3.6 s against 2.1 s in the clean third run. Run your own benchmarks more than once.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Would I use it?
&lt;/h2&gt;

&lt;p&gt;For a local assistant that turns speech into a handful of well-defined actions (timers, lights, calendar, lookups), yes. On a CPU-only VM it picked the right tool 60 out of 60 times, filled the arguments correctly 95% of the time, and completed a whole spoken turn in about four and a half seconds, with no network calls and about 4.7 GB of RAM including speech.&lt;/p&gt;

&lt;p&gt;Where I'd be careful: anything that needs arithmetic on dates or times, long open-ended answers at 11 tokens per second, and noisy audio with a tiny Whisper model. Each of those has a simple fix: resolve dates in code, keep replies short, and use small.en.&lt;/p&gt;

&lt;p&gt;Next I want to run the same loop on a Jetson-class board with the DFlash drafter and see how far that 4.5 seconds drops.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/nvidia-nemotron-3-5-bolt-e4b-running-an-offline-voice-agent-with-tool-calling-on-a-cpu-66ab95a5868a" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>nvidia</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Claude Duet vs Grok Bot vs Muse vs Dots: Always-On AI Agents Compared (2026)</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Sun, 25 Oct 2026 04:47:55 +0000</pubDate>
      <link>https://dev.to/ishank-dev/claude-duet-vs-grok-bot-vs-muse-vs-dots-always-on-ai-agents-compared-2026-526n</link>
      <guid>https://dev.to/ishank-dev/claude-duet-vs-grok-bot-vs-muse-vs-dots-always-on-ai-agents-compared-2026-526n</guid>
      <description>&lt;p&gt;Over six weeks, every big AI lab has shipped the same idea. You don't get a chatbot that waits for you to type. You get an agent with its own computer that works on a schedule, connects to your apps, and asks before it does anything risky.&lt;/p&gt;

&lt;p&gt;I'm a Lead Software Engineer, and I already run one of these every day for my writing workflow. So when Anthropic's Claude Duet launch post passed 41,785 likes and 5.16M views in under a day, I wanted to know how the four actually compare. Hype reels weren't going to answer that.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How I tested.&lt;/strong&gt; I live in India. I have ChatGPT Go, not Pro, and no Claude subscription. Meta's Muse isn't available in my country. So:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot (xAI): tested hands-on.&lt;/strong&gt; I've used it daily for weeks, and I ran the specific tests below for this post.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Claude Duet (Anthropic): docs-based, not tested.&lt;/strong&gt; I used Anthropic's product page, launch article, help centre and demo.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse (Meta): docs-based, not tested.&lt;/strong&gt; I used Meta's newsroom post and help centre.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots (OpenAI): docs-based, not tested.&lt;/strong&gt; I used OpenAI's launch post and help centre.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I checked every price on the official pages on 22 Oct 2026.&lt;/p&gt;




&lt;h2&gt;
  
  
  What "always-on" actually means
&lt;/h2&gt;

&lt;p&gt;All four share the same basic design:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;A cloud computer of its own.&lt;/strong&gt; It has a browser and sometimes a shell, so the agent can work while your laptop is shut.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Proactive runs.&lt;/strong&gt; Scheduled briefs, monitors and "tell me when" triggers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Connectors.&lt;/strong&gt; Mail, calendar, docs and code, all through your own accounts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An approval gate.&lt;/strong&gt; It drafts, and you press the button for anything that sends, pays or deletes.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;They differ on price, where they're available, and how much of that computer you actually get to use.&lt;/p&gt;




&lt;h2&gt;
  
  
  The comparison
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Price (official pages, checked 22 Oct 2026)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Claude Duet:&lt;/strong&gt; no extra cost on Claude &lt;strong&gt;Pro and Max&lt;/strong&gt; during the research preview. It uses your plan's usage limits, and its background checks count against them. &lt;em&gt;(claude.com, Duet help article)&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; &lt;strong&gt;from $30/mo&lt;/strong&gt; (Premium, via X Subscription), with Super at $100 and Ultra at $200. At the 11 Aug 2026 beta launch there was no standalone price. Access came bundled with Cursor Premium Teams ($120/seat), Cursor Ultra ($200) and SuperGrok Heavy ($300), which is where the "$120–300" number in the reels comes from. &lt;em&gt;(x.ai/bot, x.ai news)&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; &lt;strong&gt;free tier&lt;/strong&gt; with a weekly limit (card required), &lt;strong&gt;Power $20/mo&lt;/strong&gt; and &lt;strong&gt;Maximum $100/mo&lt;/strong&gt;. &lt;em&gt;(Meta help centre)&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; needs &lt;strong&gt;ChatGPT Pro&lt;/strong&gt;, or the Business Premium/Enterprise beta. Free, Go and Plus are excluded. The cheapest Pro tier is $100, which is &lt;strong&gt;₹10,699/mo&lt;/strong&gt; on the Indian App Store. &lt;em&gt;(openai.com, OpenAI help centre)&lt;/em&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Availability
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duet:&lt;/strong&gt; US-only research preview with a waitlist. Not on Team or Enterprise plans. Works in the iOS/Android apps and on claude.ai.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; generally available on paid plans, and I use it from India.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; US first, plus Canada and Mexico per Meta's coverage. &lt;strong&gt;Not available in India.&lt;/strong&gt; Works in the app, on the web and in WhatsApp.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; launched 29 Sep 2026 with a gradual rollout. Pro is excluded only in the EEA, Switzerland and the UK, so India qualifies on paper.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Its own computer
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duet:&lt;/strong&gt; a cloud VM with its own browser. You can't yet watch or take over that browser, and it checks in every few hours.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; a full persistent Linux box with a shell, a browser desktop you can watch or take over, and files that stay between sessions. &lt;em&gt;(Tested, results below.)&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; the "Muse Secure VM". Logins happen there, and Meta's line is that &lt;strong&gt;Muse never sees your passwords&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; its own cloud computer, running GPT-6 Astra.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Approvals
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duet:&lt;/strong&gt; asks before it sends, pays, signs or deletes. It &lt;strong&gt;can make payments you approve&lt;/strong&gt; (Anthropic's example is a $640 permit fee), but it &lt;strong&gt;can't move money between accounts&lt;/strong&gt; or trade securities, even with approval.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; outbound email and messages become a draft card, and nothing leaves until I press Send. Risky actions on its computer get an automatic safety review. &lt;em&gt;(Tested, see below.)&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; payments go through one-time Link by Stripe cards.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; approval before consequential actions, per OpenAI's help centre.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Connectors
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duet:&lt;/strong&gt; "more than 6,000 connectors and plugins".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; MCP connectors (Gmail, Notion, GitHub, X and more), plus anything its browser can reach.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; Meta apps plus partner integrations (no public count).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; 4,000+ apps.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Memory
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Duet:&lt;/strong&gt; carries over your Claude memory, skills and apps.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok Bot:&lt;/strong&gt; persistent memory and saved skills across chats, plus the files on its box.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Muse:&lt;/strong&gt; personal context across Meta apps (details are thin in the docs).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots:&lt;/strong&gt; builds on your ChatGPT account (memory specifics not spelled out in the launch docs).&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  What the reels got wrong
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;"Duet never sees your passwords."&lt;/strong&gt; That's &lt;strong&gt;Meta's claim about Muse&lt;/strong&gt;. Anthropic says logins run in your own cloud computer "so sensitive details stay private", which is close but not the same promise.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Grok Bot launched at $120–300."&lt;/strong&gt; Those were &lt;strong&gt;bundled plans&lt;/strong&gt;, not a standalone price. It now starts at $30/mo.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Duet can't move money."&lt;/strong&gt; Half right. It can't move money between accounts, but it &lt;strong&gt;can&lt;/strong&gt; make a payment you approve.&lt;/li&gt;
&lt;li&gt;Everything else I checked about Duet holds up, including Pro/Max inclusion, the morning brief, meeting prep, follow-ups, 6,000+ connectors and the US-only waitlist. The A/B-test proposal only appears in the demo.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Grok Bot, hands-on
&lt;/h2&gt;

&lt;p&gt;Every test was read-only or sandboxed. Nothing was sent, posted or deleted.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Its own computer&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Installed a Python CLI (&lt;code&gt;pip install --user cowsay&lt;/code&gt;) in &lt;strong&gt;0.8 s&lt;/strong&gt;, then ran it again in a later session. &lt;strong&gt;It persisted.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Started a &lt;strong&gt;12-minute background job&lt;/strong&gt; and walked away. Result: all 12 one-minute steps ran and the job finished on its own (started 09:52, done 10:11 IST). Honest caveat: the box sat idle for about 8 minutes mid-run, so the wall-clock time stretched from 12 to about 19 minutes. &lt;strong&gt;Pass, but don't count on exact timing.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Produced a CSV of these results and a PNG chart on the box, ready to hand back in chat. &lt;strong&gt;Pass.&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Connectors (read-only)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Gmail:&lt;/strong&gt; searched "Medium" over the last 7 days and got back about &lt;strong&gt;201 matching emails&lt;/strong&gt; (25 per page) in about 4 s. &lt;strong&gt;Correct.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Notion:&lt;/strong&gt; searched "Medium" and found my "Medium Topics" page at the top in about 4 s. &lt;strong&gt;Correct.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub:&lt;/strong&gt; listed my 5 most recently pushed repos, with push dates, in about 4 s. &lt;strong&gt;Correct.&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;X:&lt;/strong&gt; read the Claude Duet launch post's public metrics (41,785 likes, 5,155,927 views, 3,174 reposts, 13,141 bookmarks) in about 3 s. &lt;strong&gt;Correct.&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That's &lt;strong&gt;4/4 connector reads correct&lt;/strong&gt;, each in under 5 s.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Routines (timing)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I scheduled a one-shot routine for &lt;strong&gt;10:05 AM IST&lt;/strong&gt;. It fired at &lt;strong&gt;10:07:58 IST&lt;/strong&gt;, &lt;strong&gt;178 s late&lt;/strong&gt;, and wrote its result to a file on its own. &lt;strong&gt;It fired, but with about a 3-minute drift.&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Approval before send&lt;/strong&gt;&lt;br&gt;
By design, any outbound email or message becomes a draft card in my chat. It shows the recipient, subject and body, and nothing goes out until I press Send. Declining discards it.&lt;br&gt;
In weeks of daily use, every outgoing email or message has stopped at a draft card that only I can send. When the agent hits a login or a 2FA prompt, it hands me its screen rather than typing a password, and actions it can't undo, like deleting things, wait for my explicit yes. I never saw it send anything on its own.&lt;/p&gt;




&lt;h2&gt;
  
  
  Verdict by use case
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;You already pay for Claude Pro/Max and live in the US:&lt;/strong&gt; join the &lt;strong&gt;Duet&lt;/strong&gt; waitlist. You pay nothing extra, your memory carries over, and it has the biggest connector catalogue. The trade-off is preview limits and no control of its browser.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You want a real computer you can watch, script and take over, from anywhere:&lt;/strong&gt; &lt;strong&gt;Grok Bot&lt;/strong&gt;. At $30/mo it's the cheapest paid entry that's available outside the US, and the box behaves like a dev machine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You want something cheap or free inside WhatsApp and Instagram:&lt;/strong&gt; &lt;strong&gt;Muse&lt;/strong&gt;, if you're in a supported country.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You already pay for ChatGPT Pro:&lt;/strong&gt; &lt;strong&gt;Dots&lt;/strong&gt; is the natural pick. At ₹10,699/mo it's hard to justify just for the agent.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;My honest take: these products have converged fast. Approval gates and cloud computers are standard now. The real differences are &lt;strong&gt;where you live, what you already pay for, and how much of the agent's computer you're allowed to touch.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Duet, Muse and Dots sections are docs-based, not tested. Prices checked on official pages 22 Oct 2026.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://medium.com/@ishank.iandroid/claude-duet-vs-grok-bot-vs-muse-vs-dots-always-on-ai-agents-compared-2026-887563c2d19e" rel="noopener noreferrer"&gt;Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>agents</category>
      <category>llm</category>
      <category>productivity</category>
    </item>
    <item>
      <title>ScaleDown CDMs: Fast, Cheap Classify, Extract and Summarize Models Tested</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Sat, 24 Oct 2026 04:48:11 +0000</pubDate>
      <link>https://dev.to/ishank-dev/scaledown-cdms-fast-cheap-classify-extract-and-summarize-models-tested-1adp</link>
      <guid>https://dev.to/ishank-dev/scaledown-cdms-fast-cheap-classify-extract-and-summarize-models-tested-1adp</guid>
      <description>&lt;p&gt;Last week ScaleDown launched Constrained Decoding Models (CDMs). The launch post got about 1.8M views. The pitch is simple: most production LLM calls don't need prose. They need a label, a field, or a short summary. So instead of a general model writing an answer token by token, each CDM does one job, reads your input once, and returns a typed result. Pricing is $0.05 per million input tokens (OpenRouter lists $0.04 with a 20% promo), and output is free.&lt;/p&gt;

&lt;p&gt;The launch post made big claims: "200x faster, 467x cheaper than frontier LLM APIs" and "77.2% on DecisionBench". A thread from Bitroot pushed back the next day. Their point was that the 467x has no stated baseline, DecisionBench has no public source, and nobody should trust a classifier result without a few hundred labelled examples and confidence intervals. That's a fair critique, so I built the test around it.&lt;/p&gt;

&lt;p&gt;Here's what I ran, what held up, and what didn't.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;I'm a Lead SWE, so I tested this the way I'd vet any vendor before routing production traffic to it: public labelled data, the same prompts for every engine, and raw results saved to disk.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Classify:&lt;/strong&gt; 300 CLINC150 test utterances (10 intents plus 50 out-of-scope) and 110 Bitext customer-support tickets across 11 categories.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Extract:&lt;/strong&gt; 50 SROIE receipts (company, date, address, total) and 30 LinkedIn job posts (company, top of pay range, and work type as an enum).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Summarize:&lt;/strong&gt; 6 QMSum meeting transcripts (4k to 9k tokens), 2 long ones (31k tokens), and one 369k-token stress doc.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Baseline:&lt;/strong&gt; Qwen3 1.7B running locally on Ollama (8-core CPU, no GPU), with the same instructions and JSON-schema-constrained output. I didn't have a frontier API key on this box, so for frontier models I did list-price math from the official pricing pages (checked 22 Oct 2026).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everything goes through one OpenAI-compatible base URL. Classify also has a Jev-compatible Decisions endpoint, which is the one I used most:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.scaledown.xyz/v1/scaledown&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x-api-key&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;SCALEDOWN_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]},&lt;/span&gt;
    &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;state&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;how do I reset my pin&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;questions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;intent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;choice&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;criteria&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;pin_change&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;wants to change or reset their PIN&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;balance&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;asks for an account balance&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;oos&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;none of the above / out of scope&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;}}},&lt;/span&gt;
    &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;answers&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;intent&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="c1"&gt;# {'type': 'choice', 'choice': 'pin_change', 'confidence': 0.979, ...}
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each response includes a &lt;code&gt;usage.cost&lt;/code&gt; field. Mine always worked out to exactly $0.05 per million billed input tokens.&lt;/p&gt;

&lt;h2&gt;
  
  
  Classify: the strongest result
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;CLINC150 (n=300, 10 intents + out-of-scope)   accuracy   95% CI          OOS recall
ScaleDown Decisions (classify-1)              97.3%      94.8 - 98.6%    98%
ScaleDown /classify (rubrics)                 96.7%      94.0 - 98.2%    100%
Qwen3 1.7B local                              90.7%      86.8 - 93.5%    86%

Bitext support tickets (n=110, 11 categories)
ScaleDown Decisions                           92.7%      86.3 - 96.3%
ScaleDown /classify                           90.9%      84.1 - 95.0%
Qwen3 1.7B local                              62.7%      53.4 - 71.2%
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The confidence intervals don't overlap with the small local model on either set. On CLINC150 the out-of-scope handling impressed me most. It correctly rejected 49 of 50 out-of-scope queries, and only 6 in-scope queries were wrongly pushed to "oos". That's the failure that hurts in a real router.&lt;/p&gt;

&lt;p&gt;I didn't need any prompt engineering. I wrote one plain-English line per label, and that was it.&lt;/p&gt;

&lt;p&gt;Latency, sequential, measured from a cloud box in India: &lt;strong&gt;p50 162 ms, p95 397 ms&lt;/strong&gt; for Decisions, and p50 134 ms for &lt;code&gt;/classify&lt;/code&gt; with &lt;code&gt;reasoning: false&lt;/code&gt;. The ticket set had a longer tail (p95 about 1.0 s).&lt;/p&gt;

&lt;h2&gt;
  
  
  Extract: good spans, and the misses are interesting
&lt;/h2&gt;

&lt;p&gt;Extract takes a standard &lt;code&gt;response_format&lt;/code&gt; JSON schema. Each property's &lt;code&gt;description&lt;/code&gt; is the search criteria, and every value comes back with a confidence score and character offsets into your input.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;schema&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;object&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;properties&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;company&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;    &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;string&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;description&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;the hiring company&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;s name&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;max_salary&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;string&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;description&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;the top of the stated salary or pay range&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;work_type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;string&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;enum&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Full-time&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Contract&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Part-time&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
                   &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;description&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;employment type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;}}&lt;/span&gt;
&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;extract&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;user&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;job_post&lt;/span&gt;&lt;span class="p"&gt;}],&lt;/span&gt;
    &lt;span class="n"&gt;response_format&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json_schema&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json_schema&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;name&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;job&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;schema&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;schema&lt;/span&gt;&lt;span class="p"&gt;}},&lt;/span&gt;
    &lt;span class="n"&gt;extra_body&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;top_n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;context_chars&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;                        field acc   valid JSON   spans verbatim   p50 latency
Receipts (50 SROIE)
  ScaleDown Extract     89.5%       100%         100%             1.43 s
  Qwen3 1.7B local      89.5%       100%         94%              3.55 s
Job posts (30)
  ScaleDown Extract     90.0%       100%         n/a              0.60 s
  Qwen3 1.7B local      81.1%       100%         n/a              2.76 s
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Per field on receipts, ScaleDown got &lt;strong&gt;100% of totals&lt;/strong&gt; right, compared with 78% for the local model. Totals are the field you actually care about. It was weaker on company names (80% vs 88%).&lt;/p&gt;

&lt;p&gt;Every span ScaleDown returned appeared verbatim in the source text. The citations aren't decoration. You can highlight exactly where a value came from, which is what Bitroot said to check.&lt;/p&gt;

&lt;p&gt;I read every miss, and most were judgment calls rather than hallucinations:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The SROIE gold "date" often includes the time (&lt;code&gt;21 MAR 2018 17:59&lt;/code&gt;). ScaleDown returned just the date.&lt;/li&gt;
&lt;li&gt;It returned &lt;code&gt;KING'S CONFECTIONERY S/B&lt;/code&gt; without the registration number the label includes.&lt;/li&gt;
&lt;li&gt;On job posts it sometimes returned the parent company instead of the brand (&lt;code&gt;ManpowerGroup&lt;/code&gt; for an Experis post, &lt;code&gt;HCA Healthcare&lt;/code&gt; for MountainView Hospital).&lt;/li&gt;
&lt;li&gt;One real error: on one bakery delivery receipt it picked the buyer's name and address instead of the seller's. That template appeared 4 times in my random sample, so it cost 8 field points on its own.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Valid JSON was 100% on both engines, but that's partly because I constrained the local model's output with a schema. The bigger difference is that ScaleDown hands back offsets and a confidence score with no extra work.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summarize: fast and faithful, but watch the query focus
&lt;/h2&gt;

&lt;p&gt;For summaries I passed the QMSum query as the system instruction and the transcript as the user message.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;QMSum meetings, 4k-9k tokens (n=6)   p50       p95       ROUGE-L   summaries w/ invented facts
ScaleDown Summarize                  0.65 s    0.89 s    0.230     0 of 6
Qwen3 1.7B local (CPU)               24.3 s    34.1 s    0.201     2 of 6
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I checked every number and name in each summary against the transcript. ScaleDown's held up, including the details: a 2.39 average rating, the drop from 15 to 12.50 euros, and the 50 million euro profit target were all there in the source. The local model said one rating used "1 to 5" when the meeting used a seven-point scale. It also called an over-budget prototype "reasonably priced".&lt;/p&gt;

&lt;p&gt;Two weaknesses showed up:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Query focus.&lt;/strong&gt; In one meeting it summarised the presentation instead of answering "what did the PM question". On both 31k-token transcripts it summarised an adjacent part of the meeting rather than the exact question. It's a summarizer first and a QA model second.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The 1M-context claim.&lt;/strong&gt; I sent a 369k-token document made of 12 concatenated meetings and asked for one or two sentences per meeting. It accepted the input and returned in 14.2 s. The summary, though, covered only one of the two distinct meeting series in the document and skipped the other entirely. So it can ingest that much, but don't rely on coverage at that length.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Is it really 200x faster?
&lt;/h2&gt;

&lt;p&gt;Not against anything I could measure, and not according to ScaleDown's own docs.&lt;/p&gt;

&lt;p&gt;Against a 1.7B model on my CPU, it was 1.8x faster for short classification, 2.5 to 4.6x for extraction, and about 37x for 6k-token summaries, where CPU prompt processing dominates. A local CPU model isn't a frontier API, though. ScaleDown's own head-to-head benchmark page reports 2.2x to 13.9x faster than each provider's lineup average (2.4x vs OpenAI, 3.8x vs Anthropic). That's a solid speedup, but it's an order of magnitude below the launch post's 200x. I couldn't find what "200x" was measured against.&lt;/p&gt;

&lt;h2&gt;
  
  
  Is it really 467x cheaper?
&lt;/h2&gt;

&lt;p&gt;The price is real. Every call billed exactly $0.05 per million input tokens, and output was free. The ratio, though, depends entirely on the baseline, and there's a catch: &lt;strong&gt;ScaleDown bills more input tokens than your raw prompt&lt;/strong&gt;. Label criteria, schemas and internal prompts are counted. My classify calls billed about 481 tokens, while the same task as a plain LLM prompt was 172 tokens. Extract calls billed about 1,380 tokens against 571.&lt;/p&gt;

&lt;p&gt;Here's the cost per 1,000 calls at list price, using my measured token counts and leaving out reasoning tokens:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;                         classify   extract   summarize (6k tok)
ScaleDown CDM            $0.024     $0.069    $0.317
gpt-5-nano               $0.012     $0.057    $0.333     (0.5x - 1.0x)
gpt-5.4-nano             $0.044     $0.204    $1.300     (1.8x - 4.1x)
gpt-5.4-mini             $0.163     $0.752    $4.860     (6.8x - 15x)
Claude Haiku 4.5         $0.210     $0.931    $6.394     (8.7x - 20x)
Claude Sonnet 4.6        $0.631     $2.793    $19.18     (26x - 61x)
gpt-5.5                  $1.089     $5.015    $32.40     (45x - 102x)
Claude Fable 5           $2.102     $9.311    $63.94     (87x - 202x)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So against the cheapest small models, ScaleDown costs about the same, and on short classification gpt-5-nano is actually half the price. Against flagship models it's 45x to 200x cheaper. You only reach 467x if the frontier model also spends a few hundred reasoning tokens per call. For example, gpt-5.5 classifying with 500 reasoning tokens works out to roughly 670x. That's realistic for a reasoning model doing routing, but it isn't stated anywhere in the launch post. ScaleDown's own docs are more modest: 89x cheaper than the OpenAI average and 161x cheaper than the Anthropic average.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;/classify&lt;/code&gt; endpoint accepts up to 50 &lt;code&gt;chunks&lt;/code&gt; per request and bills the label set once. If you classify at volume, that removes most of the token overhead I measured.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Limitations:&lt;/strong&gt; I didn't benchmark Compress or concurrent load this round; my free credits ran out after ~1M input tokens, since the 50M free tokens were a launch-window promo. After that I got &lt;code&gt;402 Insufficient credits&lt;/code&gt;, so budget for a top-up before you run a real eval.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Bitroot got right
&lt;/h2&gt;

&lt;p&gt;The skeptic thread was right on every point I could check:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;467x has no stated baseline. I can reproduce it only by assuming an expensive reasoning model.&lt;/li&gt;
&lt;li&gt;I still found no public source for DecisionBench's 77.2%.&lt;/li&gt;
&lt;li&gt;Price is the easy part to verify, and it checks out.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Their main advice was to build a test set of 300+ labelled examples and report intervals, and that advice is what made this test worth anything. With n=300, the gap between 97% and 91% is real. With n=30 it wouldn't be.&lt;/p&gt;

&lt;h2&gt;
  
  
  My take
&lt;/h2&gt;

&lt;p&gt;I'd route classification traffic to ScaleDown today. It was the most accurate engine I tested, with clean out-of-scope handling, sub-200 ms p50 latency, and no prompt tuning. Extract is a strong default when you need provenance, because every value comes with an offset you can highlight. Summarize is fast and didn't invent facts in my sample, but test its query focus on your own data, and don't trust its coverage of 300k+ token inputs.&lt;/p&gt;

&lt;p&gt;The marketing ratios are a different story. If you already run gpt-5-nano, you won't save much. If you run flagship or reasoning models for routing and extraction, the savings are large and real.&lt;/p&gt;

&lt;p&gt;Test it on your own data. A few hundred labelled examples will tell you more than any launch post, mine included.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>machinelearning</category>
      <category>benchmark</category>
    </item>
    <item>
      <title>graphify: Turn Your Codebase into a Knowledge Graph for AI Coding Agents</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Mon, 19 Oct 2026 04:55:12 +0000</pubDate>
      <link>https://dev.to/ishank-dev/graphify-turn-your-codebase-into-a-knowledge-graph-for-ai-coding-agents-kb8</link>
      <guid>https://dev.to/ishank-dev/graphify-turn-your-codebase-into-a-knowledge-graph-for-ai-coding-agents-kb8</guid>
      <description>&lt;p&gt;graphify has been all over my timeline this week: it went past 126k GitHub stars, got into Y Combinator and showed up on the Vercel Marketplace. The pitch is simple. Type &lt;code&gt;/graphify&lt;/code&gt; in Claude Code, Codex or Cursor, and your agent queries a graph of your code instead of reading files one by one.&lt;/p&gt;

&lt;p&gt;"Saves tokens" is an easy claim to make and a hard one to check, so I installed it on a plain Linux box and pointed it at a real repo. No cloud trial, no API keys. Here's what happened.&lt;/p&gt;

&lt;h2&gt;
  
  
  What graphify actually is
&lt;/h2&gt;

&lt;p&gt;graphify is an open-source (Apache-2.0) Python CLI plus an agent skill. It parses your project with tree-sitter and writes three files to &lt;code&gt;graphify-out/&lt;/code&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;graph.html&lt;/code&gt;, an interactive force-directed graph you open in a browser&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;GRAPH_REPORT.md&lt;/code&gt;, a summary with "god nodes", communities and surprising connections&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;graph.json&lt;/code&gt;, the full graph your agent queries through &lt;code&gt;explain&lt;/code&gt;, &lt;code&gt;path&lt;/code&gt; and &lt;code&gt;query&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Code extraction is deterministic and local. An LLM only gets involved for docs, PDFs, images and video, or for naming communities, and only if you configure a backend.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setup
&lt;/h2&gt;

&lt;p&gt;My test box runs Debian 13 with Python 3. The PyPI package name has a double y:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python3 &lt;span class="nt"&gt;-m&lt;/span&gt; venv venv
venv/bin/pip &lt;span class="nb"&gt;install &lt;/span&gt;graphifyy      &lt;span class="c"&gt;# installed 0.9.92&lt;/span&gt;
git clone &lt;span class="nt"&gt;--depth&lt;/span&gt; 1 https://github.com/encode/httpx testbed/httpx
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For the testbed I picked httpx: a real, well-known Python HTTP client with about 17,750 lines of Python plus docs and a changelog. It's big enough to be interesting and small enough that I know roughly what the answers should be.&lt;/p&gt;

&lt;h2&gt;
  
  
  Building the graph: 2 seconds, 0 tokens
&lt;/h2&gt;

&lt;p&gt;You don't need an assistant for the code-only build. The &lt;code&gt;update&lt;/code&gt; command does it straight from the CLI:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;cd &lt;/span&gt;testbed/httpx
graphify update &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Re-extracting code files in . (no LLM needed)...
  AST extraction: 101/101 uncached files (100%) [8 workers]
[graphify watch] Rebuilt: 2233 nodes, 4453 edges, 166 communities
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A fresh build took 2.1 seconds. After I edited one file, an incremental rebuild re-parsed only that file and took 1.6 seconds. The report's own footer says &lt;code&gt;Token cost: 0 input · 0 output&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The output is not small: &lt;code&gt;graph.json&lt;/code&gt; is 1.7 MB, &lt;code&gt;graph.html&lt;/code&gt; is 1.9 MB and &lt;code&gt;GRAPH_REPORT.md&lt;/code&gt; is 24 KB. Markdown docs got pulled in as structure too, so &lt;code&gt;CHANGELOG.md&lt;/code&gt; and &lt;code&gt;docs/api.md&lt;/code&gt; show up as nodes.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwnhumn77yew9usecv44o.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwnhumn77yew9usecv44o.png" alt="graph.html for httpx" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What the report got right
&lt;/h2&gt;

&lt;p&gt;The "god nodes" list is honestly a good first page for a new contributor:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;1. Response - 170 edges
2. Request - 111 edges
3. Changelog - 79 edges
4. Client - 69 edges
5. AsyncClient - 51 edges
6. BaseClient - 45 edges
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's httpx's real core. 88% of edges were &lt;code&gt;EXTRACTED&lt;/code&gt; (read directly from the source) and 12% were &lt;code&gt;INFERRED&lt;/code&gt;, with an average confidence of 0.91. I like that every edge carries this tag, because it tells you what to trust.&lt;/p&gt;

&lt;p&gt;"Surprising connections" was less useful. Every entry was a changelog or doc heading linked to &lt;code&gt;Client&lt;/code&gt;, which is true but not surprising.&lt;/p&gt;

&lt;h2&gt;
  
  
  Querying: where the token savings come from
&lt;/h2&gt;

&lt;p&gt;This is what your agent actually runs. I counted every output with tiktoken's cl100k encoding:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;graphify explain &lt;span class="s2"&gt;"httpx/_client.py::Client"&lt;/span&gt;
graphify path &lt;span class="s2"&gt;"httpx/_client.py::Client"&lt;/span&gt; &lt;span class="s2"&gt;"DigestAuth"&lt;/span&gt;
graphify query &lt;span class="s2"&gt;"how does a request get sent through the transport"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1573ew9d1tx7w6l98hck.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1573ew9d1tx7w6l98hck.png" alt="explain output" width="800" height="408"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;explain&lt;/code&gt; on &lt;code&gt;Client&lt;/code&gt; came to 628 tokens. It listed 69 connections with file and line numbers, like &lt;code&gt;--&amp;gt; HTTPTransport [uses] [INFERRED] httpx/_client.py:L731&lt;/code&gt;. Reading &lt;code&gt;_client.py&lt;/code&gt; itself costs 13,659 tokens.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;path&lt;/code&gt; returned a 3-hop chain in a single line: &lt;code&gt;Client &amp;lt;-contains- _client.py -imports_from-&amp;gt; _auth.py -contains-&amp;gt; DigestAuth&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;query&lt;/code&gt; came to 1,728 tokens. It ran a BFS that found 536 nodes but showed only 82, and printed a clear &lt;code&gt;TRUNCATED&lt;/code&gt; warning telling me to raise &lt;code&gt;--budget&lt;/code&gt; to 14720 to see everything.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fscyi5wcg80tgcrziy138.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fscyi5wcg80tgcrziy138.png" alt="path and query output" width="800" height="312"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So yes, for "where is X and what touches it" questions, a graph lookup is roughly 20x cheaper than dumping the file. What it doesn't do is replace reading the code once the agent has to change a function. The graph points you to the right lines; it doesn't contain them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two rough edges
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Ambiguous names.&lt;/strong&gt; &lt;code&gt;graphify explain "Client"&lt;/code&gt; refused to answer, because &lt;code&gt;Client&lt;/code&gt; matched both the class in &lt;code&gt;httpx/_client.py&lt;/code&gt; and a heading in &lt;code&gt;docs/api.md&lt;/code&gt;. The error tells you to use the &lt;code&gt;path::symbol&lt;/code&gt; form, which works, but docs being indexed alongside code means common names collide a lot.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Community labels with a small local model.&lt;/strong&gt; Without an API key, the 166 communities are named after a representative file or symbol (&lt;code&gt;test_content.py&lt;/code&gt;, &lt;code&gt;AsyncByteStream&lt;/code&gt;). graphify can rename them with an LLM, Ollama included:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;openai     &lt;span class="c"&gt;# the Ollama backend needs it; the first run told me so&lt;/span&gt;
graphify label &lt;span class="nb"&gt;.&lt;/span&gt; &lt;span class="nt"&gt;--backend&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;ollama &lt;span class="nt"&gt;--model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;qwen3:1.7b
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On my box, with a 1.7B model on CPU, this ran for more than 11 minutes without finishing, and I killed it. The code graph doesn't need this step at all, but if you want nice labels, give it a real model or an API key. I saw the same kind of small-model limit when I ran &lt;a href="https://medium.com/@ishank.iandroid/i-ran-anatomys-anatomy-skill-on-a-1-7b-local-model-the-kit-passed-every-check-the-figure-was-738-8bdb8bd58cfd" rel="noopener noreferrer"&gt;Anatomy's skill on a 1.7B local model&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Is it worth installing?
&lt;/h2&gt;

&lt;p&gt;For code, yes. It's free, local and fast, and &lt;code&gt;explain&lt;/code&gt;/&lt;code&gt;path&lt;/code&gt; give an agent a cheap map before it starts opening files. I'd add &lt;code&gt;graphify update .&lt;/code&gt; to a git hook and let the agent use the graph first.&lt;/p&gt;

&lt;p&gt;What I didn't test: the &lt;code&gt;/graphify&lt;/code&gt; skill inside a paid assistant, the semantic pass over PDFs and images, and the hosted Graphify Cloud product with its "80% token savings" claim. My 628-vs-13,659 number is one lookup on one repo, not a benchmark of a whole agent session.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/Graphify-Labs/graphify" rel="noopener noreferrer"&gt;github.com/Graphify-Labs/graphify&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>python</category>
      <category>devtools</category>
    </item>
    <item>
      <title>PlxNative: A Fast Native Plex Client for LG webOS TVs, Built in Rust</title>
      <dc:creator>Ishank Choudhary</dc:creator>
      <pubDate>Sat, 17 Oct 2026 04:51:58 +0000</pubDate>
      <link>https://dev.to/ishank-dev/plxnative-a-fast-native-plex-client-for-lg-webos-tvs-built-in-rust-477g</link>
      <guid>https://dev.to/ishank-dev/plxnative-a-fast-native-plex-client-for-lg-webos-tvs-built-in-rust-477g</guid>
      <description>&lt;p&gt;Most smart-TV apps are web pages. On LG's webOS that usually means JavaScript running inside a browser engine, which then hands its output to a compositor before anything reaches the screen. &lt;a href="https://plxnative.com/" rel="noopener noreferrer"&gt;PlxNative&lt;/a&gt; is a Plex client for LG webOS TVs that takes that layer out. Its developer, Gleb Linnik, writes in the &lt;a href="https://github.com/GLinnik21/plx-native" rel="noopener noreferrer"&gt;README&lt;/a&gt;: "No Chromium, no JavaScript, no web view. It just draws."&lt;/p&gt;

&lt;p&gt;This is an explainer, built from the project's README, its docs, its source and its release notes. I don't own an LG TV, so &lt;strong&gt;nothing here was tested on a real television&lt;/strong&gt;. What I did run is the project's own Linux desktop simulator against its mock Plex server, and that section near the end shows exactly what that does and doesn't tell you.&lt;/p&gt;

&lt;h2&gt;
  
  
  What PlxNative is
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;An unofficial Plex client for LG webOS TVs.&lt;/strong&gt; It is not made by or affiliated with Plex or LG; the project says so on its site and in &lt;code&gt;TRADEMARKS.md&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Open source&lt;/strong&gt; under GPL-3.0-or-later, © 2026 Gleb Linnik. The PlxNative name and brand artwork are excluded from the licence.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One person's project.&lt;/strong&gt; The README calls it "one person's spare time".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Current release: v0.8.0&lt;/strong&gt;, published on GitHub on 7 October 2026. Nightly builds already report &lt;code&gt;0.9.0-nightly&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Young and active.&lt;/strong&gt; The repository was created on 3 July 2026, and when I checked on 16 October it had 969 stars and 95 forks, with commits landing the same morning.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Free.&lt;/strong&gt; There's no paid tier. You need a Plex account and a Plex Media Server, and the FAQ confirms a free Plex server works.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why it exists
&lt;/h2&gt;

&lt;p&gt;The origin story is short. The developer's 2019 LG TV still had a great picture, but browsing a Plex library on it "felt slower than it needed to." Rather than tune a web app, he wrote a native one. In his words: "I use it every day to watch things off my server in the next room."&lt;/p&gt;

&lt;p&gt;The site makes the argument with a pair of stacks. A typical TV app goes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;The app (a web page) -&amp;gt; JavaScript -&amp;gt; Browser engine -&amp;gt; Compositor -&amp;gt; OpenGL -&amp;gt; GPU -&amp;gt; Screen
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;PlxNative goes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;The app (native code) -&amp;gt; OpenGL -&amp;gt; GPU -&amp;gt; Screen
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The claimed result is 60 fps on that 2019 set. I couldn't measure that, and the project itself says the simulator can't either. More on that below.&lt;/p&gt;

&lt;h2&gt;
  
  
  The architecture
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Mostly Rust, with a thin C shell
&lt;/h3&gt;

&lt;p&gt;The repository is overwhelmingly Rust. A rough line count on the current &lt;code&gt;main&lt;/code&gt; gives about 576,000 lines of &lt;code&gt;.rs&lt;/code&gt; files against about 7,300 lines of C and C++. That count includes tests and generated data, so treat it as a ratio, not a measure of features.&lt;/p&gt;

&lt;p&gt;The structure is spelled out in &lt;code&gt;rust-modules/Cargo.toml&lt;/code&gt;. It's a Cargo workspace whose root crate is linked into a small C app as a static library:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="c"&gt;# Static library linked into the C app (hybrid migration): each ported module&lt;/span&gt;
&lt;span class="c"&gt;# exposes the same C ABI its src/*.h declares, so the C code calls it unchanged.&lt;/span&gt;
&lt;span class="nn"&gt;[lib]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"plxnative_modules"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The workspace is split into layer crates: &lt;code&gt;base&lt;/code&gt;, &lt;code&gt;machine&lt;/code&gt;, &lt;code&gt;net&lt;/code&gt;, &lt;code&gt;platform&lt;/code&gt;, &lt;code&gt;gfx&lt;/code&gt;, &lt;code&gt;ui&lt;/code&gt;, &lt;code&gt;plex&lt;/code&gt;, &lt;code&gt;telemetry&lt;/code&gt;, &lt;code&gt;data&lt;/code&gt;, &lt;code&gt;session&lt;/code&gt;, &lt;code&gt;media&lt;/code&gt;, &lt;code&gt;appkit&lt;/code&gt;, &lt;code&gt;screens&lt;/code&gt; and &lt;code&gt;storage&lt;/code&gt;. &lt;code&gt;docs/module-layers.md&lt;/code&gt; describes how they depend on one another.&lt;/p&gt;

&lt;h3&gt;
  
  
  A GPU-drawn interface
&lt;/h3&gt;

&lt;p&gt;The UI goes through SDL2 and OpenGL ES (both are listed in the repo's topics), so there's no DOM and no layout engine. The design language is translucent "glass" panels over your own posters and backdrops, with text sized for reading across a room. The docs folder holds design notes on liquid glass, backdrop blur and a hardware budget for the glass effect, which gives a sense of how much of the work goes into making that look cheap enough for TV silicon.&lt;/p&gt;

&lt;h3&gt;
  
  
  Video on the TV's own decoder
&lt;/h3&gt;

&lt;p&gt;Playback doesn't decode video in software. The README says the app "hands video to the same silicon the built-in apps use". In the code, that goes through a video-sink trait in the &lt;code&gt;platform&lt;/code&gt; crate. On the TV it's implemented by a &lt;code&gt;StarfishSink&lt;/code&gt;, which talks to LG's Starfish media APIs, and on the desktop by a &lt;code&gt;HostSink&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;What it can direct-play, per the README:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;H.264 and HEVC, including 4K and 10-bit&lt;/li&gt;
&lt;li&gt;Dolby Vision profiles 5 and 8&lt;/li&gt;
&lt;li&gt;E-AC-3 with Atmos&lt;/li&gt;
&lt;li&gt;AV1 or VP9 (10-bit VP9 only on webOS 4 sets), on TVs whose codec table lists that decoder&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Anything else is transcoded by your Plex server. Before you press Play, the detail page shows whether a title will play directly or be converted on the server. The app also ships its own LGPL build of FFmpeg.&lt;/p&gt;

&lt;h2&gt;
  
  
  Features
&lt;/h2&gt;

&lt;p&gt;From the README, the site and the 0.8.0 release notes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Sign in on the TV&lt;/strong&gt; by scanning a QR code with your phone, then pick a Plex Home profile. PIN-protected profiles are supported.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Browse your own and shared libraries&lt;/strong&gt;, with Continue Watching, Recently Added, sort, filters (including unwatched-only) and an A–Z rail.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Search across every server you can reach&lt;/strong&gt;, with results grouped by kind.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A full player&lt;/strong&gt; with resume, seek and scrub, chapters, audio and subtitle tracks (image subtitles included), Skip Intro, Skip Credits, Up Next with auto-advance, and progress synced back to Plex.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;New in 0.8.0:&lt;/strong&gt; subtitle search from inside the player, ASS/SSA subtitles that keep their styling, browsing Plex collections, Boost dialog and Normalize loudness on servers with Plex Pass, a Settings section for playback, and Spanish and Belarusian translations.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Works without internet&lt;/strong&gt; after the first sign-in. The app talks to your server directly, so the library keeps working on your home network when the internet is down.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Privacy defaults.&lt;/strong&gt; Crash reports and usage analytics are two separate opt-in questions, both off until you say yes. The README says no title, search term, subtitle line, server name or address can appear in any report. The app talks to your Plex server, to &lt;code&gt;plex.tv&lt;/code&gt; to sign in, and to &lt;code&gt;discover.provider.plex.tv&lt;/code&gt; for cast biographies.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;0.8.0 also changes one security behaviour. Some home servers only answer over plain HTTP. Earlier versions refused to send your Plex sign-in that way. 0.8.0 will connect, but only after asking "Connect without encryption?", and only for a server on the same network at a private address after every secure route, Plex's relay included, has failed.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to install it
&lt;/h2&gt;

&lt;p&gt;PlxNative isn't in the LG Content Store yet; the site says it's "in the process of review" with no confirmed date. There are two routes, and neither needs root.&lt;/p&gt;

&lt;h3&gt;
  
  
  Route 1: LG Developer Mode
&lt;/h3&gt;

&lt;p&gt;The &lt;a href="https://github.com/GLinnik21/plx-native/blob/main/docs/install-and-verify.md" rel="noopener noreferrer"&gt;installation guide&lt;/a&gt; needs an LG TV on webOS 4.0 or newer, a Plex account with a server, and a Mac, Windows or Linux computer on the same network. In short:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Create or sign in to an account on LG's developer site.&lt;/li&gt;
&lt;li&gt;Install the &lt;strong&gt;Developer Mode&lt;/strong&gt; app on the TV, sign in, turn on &lt;strong&gt;Dev Mode Status&lt;/strong&gt; (the TV restarts) and then &lt;strong&gt;Key Server&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Note the TV's IP address and passphrase, and connect to it from &lt;strong&gt;webOS Dev Manager&lt;/strong&gt; on your computer.&lt;/li&gt;
&lt;li&gt;Install the &lt;code&gt;.ipk&lt;/code&gt; directly, or install Homebrew Channel first and get PlxNative from its catalogue.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For a manual download, verify the package first:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sha256sum&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; ipk.sha256
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The catch is renewal. Developer Mode sessions expire, and if one lapses LG removes the apps installed through it. The guide's advice is to open the Developer Mode app and press &lt;strong&gt;EXTEND&lt;/strong&gt; before the remaining time runs out, with a calendar reminder. An expired session can't be extended.&lt;/p&gt;

&lt;h3&gt;
  
  
  Route 2: Homebrew Channel
&lt;/h3&gt;

&lt;p&gt;If your TV already runs Homebrew Channel, open it, find &lt;a href="https://repo.webosbrew.org/apps/com.beb.plxnative/" rel="noopener noreferrer"&gt;PlxNative&lt;/a&gt; (a 19.6 MB download, listed as version 0.8.0) and select &lt;strong&gt;Install&lt;/strong&gt;. Updates arrive there too, and the Channel checks the download's checksum before installing.&lt;/p&gt;

&lt;h3&gt;
  
  
  Nightlies
&lt;/h3&gt;

&lt;p&gt;Adding &lt;code&gt;https://plxnative.com/nightly/repo.json&lt;/code&gt; to Homebrew Channel installs &lt;strong&gt;PlxNative Nightly&lt;/strong&gt; as a separate app with its own tile and sign-in. The README is blunt about these builds: "It is not tested on a television."&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits and current status
&lt;/h2&gt;

&lt;p&gt;The README has a section titled "The honest scope", and it's worth taking at its word:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Movies and TV shows only.&lt;/strong&gt; No music, no photos, no Live TV or DVR.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No typing in server addresses.&lt;/strong&gt; Servers come from your Plex account.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Relay and secure-only servers&lt;/strong&gt; are supported but "haven't been watched end to end."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Not all TVs work.&lt;/strong&gt; It needs webOS 4.0 or newer. A 2016 set on webOS 3.4.3 has been tested and works.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The known issues list is specific:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Some 2019 sets on LG's k5lp or k3lp chip&lt;/strong&gt;, installed through Developer Mode, won't play video. The sandbox withholds a device the video path needs; Kodi and Moonlight hit the same wall. Without root there's no fix.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;2018 sets on webOS 3.9.3&lt;/strong&gt;: sign-in and browsing work, but the prebuilt v0.8.0 never starts video (&lt;a href="https://github.com/GLinnik21/plx-native/issues/249" rel="noopener noreferrer"&gt;#249&lt;/a&gt;). A source build at or after commit &lt;code&gt;60f99c7&lt;/code&gt; fixes it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;webOS 3.x&lt;/strong&gt; restarts the stream on every seek instead of jumping in place.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sets on the k6hp chip&lt;/strong&gt; show a crash in opt-in reports that the developer can't reproduce without the hardware (&lt;a href="https://github.com/GLinnik21/plx-native/issues/174" rel="noopener noreferrer"&gt;#174&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Building the simulator on Linux
&lt;/h2&gt;

&lt;p&gt;Since I couldn't put it on a TV, I built the desktop simulator the docs describe. The box runs Debian 13 on x86_64 with 8 vCPUs and no GPU. The only prerequisites were the SDL2, SDL2_ttf, Mesa, CMake and pkg-config dev packages plus a current stable Rust toolchain:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/GLinnik21/plx-native &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd &lt;/span&gt;plx-native
make sim-linux &lt;span class="nv"&gt;SIM_TDIR&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;rust-modules/target-sim
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;   Compiling plxnative-modules v0.8.0 (/workspace/plxnative/repo/rust-modules)
    Finished `release` profile [optimized] target(s) in 3m 08s
real    3m57.100s
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That gave a 16 MB &lt;code&gt;plxnative-sim&lt;/code&gt; binary on the first try. The extra 49 seconds went into fetching and building the libass stack (FreeType, FriBidi, HarfBuzz and friends).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft214u6y9hdf8y16dffgr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft214u6y9hdf8y16dffgr.png" alt="Building the PlxNative Linux simulator and capturing screens against the mock server" width="800" height="380"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The project's screenshot tool then boots the simulator against &lt;code&gt;tests/mock_pms.py&lt;/code&gt;. That's a synthetic Plex server that also stands in for plex.tv, so no account, token or real library is involved. I ran it headless under Xvfb with software OpenGL:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python3 tools/demo_library.py derive
&lt;span class="nv"&gt;LIBGL_ALWAYS_SOFTWARE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 xvfb-run &lt;span class="nt"&gt;-a&lt;/span&gt; &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="s1"&gt;'-screen 0 1920x1080x24'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  python3 tools/screenshots.py &lt;span class="nt"&gt;--bin&lt;/span&gt; rust-modules/target-sim/release/plxnative-sim &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--out&lt;/span&gt; shots &lt;span class="nt"&gt;--only&lt;/span&gt; home-shelves,library-grid,search,detail,signin
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each scene settled in 2.2 to 4.9 seconds. One hiccup: the &lt;code&gt;home&lt;/code&gt; scene also renders the website's link-preview card, and that step failed because the box lacked the &lt;code&gt;file&lt;/code&gt; utility, so I captured &lt;code&gt;home-shelves&lt;/code&gt; instead.&lt;/p&gt;

&lt;p&gt;The sign-in screen is the QR flow, here pointed at the mock:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fi0fkjcucfn6uk843c41v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fi0fkjcucfn6uk843c41v.png" alt="The PlxNative sign-in screen in the Linux simulator, served by the mock Plex server" width="799" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The Movies grid, with the sort and filter chips and the A–Z rail, browsing the project's demo library of openly licensed films:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fil4806zl2gzlkn8msn11.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fil4806zl2gzlkn8msn11.png" alt="The Movies library grid in the PlxNative Linux simulator" width="799" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;And a title page, with the "Direct Play" label next to the runtime:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkyvtqoxwn0lj4yoqm8w1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkyvtqoxwn0lj4yoqm8w1.png" alt="A title detail page in the PlxNative Linux simulator showing Direct Play" width="799" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Be clear about what this proves. The simulator runs the same UI and Plex code as the TV build, and it boots, signs in to a fake server and draws every screen I asked for. But &lt;code&gt;docs/building.md&lt;/code&gt; says plainly that it "cannot answer frame rate (different GPU), text rasterization, or anything about LG's decoder and video plane. Those need the set." The Linux build also has no video path at all; the Makefile notes that Play "intentionally reaches the host seam's existing 'no video path' result." So the speed and the hardware decoding, the two things PlxNative exists for, are still unverified by me.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who it's for
&lt;/h2&gt;

&lt;p&gt;If you run Plex at home and own a fairly recent LG TV, PlxNative is a well-documented alternative to the official app. Expect a project that tells you about its rough edges in the README. You'll need to be comfortable with Developer Mode renewals or Homebrew Channel until the LG Content Store listing lands. If you need music, Live TV or a set from the known-issues list, the README itself says the official app will serve you better.&lt;/p&gt;

&lt;p&gt;If you're a developer, the repository is interesting on its own terms: a Rust UI stack drawing straight to GPU on TV-grade hardware, with design and performance notes for almost every decision. The simulator builds in a few minutes on plain Linux if you want to look around.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Site: &lt;a href="https://plxnative.com/" rel="noopener noreferrer"&gt;https://plxnative.com/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Source: &lt;a href="https://github.com/GLinnik21/plx-native" rel="noopener noreferrer"&gt;https://github.com/GLinnik21/plx-native&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>rust</category>
      <category>opensource</category>
      <category>linux</category>
      <category>showdev</category>
    </item>
  </channel>
</rss>
