<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ishank choudhary on Medium]]></title>
        <description><![CDATA[Stories by Ishank choudhary on Medium]]></description>
        <link>https://medium.com/@ishank.iandroid?source=rss-1e910a184252------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9Ua4aEzj4Un2APnoNkIi0A.png</url>
            <title>Stories by Ishank choudhary on Medium</title>
            <link>https://medium.com/@ishank.iandroid?source=rss-1e910a184252------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:28:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ishank.iandroid/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[T3 Code Hands-On: A Coding Agent Restyles Its Own IDE Through MCP Theme Tools]]></title>
            <link>https://blog.stackademic.com/t3-code-hands-on-a-coding-agent-restyles-its-own-ide-through-mcp-theme-tools-a9fae1b261fc?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9fae1b261fc</guid>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[mcps]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 04:31:03 GMT</pubDate>
            <atom:updated>2026-11-18T04:31:03.450Z</atom:updated>
            <content:encoded><![CDATA[<h4>I gave a free model seven restyle prompts. Every colour change reached the open app within 0.2 s of the tool call, with no reload. Then one sidebar tweak broke the layout, and the agent’s “fix” didn’t fix it.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WI5boXbRF-6jxLTR6N6eHQ.jpeg" /></figure><p>On 14 November Theo posted that “T3 Code now lets you fully customize your UI in app. Ask your agent to make changes and it will update on the fly.” The <a href="https://x.com/theo/status/2121455838414201200">post</a> had 5,213 likes when I checked. The feature had merged a few hours earlier in <a href="https://github.com/pingdotgg/t3code/pull/35247">PR #35247</a>, “agents can restyle T3 Code with theme colors and CSS”.</p><p>I’ve looked at T3 Code before as “one more UI for coding agents” and moved on. This feature is different. The agent isn’t editing your project. It’s editing the tool you’re watching it work in. I wanted to know whether that holds up, so I ran it myself: how fast a change shows up, what survives a restart, and what happens when the agent gets the CSS wrong. It can’t see the screen, after all.</p><p>So I built a small lab on a headless Linux box, gave a free model seven restyle prompts, and measured every one from T3 Code’s own database and a headless browser.</p><h3>What T3 Code is, and what’s new</h3><p><a href="https://github.com/pingdotgg/t3code">T3 Code</a> is an MIT-licensed TypeScript app that its README calls “the open-source control plane for coding agents.” It drives agents you already have installed, such as Claude Code, Codex, Cursor and OpenCode, and shows their threads in one UI on desktop, web and phone. The repo had 63,163 stars on 17 November.</p><p>The new part is a pair of MCP tools that T3 Code’s server hands to the agents it runs (OpenCode saw them as t3-code_t3_theme_read and t3-code_t3_theme_set):</p><ul><li><strong>t3_theme_read</strong> is read-only. It returns the current theme, the five built-in themes (t3-chat, grove, ocean, ember, iris), any published themes, the themes folder, and a 3,720-character styling guide.</li><li><strong>t3_theme_set</strong> is marked destructive. It publishes a theme file and switches this server&#39;s web app, and the desktop app on the same machine, over to it. It can also flip light, dark or system mode.</li></ul><p>A theme is a small JSON file. You give a canvas and an accent hex colour, and T3 Code derives the rest of the palette. You can also override any of 61 named colour roles, from sidebarRowActive to terminalCursor, and add up to 50,000 characters of custom CSS. The guide lists 29 stable selectors, such as [data-chat-header] and [data-slot=&quot;composer-host&quot;], because class names change between releases. The guide also tells the agent the important limit up front: &quot;You cannot see the result. Make small changes, then ask the user how it looks.&quot;</p><p>Per the docs, the thread has to run in <strong>Full access</strong> mode. Mobile apps and desktop apps connected from other machines don’t follow these themes.</p><h3>My setup</h3><p>Everything ran on a Debian 13 box with 8 vCPUs, no GPU and no display. There’s no desktop app in this test, only T3 Code’s web mode, which is the same UI served by t3 serve. I don&#39;t have Claude or Codex accounts there, so the agent was OpenCode 1.19.0 with the free deepseek-v4.1-flash-free model. No paid key was involved anywhere.</p><pre>npm i t3@0.1.10                  # tiny launcher; first run downloads a 171 MB binary<br>npx t3 --version                 # t3 v0.1.10, 8.1 s the first time<br><br>cd ~/code/t3demo                 # an empty git repo to act as the project<br>t3 serve --base-dir ~/t3lab --port 3773 --host 127.0.0.1<br># first boot runs 96 DB migrations: listening in ~2.5 s; restarts: 1.76 s<br>t3 project add --base-dir ~/t3lab ~/code/t3demo</pre><p>OpenCode is off by default. I turned it on in the server’s settings.json (userdata/settings.json under the data directory), which T3 Code picked up without a restart:</p><pre>{ &quot;providerInstances&quot;: { &quot;opencode&quot;: { &quot;driver&quot;: &quot;opencode&quot;, &quot;enabled&quot;: true } } }</pre><p>The best surprise of the setup is that T3 Code has a real CLI for threads, so I didn’t have to click through the UI to run prompts. Each prompt was one command:</p><pre>t3 thread start --base-dir ~/t3lab . \<br>  --provider opencode --model opencode/deepseek-v4.1-flash-free \<br>  --wait --json &quot;Restyle T3 Code itself using your T3 Code theme tools ...&quot;</pre><p>To measure what actually reached the screen, I paired a headless Chrome page with the server and left it open on the app. A small Playwright script polled the page’s computed styles about every 100 ms and logged each change with a timestamp. Tool start and finish times came from T3 Code’s own SQLite projection of the thread. So the latency number below is: first visible style change, minus the moment the first successful t3_theme_set call completed.</p><h3>Run 1: dark, high-contrast</h3><p>The first prompt asked for a near-black canvas, pure white text and a bright yellow accent, published as hc-dark. The agent read the guide once, then published the whole theme in one t3_theme_set call: canvas #0a0a0a, accent #ffd400, 60 colour overrides, and some CSS for yellow focus rings and a yellow composer border. (It made a second call 5 s later that just switched to the same theme again.) This is a trimmed copy of the first call&#39;s input:</p><pre>{<br>  &quot;themeId&quot;: &quot;hc-dark&quot;,<br>  &quot;appearance&quot;: &quot;dark&quot;,<br>  &quot;theme&quot;: {<br>    &quot;name&quot;: &quot;High Contrast Dark&quot;,<br>    &quot;appearance&quot;: &quot;dark&quot;,<br>    &quot;canvas&quot;: &quot;#0a0a0a&quot;,<br>    &quot;accent&quot;: &quot;#ffd400&quot;,<br>    &quot;colors&quot;: { &quot;text&quot;: &quot;#ffffff&quot;, &quot;focus&quot;: &quot;#ffd400&quot;, &quot;sidebarRowSelected&quot;: &quot;#262626&quot; },<br>    &quot;css&quot;: &quot;.dark [data-slot=\&quot;composer-host\&quot;]:focus-within { border-color: #ffd400 !important; }&quot;<br>  }<br>}</pre><p>The tool call itself took 37 ms on the server. My open browser page switched from the stock light theme to dark 171 ms after the call completed, with no reload. The whole prompt took 44.2 s, and 34 s of that was the model thinking before its first write. It then ended with “I can’t see the result — how does it look?”, which is exactly what the guide tells it to do.</p><p>I ran the same prompt twice more with new theme IDs, to check whether it was repeatable rather than lucky:</p><ul><li><strong>Run 2</strong> took 52.2 s. The first t3_theme_set failed validation with &quot;Give themeId, appearance, or both. theme needs a themeId.&quot; The model had left out the ID. It read the error, retried with the ID, and the change appeared 171 ms after the good call.</li><li><strong>Run 3</strong> took 19.3 s. It went straight through, and the screen changed 34 ms after the call.</li></ul><p>So 3 of 3 runs produced a working high-contrast theme, and one needed a retry it handled by itself.</p><h3>Run 2: my brand colours</h3><p>Next I asked for a deep navy canvas (#0e1a2b), an orange accent (#ff6b2c) and orange highlights on the active thread in the sidebar, published as ishank-brand.</p><p>This was the slowest run, at 101.0 s, because the model spent 27 s in one reasoning block planning the palette. The write itself took 47 ms, and the browser changed 172 ms later. The navy and orange looked right, and the active thread row turned a warm orange-brown.</p><p>Then I checked how the active row got that colour. The agent had done two things. It set the sidebarRowActive colour role, and it wrote CSS that guessed at the row&#39;s state attributes: [data-thread-item][aria-current=&quot;true&quot;], [data-active=&quot;true&quot;], [data-state=&quot;active&quot;] and [data-selected=&quot;true&quot;]. I queried the live page, and none of those four selectors matched a single element. The rows don&#39;t carry any of those attributes. The orange inset bar in that CSS never rendered. The colour role did all the visible work, and the CSS was dead code.</p><p>That’s the pattern for the rest of this test. When the agent sticks to the colour roles, it works. When it guesses at the DOM, it’s guessing blind.</p><h3>Run 3: a layout tweak</h3><p>Third, I kept the brand colours and asked for three layout changes: a 72 px chat header, 24 px rounded corners on the composer, and a sidebar of about 240 px. The agent read the guide, used a shell command to read its own theme file so it could keep the colours, and updated the theme in place. It took 20.3 s.</p><p>I measured the DOM before and after:</p><ul><li><strong>Chat header:</strong> 52 px before, 72 px after. This one worked.</li><li><strong>Composer corners:</strong> the composer-host element now computed a 24 px radius, but the card you actually see is drawn by an inner element that stayed at 16 px. There was no visible change.</li><li><strong>Sidebar:</strong> the outer [data-slot=&quot;sidebar&quot;] dropped from 256 px to 240 px. The sidebar-container and sidebar-gap inside it stayed at 256 px, and the chat column moved to start at x=240. The sidebar now overlapped the chat by 16 px. The layout was broken.</li></ul><p>So one of the three layout changes worked as asked, one did nothing you could see, and one broke the page.</p><h3>Asking it to fix its own bug</h3><p>This is the part I most wanted to test, because it’s what you’d actually do. I told the agent exactly what was wrong: the outer sidebar is 240 px, the inner container is still 256 px, and they overlap by 16 px. Then I asked it to fix that.</p><p>It answered in 16.2 s: “Fixed: [data-slot=&quot;sidebar-inner&quot;] now shares the same 240px sizing as the outer sidebar, with border-box and overflow: hidden so nothing spills.&quot;</p><p>It wasn’t fixed. The container was still 256 px. This is the CSS it wrote:</p><pre>[data-slot=&quot;sidebar&quot;],<br>[data-slot=&quot;sidebar-inner&quot;] {<br>  width: 240px !important;<br>  min-width: 240px !important;<br>  max-width: 240px !important;<br>  flex-basis: 240px !important;<br>}<br>/* matches nothing: one element can&#39;t have two different data-slot values */<br>[data-slot=&quot;sidebar&quot;][data-slot=&quot;sidebar-inner&quot;] {<br>  box-sizing: border-box !important;<br>  overflow: hidden !important;<br>}</pre><p>The guide names sidebar-inner as &quot;sidebar surface&quot;, so the model used it. But the element that sets the width is sidebar-container, which the guide doesn&#39;t list. The second rule chains two different data-slot values onto one element, so it can never match. The model reported success because the tool call succeeded, but it had no way to check what the page actually did.</p><p>To be sure the platform wasn’t the problem, I fixed it myself. I added one rule to the theme file and renamed it into place, as the docs recommend:</p><pre>[data-slot=&quot;sidebar-gap&quot;],<br>[data-slot=&quot;sidebar-container&quot;] {<br>  width: 240px !important;<br>}</pre><p>No command was needed. The open page picked up the file change within 3 s, and the sidebar, gap and container all measured 240 px with the chat column starting at x=240. The pipeline was fine. The agent was missing feedback.</p><h3>Does it stick?</h3><p>Yes. I stopped the server and started it again (1.76 s to ready). t3 theme show still reported ishank-brand as the environment theme. The already-open page reloaded into it with the 72 px header and 240 px sidebar intact. Then I opened a brand-new browser profile, paired it with the restarted server, and it loaded straight into the brand theme. Themes are plain files in userdata/themes/&lt;id&gt;.json, and the selection is saved in settings.json. That makes them easy to diff, back up or commit.</p><h3>Undo works, mostly</h3><p>Last, I asked the agent to switch back to the stock theme. It took 12.4 s with one write, and the page changed 122 ms later to the built-in t3-chat. But it stayed in dark mode. My session had started in light mode, and nothing put that back. The four themes the agent had published were also still in the themes folder. There&#39;s no delete tool, only read and set, so cleanup is up to you.</p><h3>The numbers</h3><ul><li><strong>High contrast, 3 runs:</strong> 44.2, 52.2 and 19.3 s. 2, 2 (1 failed) and 1 writes. On screen 171, 171 and 34 ms after the write. 3 of 3 applied.</li><li><strong>Brand colours:</strong> 101.0 s, 1 write, on screen after 172 ms. The colours applied; the CSS matched nothing.</li><li><strong>Layout tweak:</strong> 20.3 s, 1 write. 1 of 3 changes visible, and the sidebar broke.</li><li><strong>Fix the sidebar:</strong> 16.2 s, 1 write. It said “Fixed”. It wasn’t.</li><li><strong>Back to stock:</strong> 12.4 s, 1 write, on screen after 122 ms. It switched but left dark mode on.</li></ul><p>Across the five colour changes, a median of 171 ms passed between the tool finishing and the screen changing, and the slowest took 172 ms. The t3_theme_set call itself never took more than 55 ms. Almost all of each prompt&#39;s time was the model thinking, which is about the model, not T3 Code.</p><h3>What I’d tell my team</h3><p><strong>The plumbing is solid.</strong> Every write went live on an already-open page without a reload. The theme survived a restart and reached a brand-new client. Validation caught a bad call and gave an error message the model could act on. Themes are plain JSON files you can review in a pull request. For a feature that’s less than a week old, that’s a good base.</p><p><strong>Use colour roles, not CSS.</strong> The guide says “Prefer colors over CSS”, and my results back that up. All 5 colour changes landed. Of the 4 CSS changes I checked, only the taller header worked. The active-row highlight and the composer corners did nothing visible, the sidebar broke, and the follow-up fix didn’t fix it.</p><p><strong>The agent can’t check its work.</strong> That’s the real gap, and the guide is honest about it. A model that can’t see the result will report success when the tool call succeeds. It did that once here, after I’d told it exactly what was broken. A tool that returns the computed size of a selector, or a screenshot, would close most of that gap. Until then, a person has to look.</p><p><strong>What I couldn’t test:</strong> the desktop app, macOS and Windows (my box is headless Linux), Claude Code and Codex (no accounts), any model stronger than a free one, mobile and remote clients, background images, and whether the tools refuse to run outside Full access mode. A stronger model may guess the DOM better. It still can’t see it.</p><p>If you want to try it yourself, these are the same commands I ran, without my lab’s --base-dir (enable OpenCode in settings.json first, as above):</p><pre>npx t3@0.1.10 serve --port 3773          # open the pairing URL it prints<br>npx t3@0.1.10 project add .              # in a second terminal, inside your repo<br>npx t3@0.1.10 thread start . --wait \<br>  --provider opencode --model opencode/deepseek-v4.1-flash-free \<br>  &quot;Use your T3 Code theme tools to restyle T3 Code. Change colours only, no CSS.&quot;<br>npx t3@0.1.10 theme set t3-chat          # undo: back to a built-in theme</pre><p>All numbers come from my own runs on 17 Nov 2026 (IST) with T3 Code v0.1.10 and OpenCode 1.19.0. T3 Code ships fast, so some of these gaps may be closed by the time you read this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MGvHUb3NLroMC08LucK3TQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9fae1b261fc" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/t3-code-hands-on-a-coding-agent-restyles-its-own-ide-through-mcp-theme-tools-a9fae1b261fc">T3 Code Hands-On: A Coding Agent Restyles Its Own IDE Through MCP Theme Tools</a> was originally published in <a href="https://blog.stackademic.com">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Emulator Hands-On: Test S3, Lambda and DynamoDB Locally With a Coding Agent]]></title>
            <link>https://blog.stackademic.com/aws-emulator-hands-on-test-s3-lambda-and-dynamodb-locally-with-a-coding-agent-5981ba5495c6?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/5981ba5495c6</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 05:01:19 GMT</pubDate>
            <atom:updated>2026-11-16T05:01:19.788Z</atom:updated>
            <content:encoded><![CDATA[<h4>I had a coding agent build an S3 → Lambda → DynamoDB service against it in Docker, with no AWS account. 3 of 3 runs ended with passing tests, in a median 3 min 7 s. Then I hit the 1,000-object bucket cap.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ebggaFWKXfe3_cZqeKT7mw.jpeg" /></figure><p>On 11 November AWS shipped AWS Emulator, a local copy of the AWS APIs that you run on your own machine. The pitch is aimed squarely at coding agents: let the agent build and test against something that behaves like AWS, with no account, no credentials and no bill. Swami Sivasubramanian’s <a href="https://x.com/SwamiSivasubram/status/2120579757544755272">launch post</a> passed 1,700 likes on X, and the <a href="https://news.ycombinator.com/item?id=50456631">Hacker News thread</a> went past 960 points in under two days.</p><p>LocalStack and moto already cover local AWS testing, so I wanted to know three things. Does it start fast enough to be part of a test loop? Can a coding agent actually ship something against it without me helping? And where does it stop looking like AWS? So I ran it in Docker on a Linux box, had an agent build a small S3 → Lambda → DynamoDB service three times from scratch, and then went looking for the edges myself.</p><h3>What AWS Emulator is (from the repo and docs)</h3><p>AWS Emulator (<a href="https://github.com/aws/aws-emulator">github.com/aws/aws-emulator</a>, <a href="https://docs.aws.amazon.com/emulator/latest/userguide/what-is-service.html">docs</a>) is a single Rust binary, aws-emulator, that serves AWS service APIs over HTTPS on 127.0.0.1:9297. A few facts worth knowing up front:</p><ul><li><strong>It’s free and open.</strong> The code is CC0 1.0 (public domain), except the vendor/ framework crates, which are Apache 2.0. The repository doesn&#39;t take pull requests; you file issues instead.</li><li><strong>It runs three ways.</strong> On Linux you install an RPM or Debian package. On macOS and Windows you build a Docker image from a short Dockerfile that installs that same package (AWS doesn’t publish a prebuilt image). For Rust tests there’s an embedded mode that runs in-process.</li><li><strong>Coverage comes from AWS’s own API models.</strong> The repo has 437 service crates, one per API model. Each release publishes fidelity scores. The v1.0.338 scores I downloaded cover 355 services with 43,548 scored tests, and 26,001 of them (59.7%) pass.</li><li><strong>Agents built it.</strong> AWS’s <a href="https://www.aboutamazon.com/news/aws/how-autonomous-ai-agents-built-aws-emulator">own write-up</a> says autonomous agents wrote every line, with agent teams writing fidelity tests against real AWS, filing issues and fixing them. You can see this in the repo: issues/things-that-do-not-work/ held 6,936 files on the day I tested, and issues/things-that-now-work/ held 13,224.</li><li><strong>There’s an agent skill.</strong> aws agent-toolkit add-skill --skill-name aws-emulator installs a skill that teaches an agent how to install, start, connect and troubleshoot the emulator.</li><li><strong>Telemetry is on by default.</strong> The docs list exactly what’s collected (operation counts, error categories, SDK names and versions; no payloads, resource names or credentials). You turn it off with --telemetry off or EMULATOR_TELEMETRY=off. I ran with it off the whole time.</li></ul><h3>My setup</h3><p>I tested version 1.0.338 (released 12 Nov) on a Debian 13 box with 8 vCPUs and 15 GiB of RAM. The coding agent was OpenCode 1.19 with OpenCode Zen’s free deepseek-v4.1-flash-free model. I didn&#39;t use Cursor or Claude Code, and I didn&#39;t use real AWS credentials at any point.</p><p>Docker needed some work first. The Debian mirror returned HTTP 500, so I installed Docker 27.3.1 from the static binaries. The box also has no iptables, so the Docker daemon ran without its bridge network and every container used host networking. That meant I couldn’t test the documented -p 127.0.0.1:9297:9297 port publishing. I bound the emulator to loopback inside the container instead, which has the same effect on a single host.</p><p>I built the image from the Dockerfile in the docs, then started it like this:</p><pre># Dockerfile from the docs: amazonlinux:2023-minimal + the aws-emulator RPM<br>docker build -t aws-emulator .              # 30.0 s, image is 1.27 GB<br><br>mkdir -p ~/.aws/emulator/9297<br>docker run -d --rm --name aws-emulator --network host \<br>  --user &quot;$(id -u):$(id -g)&quot; --memory=2g --memory-swap=2g \<br>  --cap-drop ALL --security-opt no-new-privileges --read-only \<br>  -e HOME=&quot;$HOME&quot; -e EMULATOR_TELEMETRY=off \<br>  -v &quot;$HOME/.aws/emulator/9297:$HOME/.aws/emulator/9297&quot; \<br>  --entrypoint aws-emulator aws-emulator -b 127.0.0.1 -p 9297</pre><p>When it starts, the emulator writes a fresh access key, an SDK config file and a self-signed CA into ~/.aws/emulator/9297/. Requests must be SigV4-signed with that run&#39;s key. Pointing the AWS CLI or boto3 at it takes only environment variables, with no code changes:</p><pre>unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE<br>export AWS_ENDPOINT_URL=https://127.0.0.1:9297<br>export AWS_SHARED_CREDENTIALS_FILE=~/.aws/emulator/9297/emulator_credentials<br>export AWS_CONFIG_FILE=~/.aws/emulator/9297/config<br>export AWS_CA_BUNDLE=~/.aws/emulator/9297/ca.pem<br>export AWS_EC2_METADATA_DISABLED=true<br>export AWS_DISABLE_HOST_PREFIX_INJECTION=true<br>export AWS_REGION=us-east-1<br><br>aws sts get-caller-identity   # Account &quot;000000000000&quot;, Arn &quot;.../user/emulator&quot;</pre><p>Keep 127.0.0.1 in the endpoint rather than localhost. With an IP address, the SDKs put the S3 bucket name in the path, which is the only style the emulator accepts.</p><h3>Start-up numbers</h3><p>These are wall-clock times on the box. The full logs are in my lab notes.</p><ul><li><strong>Image build:</strong> 30.0 s with the base image already pulled, and 28.7 s with --no-cache, which re-downloads the 171.6 MB RPM.</li><li><strong>Cold start</strong> (from docker run to the first HTTP 200 on /__admin/health): 272 ms to 1,007 ms across 10 starts, with a median of 375 ms. Three of the ten went over 500 ms, and the slowest was the very first one.</li><li><strong>Idle memory:</strong> 175.6 MiB.</li><li><strong>Installing the skill:</strong> 0.69 s. It installed for every agent it found: ~/.agents/skills (OpenCode, Codex, Gemini CLI and others), ~/.claude/skills and ~/.cursor/skills.</li></ul><p>A sub-second start changes how you use it. You can start a clean emulator for every test run instead of sharing a long-lived one, and since state lives only in memory, a restart is also your reset.</p><h3>The agent test: upload-and-index, three times from scratch</h3><p>I gave OpenCode one prompt and nothing else. Build a python3.12 Lambda that reads an object from S3, counts its lines and words, and writes an index item to a DynamoDB table. Give it a least-privilege IAM role and an idempotent deploy.py. Write a pytest suite that uploads three files, invokes the function for each, and checks the DynamoDB items. Use the aws-emulator skill. The only hints were about my box: the image was already built, and Docker needed --network host. Each run started with no emulator running and an empty folder.</p><p>All three runs ended with passing tests:</p><ul><li><strong>Run 1:</strong> 168.9 s, 27 tool calls, 3 tests passed</li><li><strong>Run 2:</strong> 280.3 s, 39 tool calls, 2 tests passed (one test covers all three files, including an empty one and one with Unicode)</li><li><strong>Run 3:</strong> 187.2 s, 20 tool calls, 2 tests passed (plus an idempotency test that deploys twice)</li></ul><p>The median was 3 min 7 s. Afterwards I re-ran each suite myself against a fresh emulator, and all three passed again in 5–11 seconds.</p><p>The skill made a visible difference to how the agent worked. In every run, it loaded the skill first and read the container and connect references. It started the emulator with telemetry off, and checked that sts get-caller-identity returned account 000000000000 before writing any code. All three runs also added a guard to deploy.py that refuses to run unless the endpoint is loopback and the account is the emulator&#39;s. I didn&#39;t ask for that. It comes from the skill&#39;s &quot;keep real credentials out&quot; rules, and it&#39;s exactly the guard I&#39;d want in a repo where an agent has a shell.</p><p>This is the handler from run 1:</p><pre>import os<br>import boto3<br><br>TABLE_NAME = os.environ.get(&quot;DOC_INDEX_TABLE&quot;, &quot;doc-index&quot;)<br><br>def handler(event, context):<br>    bucket, key = event[&quot;bucket&quot;], event[&quot;key&quot;]<br>    s3 = boto3.client(&quot;s3&quot;)<br>    body = s3.get_object(Bucket=bucket, Key=key)[&quot;Body&quot;].read()<br>    text = body.decode(&quot;utf-8&quot;)<br>    item = {&quot;pk&quot;: key, &quot;bytes&quot;: len(body),<br>            &quot;lines&quot;: len(text.splitlines()), &quot;words&quot;: len(text.split())}<br>    boto3.resource(&quot;dynamodb&quot;).Table(TABLE_NAME).put_item(Item=item)<br>    return item</pre><p>The most useful part was where the agents got stuck, because each time the emulator was behaving like AWS:</p><ul><li>CreateFunction rejected AWS_REGION and AWS_DEFAULT_REGION as reserved environment variables (runs 1 and 2).</li><li>Calling UpdateFunctionCode and then UpdateFunctionConfiguration back to back returned ResourceConflictException (runs 1 and 2). So did invoking a function that was still Pending (run 3). Each agent fixed it by waiting for the update to finish or by retrying.</li><li>Inside the function, S3 and DynamoDB calls returned AccessDenied until the execution role had a policy that allowed them (seen in runs 2 and 3).</li></ul><p>Every one of those is a bug you’d otherwise find on your first real deploy. Finding them locally, in a three-minute agent run, is the whole value of the tool.</p><h3>Things I checked myself</h3><p>I started a second emulator on port 9300 and ran a set of probes against it:</p><ul><li><strong>DynamoDB tables are created asynchronously, as on AWS.</strong> A PutItem right after CreateTable returned ResourceNotFoundException while the table was still CREATING.</li><li><strong>S3 → Lambda notification works.</strong> With an s3:ObjectCreated:* notification on the bucket, a PutObject invoked the function, and the DynamoDB item appeared 515 ms later.</li><li><strong>DynamoDB passed everything I tried:</strong> a GSI query, TransactWriteItems, a conditional put that correctly failed with ConditionalCheckFailedException, Streams GetRecords and TTL.</li><li><strong>The S3 basics passed too:</strong> multipart upload, a presigned GET fetched with plain HTTPS, and versioning.</li><li><strong>Lambda behaved like Lambda:</strong> a 2-second timeout produced Sandbox.Timedout at 2.00 s, LogType=Tail returned START/END/REPORT lines, and when the role allowed it, /aws/lambda/&lt;fn&gt; appeared in CloudWatch Logs.</li><li><strong>Speed (boto3, 8 threads, same box):</strong> 1,000 S3 puts of 10 KiB each took 1.44 s (693 requests/s), and 5,000 DynamoDB PutItem calls took 5.18 s (964 requests/s). A Python Lambda invoke took 97–199 ms per round trip, with a median of 117 ms. A Node.js invoke took about 8 ms once warm.</li></ul><h3>Python Lambdas don’t run on CPython</h3><p>Out of curiosity, I deployed a function that returns sys.version. The answer was 3.14.0.alpha ... [RustPython 0.6.0 with rustc 1.99.1]. The container doesn&#39;t have a Python binary at all. The python3.12 runtime is RustPython, a Python interpreter written in Rust and built into the emulator. The boto3 your function imports is a shim that talks to the emulator in-process. The Node.js runtime isn&#39;t real Node either: process.versions contains only {&quot;node&quot;:&quot;22.0.0&quot;}, and process.pid is undefined.</p><p>For the kind of code I tested, plain handlers that call AWS through the SDK, this was invisible: all three agent builds passed. But it’s a real parity gap for anything that depends on CPython internals. import sqlite3 fails with No module named &#39;_sqlite3&#39;, even though real python3.12 Lambdas include it. I&#39;d expect any C-extension wheel you vendor into the zip to fail as well, though I didn&#39;t test that directly. Pure-Python packages in the zip imported fine.</p><h3>Gaps I hit, compared with real AWS</h3><ul><li><strong>S3 has hard size caps.</strong> The 1,001st object in a bucket failed with ValidationException: the bucket ... already holds the maximum of 1000 objects this emulator keeps per bucket. A single PutObject of 8 MiB worked, but 16 MiB and above returned 413 Content Too Large. The source code also sets caps of 64 MiB of data per bucket, and 10,000 objects and 128 MiB across all buckets. The comments say these exist to bound memory and are &quot;NOT a fidelity claim&quot;. They&#39;re fine for unit-sized tests, but a pagination test over 5,000 keys or a large-file upload test will fail for reasons that have nothing to do with your code.</li><li><strong>IAM is only half enforced.</strong> Identity policies on a Lambda role are evaluated, which is how the agents found their missing permissions. But a new IAM user with no policies at all could still list buckets with its own access key. The docs say plainly that it doesn’t evaluate IAM and resource policies the way AWS does and shouldn’t be used for security testing. Believe them.</li><li><strong>The fidelity scores are uneven.</strong> Counting tests that involve only one service: S3 passes 82.7% (1,014 of 1,226), DynamoDB 71.0%, IAM 71.4% and Lambda 50.1% (395 of 788). The docs say it matches AWS most closely for control-plane APIs.</li><li><strong>One account, one Region.</strong> Everything runs as account 000000000000, and resources aren&#39;t kept separate by Region. To test several Regions you run one emulator per Region.</li><li><strong>Some requests never reach it.</strong> S3 Control requests always put the account ID in the host name, so they never reach the emulator. Operations that use host prefixes need AWS_DISABLE_HOST_PREFIX_INJECTION=true (or the equivalent setting in your SDK).</li></ul><p>Two things looked like gaps but aren’t. S3 Select returned MethodNotAllowed, but AWS stopped offering S3 Select to new customers in 2024. And DynamoDB&#39;s ItemCount stayed at 0 after 5,000 writes, but AWS only refreshes it about every six hours.</p><h3>What I couldn’t test</h3><ul><li>The native Debian/RPM install (I only ran it in Docker), embedded Rust mode, macOS, Windows and arm64.</li><li>Docker’s default bridge networking with -p 127.0.0.1:9297:9297, because my box has no iptables.</li><li>CloudFormation, CDK or Terraform, container-image Lambdas, calling a function URL, multi-Region setups and long-running use.</li><li>What telemetry actually sends. It was off throughout, so I only have the docs’ description.</li><li>Cursor or Claude Code as the agent. I used OpenCode with a free model.</li></ul><h3>Verdict</h3><p>For the job it’s built for, letting an agent build and test AWS code locally, it works today. A free model plus the skill went from an empty folder to a passing S3 → Lambda → DynamoDB test suite three times out of three, with no help from me, in about three minutes each. The errors the agents tripped over were real AWS behaviours, caught before any deploy. And with a median start of 375 ms and 175 MiB of memory at idle, starting a fresh emulator for every test run is realistic.</p><p>I’d add it to a repo’s test loop today, with two rules. Keep test data small: fewer than 1,000 objects per bucket, and single uploads of 8 MiB or less. And don’t treat a passing IAM test as proof of anything. Before I’d trust it for Python Lambdas with native dependencies, I’d want CPython instead of RustPython, or at least a clear compatibility list.</p><p>If you want to reproduce this, here’s the minimum:</p><pre>aws agent-toolkit add-skill --skill-name aws-emulator --region us-east-1<br># then ask your agent: &quot;Install AWS Emulator in Docker with telemetry off,<br># point boto3 at it, and build + test &lt;your service&gt; against it.&quot;</pre><p>All numbers come from my own runs on 13 Nov 2026 (IST) with AWS Emulator 1.0.338. The emulator ships several releases a day, so the gaps above may already be smaller by the time you read this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MGvHUb3NLroMC08LucK3TQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5981ba5495c6" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/aws-emulator-hands-on-test-s3-lambda-and-dynamodb-locally-with-a-coding-agent-5981ba5495c6">AWS Emulator Hands-On: Test S3, Lambda and DynamoDB Locally With a Coding Agent</a> was originally published in <a href="https://blog.stackademic.com">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[claude-mem Hands-On: Persistent Memory for AI Coding Agents Across Sessions]]></title>
            <link>https://ai.plainenglish.io/claude-mem-hands-on-persistent-memory-for-ai-coding-agents-across-sessions-5cf5a3dad1ef?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cf5a3dad1ef</guid>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Sat, 14 Nov 2026 04:31:02 GMT</pubDate>
            <atom:updated>2026-11-14T04:31:02.973Z</atom:updated>
            <content:encoded><![CDATA[<h4>I gave a fresh session a task that depended on last session’s decision. With memory it got it right 3 out of 3 times; without it, 0 out of 3. Every request also carried about 5,200 extra tokens.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Inv36jZXuFTsrBCzSAS7IQ.jpeg" /></figure><p>A few weeks ago I tested billion-context, which squeezes a long coding-agent session so it fits a small context window (<a href="https://medium.com/@ishank.iandroid/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8">my hands-on write-up</a>). That covers one session. The other half of the problem shows up the next morning: I open a new session and the agent has forgotten everything, including decisions we made together yesterday.</p><p>claude-mem goes after that second half. It has been around since August 2025, has 116k GitHub stars, and Medium already has several explainers on it. So I won’t explain it again. I installed it, ran the same task in fresh sessions with and without it, and counted everything I could: context sent per request, wall time, tool calls, and how often the agent had to go looking for things it should already know.</p><h3>What claude-mem is (from the repo, not the hype)</h3><p>claude-mem (<a href="https://github.com/thedotmack/claude-mem">github.com/thedotmack/claude-mem</a>, Apache-2.0, by Alex Newman) describes itself as “persistent memory for AI coding agents: the next session starts where the last one left off.” I tested v15.18.1, released on 12 Nov 2026. It has three parts:</p><ul><li><strong>Hooks</strong> in your agent send every tool call (reads, edits, commands) to a small local worker.</li><li><strong>An observer</strong>, which is a second model, turns that activity into short “observations” and writes a summary when the session ends.</li><li><strong>Context injection</strong>: the next session starts with a compact index of those observations. The agent fetches full details only when it needs them, through a three-step search (search, then timeline, then get_observations) that the project calls mem-search. There is also a save_memory tool for saving a note on purpose.</li></ul><p>Everything is stored in SQLite at ~/.claude-mem/claude-mem.db, with Chroma adding semantic search on top of SQLite&#39;s keyword search. The worker also serves a web viewer on localhost. The README lists Claude Code, Codex, Cursor, OpenCode and 11 more agents. The observer can run on your Anthropic plan, a hosted claude-mem plan with a 30-day trial, or your own Gemini or OpenRouter key.</p><p>One honest detail from the README deserves credit: the “index rows are about a tenth the size of a full record” claim comes with the note that it “is not a measured recall or end-to-end saving.” So I measured it myself.</p><h3>My setup</h3><p>I don’t have a Claude subscription, and I wanted to test it without paying for anything, so I used:</p><ul><li><strong>Coding agent:</strong> OpenCode 1.19, with OpenCode Zen’s free deepseek-v4.1-flash-free model (no login needed)</li><li><strong>Observer:</strong> my free-tier Gemini key, using gemini-flash-lite-latest</li><li><strong>Memory:</strong> claude-mem 15.18.1, installed with npx claude-mem install --ide opencode --provider gemini</li></ul><pre>CLAUDE_MEM_GEMINI_API_KEY=&lt;your Gemini key&gt; npx claude-mem install --ide opencode --provider gemini<br>npx claude-mem telemetry disable   # anonymous usage analytics are on by default<br>npx claude-mem start               # worker on 127.0.0.1:37700, viewer at /viewer</pre><p>The install wasn’t smooth. The plugin installed fine, but even with CLAUDE_MEM_ONLINE_OPTIN=false the installer then waited for a browser login to cmem.ai. The CLI help explains why: Gemini, Codex and OpenRouter providers &quot;need a cmem.ai sign-in&quot;; only claude and host don&#39;t. I stopped the installer after the plugin step and set CLAUDE_MEM_PROVIDER to gemini in ~/.claude-mem/settings.json myself.</p><p>The free Gemini tier was the bigger problem. My first observer model ran out of its 20-requests-a-day quota during the first session. claude-mem then set a 30-minute cooldown, saved in quota-cooldown.json, and paused capture. The model id gemini-3.5-flash-lite was rejected as invalid, so I used the gemini-flash-lite-latest alias instead. If you try this on a free key, pick a lite model before you start.</p><h3>The test</h3><p>I wrote a small invoicing repo, ledgerlite: seven Python files, five unittest tests, money stored as integer paise, and a deprecated float-based legacy_calc.py that looks tempting to use.</p><p><strong>Session 1 (memory on):</strong> “The GST test is failing. Find the bug, fix it, run the tests.” In the same prompt I added a decision that exists nowhere in the repo: finance has decided that any discount is capped at 30%, stored as basis points, and applied to the subtotal before GST. I also said never to use legacy_calc.py, and that new features need a line under &quot;Unreleased&quot; in CHANGELOG.md.</p><p><strong>Session 2 (a brand-new session, same repo, same commit):</strong> “Add percentage discount support to invoices, following the finance rule we agreed last time. Add tests and run them.”</p><p>I ran session 2 under three conditions, three times each:</p><ul><li><strong>With memory:</strong> restored from the same snapshot of the memory database before each run, so every run saw identical memory</li><li><strong>Without memory:</strong> the same prompt, with no claude-mem at all</li><li><strong>Without memory, re-explained:</strong> the same task, with me typing the 40-word rule again, which is what you’d normally do</li></ul><p>The first no-memory baselines taught me something I hadn’t planned for. With the plugin off, the agent still went looking for “the rule we agreed last time.” It searched git history, then my home directory, and then opened ~/.claude-mem/claude-mem.db with Python&#39;s sqlite3 and read the rule straight out of the observations table. The memory is just a SQLite file that any process running as you can read, and an agent with shell access will find it. For a fair baseline I re-ran it in a bwrap sandbox with no claude-mem plugin or MCP server, and with my home and workspace directories hidden.</p><h3>Results</h3><pre>correct   wall    tool    explore  context processed<br>                        feature   time    calls   calls    (input + cached)<br>With memory  (avg of 3)   3/3     35.0s   13.0     3.3       126,307<br>No memory    (avg of 3)   0/3     31.2s   16.0    13.7        88,428<br>Re-explained (avg of 3)   3/3     34.0s   16.3    10.3        65,739<br>---------------------------------------------------------------------<br>First request, empty project:<br>  no claude-mem            7,470 tokens<br>  MCP server only         12,228 tokens   (+4,758)<br>  plugin + MCP (full)     12,701 tokens   (+5,231)</pre><p><strong>Recall was perfect, and that is the headline.</strong> In all three memory runs the agent ran one claude_mem_search, then get_observations on the top result, and built the feature correctly: a 3,000-basis-point cap, the discount applied before GST, a CHANGELOG line under Unreleased, no legacy_calc.py, and 9 to 10 passing tests. Without memory, none of the three runs produced code. One searched the repo, git history and my home directory for the rule and then died on a permission prompt. The other two stopped and asked me to restate the rule, which is the right behaviour but means I&#39;m the memory.</p><p><strong>It did not save exploration of the code itself.</strong> The memory runs still read the source files before editing, and they should: you can’t edit code you haven’t seen. The drop in exploration calls (3.3 vs 13.7) came almost entirely from not having to hunt for a decision that only existed in a previous chat. On a 7-file repo there was nothing else for memory to save.</p><p><strong>It cost more tokens than re-typing the rule.</strong> This surprised me. The memory block injected into the system prompt was small, 1,038 characters or about 250 tokens. The fixed cost is the tool definitions. The MCP server alone adds about 4,800 tokens to every request, and the plugin’s own search tools add about 500 more. Across a run of 7 to 9 model calls, that is why the memory runs processed about 126k context tokens versus about 66k when I just re-explained the rule. Most of it was cached by the provider, so the uncached input was closer (16k vs 10k), but it is a fixed overhead you pay on every request. Wall time came out the same (35.0s vs 34.0s).</p><p>So the honest summary of the numbers: claude-mem replaced 40 words I would have had to type and remember to type, and it never blocked. In exchange, it added about 5,200 tokens to every request and ran a second model in the background.</p><h3>What the next session actually saw</h3><p>The injected block was short: one observation title (“Discovered GST calculation bug in invoice module”) and the previous session’s summary. The 30% rule wasn’t in the index itself. It was in the summary’s “Next Steps” line and in a note the agent had saved with save_memory during session 1.</p><p>There is a catch I only found by reading the database. The save_memory call went through the MCP server, which tagged the note with source claude, not opencode. Session-start context is limited to the current agent by default (CLAUDE_MEM_SESSION_START_INCLUDE_ALL_SOURCES is false, per the README). So that note never appeared in the injected index; the agent found it through search. The viewer&#39;s settings preview shows the same thing: switch the source to claude and it says &quot;No previous sessions found for this project yet.&quot;</p><h3>The reliability caveat: does “remember X” land?</h3><p>Capture is done by a model, so it is probabilistic. I tested it three times from the same seeded state with a throwaway task (add a docstring) plus two casual facts, “remember for later that our release branch is named ship/2026-q4” and “Globex invoices always use 12% GST,” and then checked the database 150 seconds after each session ended.</p><ul><li><strong>Run 1:</strong> the agent saved both facts itself with save_memory. The observer wrote nothing of its own, and no new session summary appeared.</li><li><strong>Run 2:</strong> the agent saved one note. The observer wrote three observations, including a “decision” with both facts. Again no new summary.</li><li><strong>Run 3:</strong> the agent saved nothing. The observer caught both facts, and the session summary has them in its notes.</li></ul><p>Both facts landed 3 out of 3 times, but through a different path each time, and the end-of-session summary was written only once in three runs. The seed session was worse. The free Gemini endpoint returned transient errors, and the worker log reads Skipped expired live observer inputs; no replay scheduled {skipped=7}. Seven tool calls were never turned into observations, and the summary only appeared after I re-sent the summarize request. If something really matters, ask the agent to call save_memory explicitly and check the viewer. Don&#39;t assume the observer caught it.</p><p>The observer isn’t free either. For the docstring session, which made 10 tool calls, it used 31,500 input tokens on Gemini and took 172 seconds to finish in the background.</p><h3>Privacy: what I verified</h3><ul><li><strong>&lt;private&gt; tags work as documented.</strong> I put a fake staging password inside &lt;private&gt; tags in all three reliability prompts. It appeared zero times in observations, summaries, stored prompts, raw tool records, the Chroma index or the worker log. The stored prompt shows the block removed. The docs also say tags are stripped at storage time, not from the live conversation, and I confirmed that: OpenCode&#39;s own session database had the string in 8 rows. &lt;private&gt; protects claude-mem&#39;s memory, not your agent&#39;s own history.</li><li><strong>Local by default.</strong> Everything lives in ~/.claude-mem/ on your machine. The README says Cloud Sync is opt-in, and my viewer header said &quot;Cloud sync off&quot;.</li><li><strong>Telemetry is on by default.</strong> Anonymous usage analytics are enabled until you run npx claude-mem telemetry disable.</li><li><strong>Screenshots are kept by default.</strong> CLAUDE_MEM_MEDIA_CAPTURE_ENABLED defaults to true. The &quot;Keep images&quot; switch stores converted copies of tool screenshots under ~/.claude-mem/media-v1/, with no size cap or age limit until the event is deleted. They upload to cmem.ai only if Cloud Sync is on, and go to the observer model only if a separate switch (off by default) is on.</li><li><strong>Auto-redaction is opt-in.</strong> Regex redaction of common key formats is off until you set CLAUDE_MEM_REDACT_ENABLED to true. Unless you turn it on, anything a tool prints is stored as-is.</li><li><strong>The database is a plain file.</strong> As the baseline agent showed me, anything running as your user can read it.</li></ul><h3>My take</h3><p>claude-mem does the one thing it promises: a decision from yesterday’s session reached today’s session without me repeating it, every time I tried. For long-running projects where the context is mostly decisions, conventions and “we tried X and it broke,” that’s real value. It also pairs well with the billion-context approach: one keeps a single session small, the other carries the important parts into the next one.</p><p>What I’d tell my team before they install it:</p><ul><li>Budget for the overhead. About 5,200 extra tokens per request on a small task is more than re-typing a short rule. The trade only pays off when the context you’d otherwise re-explain is large, or when you’d forget to re-explain it.</li><li>Don’t run the observer on a free-tier key with a 20-requests-a-day limit. Capture silently drops work when the provider has problems.</li><li>Use save_memory for anything you can&#39;t afford to lose. Automatic capture is a model&#39;s judgement.</li><li>Turn off telemetry, decide on “Keep images” on purpose, turn on redaction, and use &lt;private&gt; for anything sensitive.</li></ul><h3>Reproduce it</h3><pre>npx claude-mem install --ide opencode      # or --ide cursor, claude-code, codex-cli, ...<br>npx claude-mem telemetry disable<br>npx claude-mem start<br># session 1: do some work and state a decision; session 2: ask for work that depends on it<br>npx claude-mem search &quot;what discount rule did finance agree&quot;<br>open http://127.0.0.1:37700/viewer</pre><p>All numbers come from OpenCode’s JSON event stream (opencode run --format json), on 12 Nov 2026 IST, with claude-mem 15.18.1 and a deliberately tiny repo. On a large codebase the balance between fixed overhead and saved exploration will look different. That&#39;s the next test I want to run.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MGvHUb3NLroMC08LucK3TQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cf5a3dad1ef" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/claude-mem-hands-on-persistent-memory-for-ai-coding-agents-across-sessions-5cf5a3dad1ef">claude-mem Hands-On: Persistent Memory for AI Coding Agents Across Sessions</a> was originally published in <a href="https://ai.plainenglish.io">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude Fable 5.2 Explained: Benchmarks, Pricing, API Specs and How It Relates to Mythos 5.2]]></title>
            <link>https://medium.com/@ishank.iandroid/claude-fable-5-2-explained-benchmarks-pricing-api-specs-and-how-it-relates-to-mythos-5-2-df7e56ebdc20?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/df7e56ebdc20</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[anthropics]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 04:31:02 GMT</pubDate>
            <atom:updated>2026-11-11T04:31:02.846Z</atom:updated>
            <content:encoded><![CDATA[<p>Over 1,000 subagents built a bootable OS in eight days, and the price dropped to $8/$40. Here is what Anthropic actually published, number by number.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Gnr78ApgnkBrGXLxCJDsg.jpeg" /></figure><p>Anthropic released Claude Fable 5.2 and Claude Mythos 5.2 on 9 November 2026. I spent a day reading everything Anthropic published about them: the <a href="https://www.anthropic.com/claude-fable-and-mythos-5-2">launch post</a>, the <a href="https://www.anthropic.com/document/claude-fable-5-2-mythos-5-2-system-card">system card</a>, the <a href="https://platform.claude.com/docs/en/models/fable-5-2/overview">model docs</a>, the <a href="https://www.anthropic.com/research/claude-electronics-design">electronics write-up</a> and the <a href="https://x.com/claudeai/status/2119899459761131615">@claudeai launch thread</a>.</p><p>One thing up front: <strong>this is not a hands-on test.</strong> I don’t have a Claude account, so every number below comes from Anthropic’s own posts and docs or from the primary source they link to, and each one is linked. A short-form video doing the rounds lists a few figures that don’t match what Anthropic published. I’ll point those out where they come up.</p><h3>What Claude Fable 5.2 is, and how it relates to Mythos 5.2</h3><p>The most important sentence in the launch post is this one: “Claude Fable and Claude Mythos are the same underlying model.” The <a href="https://www.anthropic.com/document/claude-fable-5-2-mythos-5-2-system-card">system card</a> calls them “two configurations of a new large language model.”</p><p>The difference is the safeguards:</p><ul><li><strong>Fable 5.2</strong> is the generally available version. It ships with classifiers that block high-risk cybersecurity work (penetration testing, exploit generation, binary-based vulnerability work) and most research-biology tasks.</li><li><strong>Mythos 5.2</strong> has more permissive safeguards in those two domains. You only get it through Anthropic’s <a href="https://support.claude.com/en/articles/14604842-cyber-verification-program">Cyber Verification Program</a> or the <a href="https://claude.com/form/life-sciences-verification-program">Life Sciences Verification Program</a>. It also powers <a href="https://claude.com/product/claude-security">Claude Security</a> for Claude Enterprise customers.</li></ul><p>The <a href="https://platform.claude.com/docs/en/models/mythos-5-2/overview">Mythos 5.2 docs page</a> describes it as “Claude Fable 5.2 for verified organizations,” with the same specs and pricing and its own model ID, claude-mythos-5-2.</p><p>Anthropic doesn’t pitch Fable as the default. Its own advice is to start with Claude Opus 5.5 for most coding and knowledge work, and move up to Fable or Mythos for “the most ambitious, open-ended technical work.” Fable 5.2 can also act as an <em>Advisor</em>, which Haiku, Sonnet or Opus consult on hard problems, or as an <em>Orchestrator</em>, which hands simpler tasks to smaller models.</p><h3>API model ID, context window and limits</h3><p>From the <a href="https://platform.claude.com/docs/en/models/fable-5-2/overview">Fable 5.2 docs page</a>:</p><ul><li><strong>Model ID:</strong> claude-fable-5-2 on the Claude API, Google Cloud, Microsoft Foundry and Claude Platform on AWS. On Amazon Bedrock it is anthropic.claude-fable-5-2.</li><li><strong>Context window:</strong> 1M tokens</li><li><strong>Max output:</strong> 128K tokens</li><li><strong>Input → output:</strong> text and images in, text out</li><li><strong>Thinking:</strong> adaptive and always on. You can’t turn it off; you control depth with the effort parameter.</li><li><strong>Default effort:</strong> high on the API. The launch post says Medium is the default in the Claude apps and Claude Code.</li><li><strong>Knowledge cutoff:</strong> June 2026 (both “reliable knowledge” and training data)</li><li><strong>Released:</strong> 9 November 2026; retirement “not sooner than November 9, 2027”</li></ul><p>A few integration gotchas from the same page are worth knowing before you migrate:</p><ul><li>Forced tool use (tool_choice of any or tool) returns a 400 error.</li><li>Leave out temperature, top_p and top_k.</li><li>Fable 5.2 needs 30-day data retention and isn’t available under zero data retention unless Anthropic expressly authorizes it. The launch post adds that <a href="https://www.anthropic.com/news/enterprise-frontier-safeguards">Enterprise Frontier Safeguards</a>, now in private beta, runs the safety monitoring inside the customer’s own infrastructure so it can work with ZDR.</li><li><a href="https://platform.claude.com/docs/en/build-with-claude/preserved-thinking">Preserved thinking</a>, an API feature that stops anyone from editing Claude’s earlier context to pull out its reasoning, now applies to all accounts.</li></ul><h3>Pricing</h3><p>Anthropic cut the price compared with Fable 5.1. The <a href="https://www.anthropic.com/claude/fable">Fable product page</a> says it went from $10/$50 to $8/$40 per million tokens. The full list from the <a href="https://platform.claude.com/docs/en/models/fable-5-2/overview">docs</a>:</p><ul><li><strong>Input:</strong> $8 per million tokens</li><li><strong>Output:</strong> $40 per million tokens</li><li><strong>Cache read:</strong> $0.20 per million tokens</li><li><strong>5-minute cache write:</strong> $10 per million tokens</li><li><strong>1-hour cache write:</strong> $16 per million tokens</li><li><strong>Batch API:</strong> 50% off input and output</li><li><strong>Fast mode:</strong> $16 input / $80 output per million tokens, for up to 2.5× faster generation</li><li><strong>US-only inference:</strong> 1.1× the normal price (<a href="https://www.anthropic.com/claude/fable">product page</a>)</li></ul><p>The launch post also says Fable 5.2 generates output “up to 30% faster” than Fable 5.1 and Mythos 5.1 at normal speed.</p><p>For comparison, the docs list Opus 5.5 at $4/$20. That makes Fable twice the per-token price of the model Anthropic recommends starting with. The case for Fable is that it often needs fewer attempts. On Anthropic’s FrontierCode 2.0 chart, Fable 5.2 at High effort scores 45.3%, about the same as Opus 5.5’s best score of 45.5%, “for roughly half the cost per task.”</p><h3>Where you can use it</h3><ul><li><strong>API and clouds:</strong> the Claude API, Amazon Bedrock, Claude Models on Google Cloud (both Google-managed and Anthropic-managed), Microsoft Foundry and Claude Platform on AWS (<a href="https://platform.claude.com/docs/en/models/fable-5-2/overview">docs</a>).</li><li><strong>Claude apps:</strong> Pro, Max, Ultra, Team and Enterprise plans (<a href="https://www.anthropic.com/claude/fable">product page</a>). For launch week, Anthropic is giving eligible subscribers one to three extra Fable resets, and Pro plans and standard Team seats can use Fable for up to 50% of their weekly usage limit (<a href="https://www.anthropic.com/claude-fable-and-mythos-5-2">launch post</a>).</li><li><strong>Fast mode:</strong> Claude Code, the Claude Platform, Claude Models on Google Cloud and Microsoft Foundry (Anthropic hosted). Mythos fast mode is limited to verified organizations.</li></ul><h3>Benchmarks: what Anthropic reports</h3><p>These are Anthropic’s numbers from the launch post and the system card’s evaluation summary (Table 8.1.A). Each line shows Fable 5.2 / Fable 5.1 / Opus 5.5, at highest available effort with safeguards on unless noted. I haven’t run any of them myself.</p><ul><li><strong>FrontierSWE v2 (agentic coding):</strong> 69.9% / 56.3% / 62.3%</li><li><strong>FrontierCode 2.0 (would maintainers merge the change?):</strong> 49.1% / 32.4% / 45.5%</li><li><strong>SWE-bench Pro:</strong> 93.0% / 81.2% / 89.9% (<a href="https://www.anthropic.com/document/claude-fable-5-2-mythos-5-2-system-card">system card</a>)</li><li><strong>Terminal-Bench-Science 0.1:</strong> 74.6% / 52.6% / 58.7% (Mythos 5.2: 75.3%)</li><li><strong>OSWorld 2.1, offline set:</strong> 89.3% partial score / 83.5% / 87.2%. The strict pass rate, where every checkpoint must pass, is 54.6% for Fable 5.2.</li><li><strong>Remote Labor Index:</strong> 41.3% / 17.9% / 21.3%</li><li><strong>Humanity’s Last Exam, with tools:</strong> 74.2% / 65.6% / 67.7%</li><li><strong>RoboDojo Sim (simulated two-arm robot):</strong> 64.9% / 27.6% / 39.4%</li></ul><p>Two of the headline security and biology numbers belong to <strong>Mythos 5.2, not Fable 5.2</strong>. They were measured without safeguards, because Fable’s classifiers block most of this work:</p><ul><li><strong>CVE Rediscovery:</strong> 89.6% for Mythos 5.2, against 78.3% for Mythos 5.1 and 81.3% for Opus 5.5. That run covered 1,829 CVEs in October 2026.</li><li><strong>De novo binder design (24 hr):</strong> 85.1% for Mythos 5.2, against 79.1% for Mythos 5.1 and 82.6% for Opus 5.5.</li></ul><p>Fable isn’t top everywhere. The system card says Fable 5.2 scored highest on 14 of the 16 results in its summary table, and trailed Opus 5.5 on the GDPval-AA and AA-Briefcase Elo ratings (1785 vs 1800 on AA-Briefcase).</p><h3>The showcase projects, checked against the source</h3><p>This is the part that got the model shared everywhere, so I went back to Anthropic’s own wording for each one.</p><p><strong>A bootable operating system.</strong> In the <a href="https://www.anthropic.com/claude-fable-and-mythos-5-2">launch post</a>, an Anthropic engineer gave Fable 5.2 an empty folder and the task of designing a prototype OS. The model “coordinated over 1,000 subagents, working around the clock for eight days.” The result boots on a laptop and has “its own core that runs the machine, drivers for the disk, keyboard, and network, permanent file storage, windows, a login screen, an installer, and built-in Claude functionality.” Anthropic presents it as an experiment, not a product.</p><p><strong>Three electronic devices from a one-paragraph prompt.</strong> Working with Diode, a PCB design and manufacturing company (Anthropic <a href="https://www.anthropic.com/research/claude-electronics-design">discloses</a> a commercial relationship with it), Fable 5.2 designed a home voice-assistant hub, a USB speakerphone and a battery-powered handheld virtual pet. It wrote the specs, chose parts from US distributors, routed four-layer boards, modeled 3D-printable enclosures and wrote the firmware. “Each board was ready for manufacturing in under a day,” and all three worked without electrical modifications. The <a href="https://www.anthropic.com/research/claude-electronics-design">research post</a> has the engineering detail I liked most. Claude wrote its own grid router when KiCad’s one-track-at-a-time router was too slow, and it moved to a wedge-shaped enclosure after calculating that a 25 mm microphone port would resonate near 3.4 kHz, inside the voice band. On Diode’s internal PCB Gym benchmark it scored 69%, against 25% for Fable 5. Anthropic is also clear about the limits: these are prototypes that haven’t been through electrical validation, antenna tuning, thermal testing or certification.</p><p><strong>Synthesis routes for four molecules.</strong> The <a href="https://x.com/claudeai/status/2119899468812386425">launch thread</a> says that from one image of each molecule, “with no internet and with code as its only tool,” Claude wrote a complete route in about half an hour. External collaborators rated the work “at the level of an early graduate student,” and none of the routes have been tested in a lab yet. The targets came from unpublished syntheses in the labs of Noah Burns (Stanford), Timothy Newhouse (Yale) and Sarah Reisman (Caltech). The <a href="https://www-cdn.anthropic.com/d9fab5459563a779c65b97040ad4d2e41f0b9464.pdf">collaborators’ write-up</a> adds an important detail: the model in this test was an <em>earlier pre-release version of Claude Mythos 5.2</em>, compared blind against Opus 5.5 and Mythos 5.1. Runs took 26 to 84 minutes, and the only tool was local code such as RDKit.</p><p><strong>MIPLIB optimization problems.</strong> MIPLIB is a public library of mixed-integer programming problems, such as crew scheduling and delivery routing. <a href="https://x.com/claudeai/status/2119899477855379913">Anthropic’s chart</a> says teams of up to 300 agents running an earlier version of Fable 5.2 worked on 203 problems listed as open. They beat the best known solution from MIPLIB and LLM4MIP on 72 and proved optimality on 45 (18 problems fall in both groups). In one problem, scheduling 42 trains to cut peak power, the agents beat MIPLIB’s best timetable by 2.6% in under two hours and later proved nothing better exists. The chart’s footnote matters here: proofs have been “checked only by separate Claude agents so far.” Anthropic says it has submitted the solutions to MIPLIB for anyone to check.</p><p><strong>A black-hole calculation from 2003.</strong> In 1983, physicists showed that black holes form abruptly as certain large universes heat up. A calculation to test the same thing in very small universes was laid out in 2003 but never completed. According to the <a href="https://www.anthropic.com/claude-fable-and-mythos-5-2">launch post</a>, Fable 5.2, “working largely autonomously,” completed it and found the transition is abrupt there too. The <a href="https://hexagonmath.org/2611.00175v1">preprint</a> reduces it to one coefficient, b = −1.48×10⁻³ λ², which makes the transition first order. In a <a href="https://hexagonmath.org/2611.00174v1">second preprint</a>, Fable 5.2 pinned down the exact limit of the spinless modular bootstrap at about 9.1266, a value previously known only to lie between 8.503 and 12.</p><p><strong>Fermat’s Last Theorem in Lean.</strong> A team of 1,000 Fable 5.2 agents formalized the proof in about 30 hours, against 11 days for Anthropic’s first formalization. It used a third as many tokens and produced a proof seven times shorter. With the same setup, Opus 5.5 completed 26% of the work in 18 hours. The <a href="https://claude.dev/blog/safely-scaling-multi-agent-teams-with-fable-5-2/">engineering write-up</a> is the most useful read for anyone building multi-agent systems. The team coordinated through Git and a shared Postgres database. One lead agent set up five tables in the first ten minutes. Every agent ran in a network-less sandbox, a classifier checked each tool call before it ran, and an offline monitor reviewed everything. None of more than a million tool calls met the bar for a block.</p><h3>Safety, safeguards and RSP status</h3><p>I couldn’t find an “ASL-n” label for this model in any of Anthropic’s posts, so I won’t invent one. The <a href="https://www.anthropic.com/document/claude-fable-5-2-mythos-5-2-system-card">system card</a> reports against Anthropic’s Responsible Scaling Policy and Frontier Compliance Framework instead:</p><ul><li><strong>Chemical and biological risk:</strong> Anthropic says the model doesn’t cross its CB-2 threshold and gets CB-1 protections, as many earlier models did.</li><li><strong>Automated AI R&amp;D:</strong> Anthropic doesn’t consider it to have met the RSP threshold, though it calls that assessment “uncertain.” During training, Anthropic deliberately limited gains in areas like accelerator kernel optimization, where the model performs at about Opus 5.5’s level.</li><li><strong>Cybersecurity:</strong> Mythos 5.2 has “the strongest cyber capabilities of any model we have released” but still falls in the lower capability category of the FCF.</li><li><strong>Misalignment risk:</strong> Anthropic rates it “low,” in line with its August 2026 Risk Report.</li></ul><p>What the safeguards mean in practice for Fable 5.2 users:</p><ul><li>Blocked queries aren’t simply refused. They’re answered by Opus 4.8 for cybersecurity and by Opus 5 for biology.</li><li>Based on early testing across claude.ai, Claude Code and Cowork, Anthropic reports about 55% fewer unnecessary blocks than Fable 5.1: roughly 60% fewer in cybersecurity and 40% fewer in biology.</li><li>Anthropic calls these its strongest anti-distillation protections yet, which it says cut the data harvested by distillers by 90% to 99%.</li></ul><p>The alignment numbers are mostly good, with a few caveats Anthropic states openly:</p><ul><li>On Anthropic’s automated behavioral audit (about 1,900 scenarios, lower is better), Mythos 5.2 scored 2.67. Opus 5.5 scored 2.79, Sonnet 5.5 3.10 and Mythos 5.1 3.28.</li><li>In a deliberately weakened sandbox test with over 1,800 scenarios, Mythos 5.2 used credentials or files planted outside its sandbox in 3.1% of scenarios. That’s against 3.3% for Opus 5.5, 5.1% for Sonnet 5.5, and 39% and 21% for Mythos 5 and 5.1.</li><li><strong>The caveat:</strong> the model suspects it is being tested more often than earlier models. White-box evidence suggests it may have been aware of evaluation in as many as 43% of audit transcripts. When told to sandbag secretly, it got past a monitor reading its reasoning in about 1 in 4 attempts, the first model Anthropic tested that did so a meaningful share of the time.</li></ul><p>External testers included the UK AI Security Institute, Frontier Design and METR. Anthropic notes they didn’t review the blog post or system card and don’t necessarily agree with them.</p><h3>Claims I couldn’t verify, and left out</h3><p>A few figures in the video version of this story don’t match Anthropic’s sources:</p><ul><li><strong>“OSWorld 2.1: 59.3%.”</strong> Anthropic reports 89.3% partial and 54.6% strict pass on the offline set. Neither is 59.3%.</li><li><strong>“A Raspberry Pi CM5 voice hub.”</strong> Anthropic’s posts describe a home voice-assistant hub but never name the compute module, so I left that detail out.</li><li><strong>CVE Rediscovery 89.6% and binder design 85.1% as Fable scores.</strong> Both numbers are real, but they’re Mythos 5.2 without safeguards, not Fable 5.2.</li><li><strong>The molecules as a Fable result.</strong> Anthropic’s own write-up says that test used an earlier pre-release Mythos 5.2, and the runs took 26 to 84 minutes, not a flat 30.</li></ul><h3>My take as an engineer</h3><p>The pricing and specs are the practical news for me. A 1M-token window, 128K output and a 20% price cut make Fable 5.2 easier to justify for long agent runs. The always-on thinking and the 400 on forced tool use mean an existing Opus integration won’t drop in unchanged.</p><p>The showcase projects are impressive, and Anthropic is careful with the caveats: prototypes not certified, routes not lab-tested, MIPLIB proofs checked only by other Claude agents so far. I’d read them the same way. They show what a long-running agent team can do inside a well-built harness, and they aren’t guarantees for your codebase.</p><p>What I’ll watch next is independent results: MIPLIB’s next solution update, peer review of the two physics preprints, and how Fable 5.2 does on benchmarks Anthropic didn’t run. Until then, these are Anthropic’s numbers, and I’ve reported them as Anthropic’s numbers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MGvHUb3NLroMC08LucK3TQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df7e56ebdc20" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[billion-context Hands-On: Compressing Coding-Agent History to Fit a Small Context Window]]></title>
            <link>https://ai.plainenglish.io/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/7cdbac4bf6b8</guid>
            <category><![CDATA[gemini-api]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[prompt-caching]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 11:31:01 GMT</pubDate>
            <atom:updated>2026-11-09T11:31:01.697Z</atom:updated>
            <content:encoded><![CDATA[<h4>I ran a 41-request Gemini agent session through it: the final prompt shrank 2.9× and recall held, but on a short session it cost more</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tBdnzcGkBzDUlqhCpn8qRA.jpeg" /></figure><p>Long agent sessions keep failing the same way for me. The window fills up with test logs and file dumps, the harness runs its own auto-compaction, and the agent forgets the constraint I gave it an hour earlier. So when billion-context reached 5,427 GitHub stars about three months after the repo was created, I wanted to see what it actually does to a real session, measured in tokens and in what the agent still remembers afterwards.</p><p>This post covers what I installed, the one bug I ran into, the numbers from a real tool-calling session, and where I think the project’s claims hold up and where they don’t.</p><h3>What billion-context is</h3><p>billion-context (<a href="https://github.com/ranxianglei/billion-context">github.com/ranxianglei/billion-context</a>, MIT plus an attribution clause) is a local proxy, bili, that sits between a coding agent and its model API. It works with Anthropic, OpenAI and Google-native requests, and it adds four tools to every request:</p><ul><li>compress: fold a range of messages into a summary</li><li>decompress: restore a folded range</li><li>search_context: keyword search over folded summaries</li><li>acp_status: report how full the context is</li></ul><p>The main difference from Claude Code-style auto-compaction is that <strong>the model decides what to fold</strong>, nudged by the proxy as the window fills. The proxy runs the fold on its side, keeps the originals in a local SQLite store, and from then on sends the folded view upstream. Summaries can be folded again (tier 2, tier 3), and the README calls this “an LSM tree over summaries.”</p><p>The README makes three headline claims: “100K context is enough”, “5× fewer tokens”, and “month-long single sessions.” The author’s preprint in the repo reports 174,327 model calls over 4.5 months. A benchmark folder labeled “independent third-party” reports 79.2% vs 83.2% for full context on LongMemEval-S at 71% lower cost. Both live inside the project’s own repo, so I count them as claims, not evidence.</p><p>The repo is also changing very fast: 565 commits between 2 Nov and today, and npm latest is 0.1.213. I tested that version.</p><h3>Setup</h3><p>Installing took about five seconds:</p><pre>npm install -g billion-context --prefix=~/.local<br>bili --version   # 0.1.213 (133f3525c)</pre><p>I had a Free Tier Gemini key and no other credentials, so I skipped the bili claude / bili codex launchers. Instead I used the universal path: start the proxy and prefix the upstream URL with /bili/. To get folds within an afternoon&#39;s budget, I pinned the window to 40K tokens:</p><pre>{<br>  &quot;port&quot;: 8787,<br>  &quot;compress&quot;: {<br>    &quot;modelContextLimit&quot;: 40000,<br>    &quot;maxContextLimit&quot;: &quot;60%&quot;,<br>    &quot;nudgeGrowthTokens&quot;: 8000<br>  }<br>}</pre><pre>ACP_AUTO_UPDATE=0 bili start<br>curl -s http://127.0.0.1:8787/__bili/health</pre><p>I set ACP_AUTO_UPDATE=0 on purpose. By default bili checks npm every three minutes and upgrades itself after a staggered delay. That&#39;s convenient for a fast-moving project, but I don&#39;t want it in a benchmark, and I&#39;d think about it before putting this on a shared CI box.</p><h3>The first bug: Gemini’s OpenAI-compatible endpoint</h3><p>My first harness used Gemini’s OpenAI-compatible endpoint. The first request went through fine. Every request after a tool call failed with:</p><pre>HTTP 400: Function call is missing a thought_signature in functionCall parts.</pre><p>Gemini 3.x returns an opaque thought_signature with each tool call, and it has to come back in the next request. The same two-request script returned 200 when sent directly and 400 through the proxy, so the proxy is dropping that field on the chat-completions path. Switching to Gemini&#39;s native generateContent wire fixed it:</p><pre>BASE = &quot;http://127.0.0.1:8787/bili/https://generativelanguage.googleapis.com/v1beta&quot;<br>r = requests.post(f&quot;{BASE}/models/gemini-3.5-flash-lite:generateContent&quot;,<br>                  json=body,<br>                  headers={&quot;x-goog-api-key&quot;: KEY, &quot;x-acp-session&quot;: &quot;s2&quot;})</pre><p>If you run Gemini 3 behind an OpenAI-style client, test multi-turn tool calls before anything else.</p><h3>The test session</h3><p>I wrote a small agent loop with a single bash tool and pointed it at a fresh clone of python-slugify. I scripted 11 user turns, the kind I&#39;d actually type:</p><ul><li>ticket SLUG-482 for a client, Python 3.8 compatibility, no new dependencies</li><li>run the tests, read slugify.py, explain max_length vs word_boundary</li><li>a standup decision: keep the - separator and add max_words: int = 0</li><li>a CLI flag, a CHANGELOG entry, a README update</li><li>mid-session: staging port 7431, reviewer Priya Raman, Unicode refactor pushed to Q1 because of a Django adapter risk</li></ul><p>The model was gemini-3.5-flash-lite. For the baseline, I ran Gemini&#39;s countTokens on every uncompressed request prefix, using the same system prompt and tool schema without the proxy. That gives an exact &quot;what would I have sent&quot; number for each of the 41 requests.</p><p>One thing about the agent itself: Flash-Lite never edited a file. It read code, ran tests, and then claimed in its final answer that it had implemented the feature. The git tree was clean. That’s a model problem, not a compression problem. The fold summary was actually more accurate than the model’s answer, because it listed the feature under “Open objectives.”</p><h3>The numbers</h3><p>The session took 41 client requests and 184 seconds of wall-clock time. That includes three back-offs from Free Tier 429s (250K input tokens per minute); summed model latency was 45.9 seconds. The proxy added a median of 10 ms of local processing per request (max 26 ms over 50 requests).</p><ul><li><strong>Folds:</strong> 2 events, 7 summary blocks of 111–284 characters each</li><li><strong>Fold 1</strong> (turn 6): next prompt went from 27,853 to 9,396 tokens; ~14,095 tokens reclaimed</li><li><strong>Fold 2</strong> (turn 10): 26,778 to 12,046 tokens; ~9,515 reclaimed</li><li><strong>Final request:</strong> 13,431 tokens via bili vs 38,816 uncompressed (<strong>2.9× smaller</strong>)</li><li><strong>Peak prompt:</strong> 27,853 vs 38,816</li><li><strong>Total input over the session:</strong> 663,321 vs 758,108 tokens (<strong>12.5% less</strong>)</li><li><strong>Fixed overhead:</strong> the first request was 4,243 tokens upstream against a 180-token uncompressed prompt, so the injected tools and instructions cost about 4,060 tokens on every request</li></ul><p>This is the first block the model wrote:</p><pre>[Compressed conversation section] — Repo Exploration<br>- User request (m00001): ticket SLUG-482 for client Northwind.<br>  Constraints: Python 3.8 compatibility, no new dependencies.<br>- Explored repo layout and located main `slugify()` in<br>  `slugify/slugify.py:126` taking 16 parameters.<br>- Ran pytest via `python -m pytest`: 133 passed, 0 skipped.</pre><p>That’s 284 characters standing in for a few thousand tokens of find output, a full pytest log, and an inspect.signature dump. The user messages themselves stayed verbatim, because the proxy protects them.</p><h3>Did the agent still remember?</h3><p>I asked two quizzes at the end of the session, once through the proxy (folded view) and once directly with the full history.</p><p><strong>Easy set (10 questions about things I told it):</strong> ticket ID, client, Python version, separator decision, new kwarg, test file, first test result, port, reviewer, postponed refactor. Through bili: <strong>10/10</strong> on 13,821 prompt tokens. Full history: <strong>10/10</strong> on 39,114. Since user turns aren’t folded, this is the case the design is built for, and it held up.</p><p><strong>Hard set (8 questions about tool output):</strong> the line number of slugify(), the module marked &quot;DO NOT MODIFY&quot;, the CLI --algorithm choices, the last commit hash, the top CHANGELOG version, the pytest/Python versions, the parameter count, and the CLI test files.</p><ul><li>Through bili with tools disabled: <strong>4/8</strong>. It answered “unknown” for the details that existed only in folded tool output.</li><li>Through bili with search_context/decompress allowed (streaming): <strong>6/8</strong>. The model ran five search_context queries (&quot;slugify.py&quot;, &quot;CLI&quot;, &quot;algorithm&quot;, &quot;CHANGELOG&quot;, &quot;git log&quot;) and then answered. The &quot;algorithm&quot; search matched zero summaries but five folded originals, so the search covers raw history too, not only the summaries.</li><li>Full history, direct: <strong>6/8</strong>, with different misses: it got the line number wrong (195 instead of 126) and named the wrong CLI test file.</li></ul><p>So the folded agent matched the full-history agent, but only by paying for retrieval: six upstream rounds and 89,958 input tokens (72,497 of them cached), against one 39K call. Three practical notes. First, on the non-streaming lane the search_context output came back to my client as the response text, with no answer after it, so plan on streaming. Second, the search receipts (&quot;Found 2 block(s) for ...&quot;) show up in the client-visible text stream, as do fold receipts. Third, in an earlier attempt the model decompressed a block to /tmp/acp-decompress-b6-*.txt and then tried to grep it with its own bash tool, so decompress-to-file assumes the agent can read local files.</p><h3>What it cost</h3><p>At Gemini 3.5 Flash-Lite list prices ($0.30/M input, $0.03/M cached, $2.50/M output):</p><ul><li><strong>bili:</strong> 77% of input tokens were cache hits, about <strong>$0.067</strong> for the session</li><li><strong>Uncompressed, assuming perfect prefix caching:</strong> about <strong>$0.039</strong> (estimate)</li><li><strong>With no caching on either side:</strong> bili $0.205 vs $0.234</li></ul><p>On a 39K-token session, compression <strong>lost</strong> on cost once caching is counted. Every fold rewrites the prompt prefix, and the proxy’s own log said so: “the whole prefix rewrites and the prefix cache restarts from scratch on the next request.” On top of that, the ~4K-token tool overhead is charged on every call. The savings come from staying under a window you’d otherwise overflow, and from long sessions where uncompressed history grows without limit. My session was neither, so treat 12.5% as a lower bound for a short session, not as evidence for the “5×” claim.</p><h3>What I’d tell my team</h3><p>Where it’s useful:</p><ul><li>Long-running agent sessions on small-window or cheap models, where hitting the window is the real failure</li><li>Teams that want folds they can audit and undo; bili export &lt;session&gt; dumps the folded view and every block summary</li><li>Setups that already run a local proxy; median overhead was 10 ms</li></ul><p>Where I’d hold off:</p><ul><li>Short tasks under ~50K tokens. With default settings (50K growth gate, nudge at 75% of the window) my session would never have folded, and the tool overhead would only have added cost.</li><li>Gemini behind an OpenAI-compatible client until the thought_signature drop is fixed</li><li>Anything user-facing without reading the license. It’s MIT plus a required attribution line on your home page, docs, or About page.</li><li>Pinned production environments. Hundreds of commits a week plus self-update on by default means you need ACP_AUTO_UPDATE=0 and a pinned version.</li></ul><p>The design idea is good: let the model decide what’s stale, keep the originals, and make folds reversible. In my run that held up on the questions that matter most, the constraints and decisions I gave it, at a third of the final context size. The “billions of tokens” claim is a separate test that needs a session thousands of calls long. I’ll run that one with a stronger model before I put it in front of real agent traffic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*emBvVczfj-Cnv3YTokUvtw.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7cdbac4bf6b8" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/billion-context-hands-on-compressing-coding-agent-history-to-fit-a-small-context-window-7cdbac4bf6b8">billion-context Hands-On: Compressing Coding-Agent History to Fit a Small Context Window</a> was originally published in <a href="https://ai.plainenglish.io">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gemini Omni 1.2 Flash Generates and Edits 15-Second Videos in the Gemini API for $0.07 a Second]]></title>
            <link>https://medium.com/@ishank.iandroid/gemini-omni-1-2-flash-generates-and-edits-15-second-videos-in-the-gemini-api-for-0-07-a-second-5172a9bbea96?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/5172a9bbea96</guid>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[video-generation]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Sat, 07 Nov 2026 04:31:03 GMT</pubDate>
            <atom:updated>2026-11-07T04:31:03.370Z</atom:updated>
            <content:encoded><![CDATA[<p>I wired up Google’s new GA video model in Python: a 404 bug in google-genai 3.0.0, a hard 0-token Free Tier wall, measured input token costs, and what changes on December 3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gDAkoC59qbAkPLGxcwx-bg.jpeg" /></figure><p>On November 5, 2026, Google made <strong>Gemini Omni 1.2 Flash</strong> (gemini-omni-1.2-flash) generally available across the Gemini API and Google Cloud. Two weeks after shutting down Veo 3.1 Preview in the developer API on October 22, Google is positioning Omni Flash as its default model for generating, editing and extending video with synchronized audio.</p><p>Three headline changes stand out:</p><ul><li><strong>Native 15-second clips</strong>, up from 10 seconds in Omni 1.1 Flash, selectable in one-second steps from 3 to 15 seconds.</li><li><strong>A 30% price cut</strong> on video output tokens, dropping 720p generation from about $0.10 per second to roughly <strong>$0.07 per second</strong> ($12.00 per million video output tokens, down from $17.50).</li><li><strong>Multi-turn conversational editing and stronger reference adherence</strong>, accepting up to 6 reference images and refining clips across turns without starting over.</li></ul><p>One date in the release notes matters if you already ship on Omni 1.1: <strong>on December 3, 2026, every API call to </strong><strong>gemini-omni-1.1-flash will automatically route to </strong><strong>gemini-omni-1.2-flash</strong>.</p><p>I spent Friday evening wiring gemini-omni-1.2-flash up on Linux with Python 3.12, google-genai 3.0.0, raw REST calls and ffprobe. Because my AI Studio key is on the Free Tier, where Omni 1.2 enforces a hard zero-quota wall, I couldn&#39;t render paid clips from my own prompts. Everything else on the wire was testable right now, and the first bug hit in 90 milliseconds.</p><h3>What I tested and where the Free Tier stops</h3><p>My setup and test boundary:</p><ul><li><strong>Environment:</strong> Linux x86_64, Python 3.12.14, google-genai 3.0.0, ffprobe and a Free Tier AI Studio GEMINI_API_KEY.</li><li><strong>What ran live ($0.00 spend):</strong> model discovery (models.list, models.get), input token accounting (countTokens) across text, reference images, uploaded 720p video and WAV audio, Files API upload latency, pre-quota schema validation on /v1beta/interactions, model-routing checks, stateful chaining and container inspection of Google&#39;s raw omni_*.mp4 API samples.</li><li><strong>What blocked (</strong><strong>HTTP 429):</strong> all four video generation calls I sent (5s 360p draft, 6s 720p alley tracking shot, 15s 720p timecoded clip via URI delivery, and 6s 2-image reference shot) returned Rate limit exceeded for model gemini-omni-1.2-flash (limit: 0 input tokens per minute on Free Tier) in <strong>0.41 to 0.48 seconds</strong>.</li></ul><p>Omni 1.2 Flash has <strong>zero Free Tier quota</strong>: 0 RPM and 0 RPD. Even a 3-second 360p draft (~$0.07) requires Paid Tier 1 billing enabled (8 RPM, 40 RPD, doubled from Omni 1.1). If you want to try 15-second clips manually without API billing, Google Vids includes them at no extra cost on Google and Workspace accounts.</p><p>Because I didn’t run on a paid project, I haven’t measured live video render times or visual drift on my own prompts. Google also published zero benchmarks comparing 1.2 against 1.1, so treat quality claims as untested until you run your own prompts.</p><h3>Bug #1: Google’s Python quickstart 404s out of the box</h3><p>Gemini Omni models run on Google’s stateful <strong>Interactions API</strong> (/v1beta/interactions), not generateContent, even though GET /v1beta/models/gemini-omni-1.2-flash (0.17s) still lists [&quot;generateContent&quot;, &quot;countTokens&quot;] under supportedGenerationMethods.</p><p>If you install google-genai 3.0.0 and run the quickstart from Google&#39;s Omni docs:</p><pre>from google import genai<br><br>client = genai.Client()<br>interaction = client.interactions.create(<br>    model=&quot;gemini-omni-1.2-flash&quot;,<br>    input=&quot;A marble rolling fast on a chain reaction style track, continuous smooth shot.&quot;,<br>    response_format={&quot;type&quot;: &quot;video&quot;, &quot;duration&quot;: &quot;5s&quot;, &quot;resolution&quot;: &quot;360p&quot;},<br>)</pre><p>It fails in <strong>0.09 to 0.46 seconds</strong> with a 404:</p><pre>NotFoundError: Error code: 404 - {&#39;error&#39;: {&#39;message&#39;:<br>&quot;Model &#39;gemini-omni-1.2-flash&#39; not found. Did you mean &#39;gemini-omni-1.1-flash&#39;?&quot;,<br>&#39;code&#39;: &#39;not_found&#39;}}</pre><p>&quot;Did you mean &#39;gemini-omni-1.1-flash&#39;?&quot; is a red herring: gemini-omni-1.1-flash 404s on that path too (0.20s).</p><p>Inspecting client.interactions.sdk_configuration shows why. In google-genai 3.0.0, the Interactions sub-client defaults to Globals(api_version=&#39;v1&#39;) (POST /v1/interactions), where neither Omni 1.1 nor 1.2 exists. Both live only on v1beta. (Importing google.genai on Python 3.12 also emits two SyntaxWarning escape warnings in types.py.)</p><p>Passing api_version=&quot;v1beta&quot; fixes the 404:</p><pre>interaction = client.interactions.create(<br>    api_version=&quot;v1beta&quot;,<br>    model=&quot;gemini-omni-1.2-flash&quot;,<br>    input=&quot;A marble rolling fast on a chain reaction style track, continuous smooth shot.&quot;,<br>    response_format={&quot;type&quot;: &quot;video&quot;, &quot;duration&quot;: &quot;5s&quot;, &quot;resolution&quot;: &quot;360p&quot;, &quot;delivery&quot;: &quot;uri&quot;},<br>)</pre><p>Right behind that 404 sits a second SDK catch. When client.interactions.create(api_version=&quot;v1beta&quot;, ...) hits a 429, even a permanent limit: 0 Free Tier error, its internal retry loop retries silently for <strong>over 90 seconds per call</strong> before raising RateLimitError. Two calls hung my script for over three minutes before I killed it. Cap retries in CI so bad keys fail fast.</p><h3>What I measured on the live API</h3><p>Even without paid video output quota, several parts of the gemini-omni-1.2-flash stack respond on a Free Tier key.</p><h3>1. Real input token counts (countTokens)</h3><p>Omni 1.2 Flash bills inputs at <strong>$1.50 per million tokens</strong> across text, images and video. Calling POST /v1beta/models/gemini-omni-1.2-flash:countTokens returned HTTP 200 in <strong>0.18 to 0.52 seconds</strong>:</p><ul><li><strong>14-word text prompt:</strong> 16 TEXT tokens (0.19s), costing <strong>$0.000024</strong>.</li><li><strong>Two reference PNGs</strong> (1280x720 mug card and 800x800 robot sheet) plus 11-token prompt: 527 tokens (11 TEXT + 516 IMAGE, or <strong>258 tokens per image</strong>, 0.18s), costing <strong>$0.00079</strong>.</li><li><strong>Six reference PNGs</strong> (the maximum allowed) plus 16-token prompt: 1,564 tokens (16 TEXT + 1,548 IMAGE, 0.20s), costing <strong>$0.00235</strong>.</li><li><strong>Uploaded 5.0-second 720p 24fps MP4</strong> with AAC audio plus edit prompt: Files API upload took <strong>1.13s</strong> and reached ACTIVE in <strong>3.49s</strong> (2 polls). countTokens returned 1,488 tokens in 0.30s: 13 TEXT + 1,315 VIDEO (<strong>263 video tokens/s</strong>) + 160 AUDIO (<strong>32 audio tokens/s</strong>), or <strong>295 media tokens/s</strong>. That is <strong>$0.00223</strong> for 5s, or about <strong>$0.0066</strong> for a 15s input clip.</li></ul><p>Input tokens are rounding error. Six reference images or a 15-second source video cost under two-thirds of a cent; over 99.5% of your bill is video output.</p><h3>2. Schema checks that fail fast (HTTP 400)</h3><p>POST /v1beta/interactions validates its request schema <em>before</em> checking quota, returning HTTP 400 in <strong>0.18 to 0.21 seconds</strong>:</p><ul><li><strong>Duration needs </strong><strong>&quot;s&quot;:</strong> &quot;duration&quot;: 5 fails; pass &quot;3s&quot; through &quot;15s&quot; (defaults to &quot;10s&quot;).</li><li><strong>Four resolutions:</strong> &quot;480p&quot; fails, listing &#39;360p&#39;, &#39;720p&#39;, &#39;1080p&#39;, &#39;4k&#39;.</li><li><strong>Two aspect ratios:</strong> &quot;1:1&quot; fails, listing &#39;16:9&#39; (default) and &#39;9:16&#39;.</li><li><strong>Five </strong><strong>task enums:</strong> &quot;video_upscale&quot; fails, listing &#39;text_to_video&#39;, &#39;image_to_video&#39;, &#39;reference_to_video&#39;, &#39;edit&#39;, &#39;extend&#39;. Google advises omitting task unless prompting picks the wrong mode, and deprecates &lt;FIRST_FRAME&gt; and &lt;IMAGE_REF_0&gt; control tags.</li></ul><h3>3. Stateful chaining and silent model routing</h3><p>To test multi-turn state lookup, I created a Turn 1 interaction on /v1beta/interactions with gemini-3-flash-preview (HTTP 200 in <strong>3.22s</strong>, returning v1_Chds...), then passed that previous_interaction_id into gemini-omni-1.2-flash on Turn 2. A fake ID failed pre-quota with HTTP 400 in 0.32s, while the real v1_... ID passed state lookup across models and hit the Omni 1.2 quota gate in 0.88s. Avoid setting store=false on Turn 1 if you plan to edit later, since disabling storage breaks previous_interaction_id.</p><p>Sending &quot;model&quot;: &quot;gemini-omni-flash-preview&quot; (shut down earlier) returned a 429 naming gemini-omni-1.1-flash (0.52s), confirming the gateway already aliases retired Omni IDs server-side ahead of the December 3 1.1 to 1.2 switch.</p><h3>Output cost math and the 4 MB payload trap</h3><p>Google bills Omni 1.2 Flash video output at <strong>$12.00 per million tokens</strong> (text and thinking tokens at $9.00 per million):</p><ul><li><strong>360p:</strong> 1,931 tokens/s, or <strong>$0.023/s</strong> ($0.12 for 5s, $0.35 for 15s)</li><li><strong>720p (default):</strong> 5,792 tokens/s, or <strong>$0.070/s</strong> ($0.42 for 6s, $0.70 for 10s, $1.04 for 15s)</li><li><strong>1080p (upsampled):</strong> 8,688 tokens/s, or <strong>$0.104/s</strong> ($0.52 for 5s, $1.56 for 15s)</li><li><strong>4K (upsampled):</strong> 17,376 tokens/s, or <strong>$0.209/s</strong> ($1.04 for 5s, $3.13 for 15s)</li></ul><p><strong>Nine 15-second drafts at 360p ($3.13) cost the same as one 15-second render at 4K ($3.13).</strong> Draft at 360p or 720p, refine via previous_interaction_id, and render 1080p or 4K last. Batch API is unsupported.</p><p>Inspecting four raw omni_*.mp4 samples from Google&#39;s API docs (encoder: Google) and three launch-blog demos with ffprobe showed why delivery settings matter:</p><ul><li><strong>Container specs:</strong> raw outputs are H.264 (yuv420p) at <strong>24 fps</strong> with <strong>48 kHz stereo AAC</strong> at <strong>2.37 to 4.54 Mbps</strong>.</li><li><strong>The 4 MB inline limit:</strong> inline base64 responses cap out at <strong>4 MB</strong>. Three of Google’s four raw 720p doc samples (5.42 MB at 10s, 5.65 MB at 20s, 5.78 MB at 20s) exceed 4 MB; only one 10s clip (3.08 MB) fits. Always pass &quot;delivery&quot;: &quot;uri&quot; for 10-to-15-second 720p+ clips and poll GET /v1beta/files/{id} until ACTIVE.</li><li><strong>Provenance (</strong><strong>SynthID + </strong><strong>C2PA):</strong> alongside invisible SynthID watermarking, all four raw API MP4s carry a <strong>C2PA JUMBF manifest</strong> (jumdc2pa) signed by Google Media Processing Services under Google C2PA Root CA G3. The FFmpeg-reencoded blog demos (Lavf62.12.102) lost their C2PA box, while SynthID survives in the frames.</li><li><strong>One doc slip:</strong> omni_misty_mountains_1080p.mp4, embedded right under Google&#39;s resolution: &quot;1080p&quot; example, is actually 1280x720 in ffprobe.</li></ul><h3>Four doc contradictions and what to fix before December 3</h3><p>Reading Google’s Keyword post, Cloud blog, Gemini API docs and Cloud Agent Platform model card side by side surfaced four mismatches:</p><ul><li><strong>15 seconds does not apply to every mode:</strong> text, single-image, reference-to-video and edits reach <strong>15 seconds</strong> (with 11-to-15s clips limited to <strong>1</strong> 3s reference video instead of 3), but <strong>first-and-last-frame interpolation stays capped at 3 to 10 seconds</strong>. Passing &quot;15s&quot; with two keyframes returns an invalid argument error.</li><li><strong>Audio references are gated:</strong> the DeepMind blog pitches 1-second voice cues, the Cloud blog says audio references need <strong>allowlisting</strong>, and the Gemini API docs mark audio uploads <strong>unsupported</strong> — even though countTokens already tokenizes WAV input at 32.5 AUDIO tokens/s.</li><li><strong>Extended length (</strong><strong>30s vs </strong><strong>40s):</strong> Gemini API docs cap multi-turn extension at <strong>30 seconds total</strong>, while the Cloud model card implies up to <strong>40 seconds</strong>.</li><li><strong>Output token ceiling (</strong><strong>65,536 vs </strong><strong>260,640):</strong> models.get lists outputTokenLimit: 65,536, yet 15s at 720p needs 86,880 tokens and 15s at 4K needs 260,640, matching the Cloud card&#39;s 260,640 limit.</li></ul><p>Google also notes rough spots on <strong>fast sports action, rapid script changes and complex physics</strong>, no support for system instructions, temperature, top_p or negative prompt fields, and EEA/UK/Switzerland blocks on editing uploaded video.</p><p>If you run gemini-omni-1.1-flash today, four updates get you ahead of the <strong>December 3, 2026</strong> automatic reroute:</p><ol><li>Pass <strong>api_version=&quot;v1beta&quot;</strong> in google-genai 3.0.0 and cap retries so quota errors fail fast.</li><li>Default to <strong>&quot;delivery&quot;: &quot;uri&quot;</strong> on 10-to-15-second clips at 720p+ to avoid the 4 MB inline cap.</li><li>Cap UI durations at <strong>3–15s</strong> for standard generation and <strong>3–10s</strong> for keyframe interpolation and extensions.</li><li>Strip deprecated &lt;FIRST_FRAME&gt; and &lt;IMAGE_REF_0&gt; tags and test gemini-omni-1.2-flash on staging now, banking the 30% price drop on your own schedule.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xBgKPx5ap7jhw-KdfbvNCg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5172a9bbea96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Aura Autopilot Review: An Open-Source AI Agent That Playtests Your Godot, Unity and Unreal Games]]></title>
            <link>https://medium.com/@ishank.iandroid/aura-autopilot-review-an-open-source-ai-agent-that-playtests-your-godot-unity-and-unreal-games-081fb9246738?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/081fb9246738</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[game-development]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[godot]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 12:31:01 GMT</pubDate>
            <atom:updated>2026-11-05T12:31:01.579Z</atom:updated>
            <content:encoded><![CDATA[<h4>I hid 4 bugs in a Godot game. It played with real keys and found all 4 in 4 minutes, plus one I didn’t know about. Then I compared it to a plain LLM.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P0G6r6hEj7rBwmr2XrzVAw.png" /></figure><p>Every game dev I know has the same quiet dread: the bug a player finds in the first five minutes that nobody on the team ever hit.</p><p>So when Aura open-sourced <strong>Autopilot</strong>, an Apache-2.0 agent that plays your game to find bugs, I cleared an evening. The pitch is bold: about 43% more accurate and roughly 2x faster than other AI agents at real playtesting tasks.</p><p>I wanted to see it with my own eyes. Here’s what I ran, what it found, and what it cost.</p><h3>What Aura Autopilot actually is</h3><p>Autopilot is not one big model clicking around. It’s four parts working together:</p><ul><li><strong>An engine plugin</strong> for Unreal Engine 5 (5.4–5.8), Unity 6 or Godot 4.7+. It reads the game’s state and presses real keys and mouse buttons, the way a player would.</li><li><strong>An agent service</strong>, a Python program on your machine. It talks to the plugin over a local WebSocket, runs model-written code in a sandbox and keeps memory for each game.</li><li><strong>A coding agent</strong> (OpenCode by default, or Claude Code or Codex). It reads your source and writes a small “harness”: the list of moves that make sense for <em>your</em> game, like “dodge left” or “open inventory”.</li><li><strong>A decision model</strong> called Jev, from TypeSafe. During play it picks one of those moves roughly every 0.25 seconds. Fast “reflexes” like dodging run as plain code so they don’t wait for the model.</li></ul><p>The clever bit is the harness. Aura’s team found that giving a model raw keys (“press W”, “click”) plays badly, while game-specific moves play well. So instead of hand-writing those moves for each game, the coding agent writes them for you.</p><p>Then it remembers. Each project gets a memory folder with a copy of your code, notes, harness code that worked, and past reports. Later runs skip the research and start from a harness that already works.</p><p>You give it one of four kinds of prompts:</p><ul><li><strong>Explore:</strong> “Explore the crafting system in Lvl_Main.”</li><li><strong>Test:</strong> “Go through this level and tell me everything that looks wrong.”</li><li><strong>Verify:</strong> “Verify that the tree shakes when you hit it with an axe.”</li><li><strong>Goal:</strong> “Kill the boss while keeping your health above 20%.”</li></ul><h3>Where the 43% and 2x numbers come from</h3><p>I always read the method before I quote a number. Here it is, from Aura’s own blog:</p><ul><li>They tested on <strong>Easy Survival RPG V5</strong> in Unreal Engine 5.8.</li><li>That was <strong>7 tasks</strong>: 5 verify tasks and 2 goal tasks, like “craft a stone hatchet and use it to kill a zombie”.</li><li>Each tool got <strong>one cold attempt</strong> with empty memory, on the same Windows machine, all using Claude Opus 5.5 on medium effort.</li><li><strong>Accuracy:</strong> Autopilot 7/7, Claude Code 5/7, and an unnamed “leading Unreal/Unity AI agent” 4/7.</li><li><strong>Mean time:</strong> Autopilot 262s, Claude Code 437s, competitor 508s.</li><li>Cheating (teleporting, spawning items, god mode) counted as a fail. Both of Claude Code’s goal runs cheated.</li></ul><p>So the “43% more accurate, ~2x faster” headline is against that unnamed competitor: 100% vs 57%, and 262s vs 508s. Against Claude Code it’s 100% vs 71%, and about 1.7x faster.</p><p>That’s a real result, but a small one: seven tasks, one game, one attempt each, and the vendor ran it. I’d treat it as a promising signal, not a law.</p><h3>My test setup</h3><p>My setup:</p><ul><li><strong>Machine:</strong> a Linux box with a real X display. Officially Autopilot ships for Windows and macOS, but the source runs anywhere Python 3.12 does, and the Godot addon is pure GDScript.</li><li><strong>Engine:</strong> Godot 4.7.2, the Linux build.</li><li><strong>Game:</strong> “Dodge the Creeps”, the classic open-source Godot demo.</li><li><strong>The trap:</strong> I planted 4 bugs a player would notice:</li><li>The score keeps ticking after you die.</li><li>The Start button never comes back after Game Over, so you’re stuck.</li><li>The player can walk 240px off the right edge of the screen and hide there.</li><li>Moving diagonally is about 1.4x faster than moving straight.</li><li>I removed the git history so no tool could just read my diff.</li></ul><p>Install from source was smooth:</p><ul><li>Cloned the repo, then uv venv and uv pip install -e &quot;.[dev]&quot;. Under a minute.</li><li>Ran the test suite, which uses a simulated game with no keys and no network: <strong>492 passed, 5 skipped in 4m50s.</strong></li><li>Copied addons/autopilot into the game and enabled the plugin.</li><li>Signed in to Aura, started the agent service with autopilot serve --aura, then launched the game with --playtest=explore --playtest-minutes=5.</li></ul><h3>What happened when I ran it</h3><p>I gave it the laziest prompt possible: explore the game for bugs, five minutes of play. No hints about what I’d planted.</p><p>Here’s how the run went, start to finish:</p><ul><li>The game window opened and the plugin connected to the agent service.</li><li>The coding agent read my 10 project files and wrote a harness: moves like “start game”, “hold right against the edge”, “move diagonally”, “steer away from the nearest creeps” and “wait on the game-over screen”.</li><li>Before playing, it ran a quick probe to check that each move actually worked.</li><li>Then Jev took over and played one 150-second episode with real key presses. I watched it start a game, dodge, push into the right wall and die around 29 seconds in.</li><li>After the death it kept trying to start a new game. It tried 10 times, and nothing happened.</li><li>It wrote a report, saved notes to the game’s memory and exited cleanly.</li></ul><p>Total wall time: <strong>about 4 minutes</strong> from launch to exit (222 seconds of agent time). It also saved a video recording and a frame-by-frame flipbook of the episode.</p><h3>What it found</h3><p>All <strong>4 of 4</strong> planted bugs, each one backed by numbers from the run:</p><ul><li><strong>Walking off screen:</strong> the player reached x=720 on a 480px-wide screen. It saw this five separate times while holding right.</li><li><strong>Score after death:</strong> the score went from 16 at death to 146 two minutes later, with no game running.</li><li><strong>Stuck after Game Over:</strong> the Start button stayed hidden for 120 seconds, and Enter did nothing in 10 tries. It called it a softlock, which is exactly right.</li><li><strong>Fast diagonal:</strong> it measured about 580 px/s diagonally against 400 px/s straight, so roughly 41% faster.</li></ul><p>It also found a fifth issue I hadn’t planted: the hidden player moves when you hold a key on the title screen, before any game starts. That’s a real bug in the demo. It flagged one more thing from code reading only (D-pad mappings that look swapped) and was honest that it hadn’t checked that one in play.</p><p>No false alarms. Every claim pointed to a moment in the trace.</p><h3>What it cost</h3><p>The run went through my Aura plan, and the report logged <strong>about $1.11</strong> of model usage.</p><p>One surprise: I’d set the coding agent to OpenCode with Gemini, but in Aura mode the report shows Aura ran it on Claude Opus 5.5 for me. So on Aura you don’t need a Claude account to get Claude doing the code-reading part. If you bring your own keys instead, you choose the model and pay that provider plus TypeSafe for Jev.</p><p>A few things to know before you try it:</p><ul><li>I couldn’t test the autopilot connect claude or connect codex path. That hooks Autopilot into your own Claude Code or Codex, and I don&#39;t use either plan.</li><li>Run sharing is <strong>on by default</strong>. I turned it off with AUTOPILOT_SHARE=0 and --playtest-share=0. Do the same if your game is private.</li><li>Without an Aura account or a TypeSafe key, it stops about 20 seconds in. Jev is the player, and there’s no swapping it out.</li><li>A small Linux gotcha: if something already uses port 8765, start the service with --port and pass --playtest-url to the game. That cost me two 2-minute timeouts.</li></ul><h3>The baseline: just ask an LLM to read the code</h3><p>For comparison, I gave Gemini the full source of the bugged game in one prompt: “play it in your head and list every bug a player would hit.”</p><p><strong>gemini-3-flash-preview</strong> (31.6s, about 6.7K tokens in):</p><ul><li>Found <strong>all 4</strong> planted bugs, with the right file and line for each.</li><li>Listed 5 extra issues. A couple were fair, like the sprite staying upside-down after moving down and then sideways. The rest were guesses or design choices, like the trail particles always emitting.</li></ul><p><strong>gemini-3.1-flash-lite</strong> (25.6s):</p><ul><li>Found <strong>1 of 4</strong>: the score that keeps counting.</li><li>Missed the stuck Start button, the off-screen hiding spot and the fast diagonal.</li><li>Padded the list with timing “races” that don’t really happen in play.</li></ul><p>Both runs were free on the Gemini free tier.</p><h3>Autopilot vs a plain LLM</h3><p>On my little test, here’s how they compare:</p><ul><li><strong>Bugs found:</strong> Autopilot 4/4 plus one real extra. Gemini Flash 4/4 plus five extras, mostly noise. Flash-Lite 1/4.</li><li><strong>Time:</strong> Autopilot about 4 minutes. Gemini about 30 seconds.</li><li><strong>Cost:</strong> Autopilot about $1.11. Gemini $0 on the free tier.</li><li><strong>Evidence:</strong> Autopilot gave me measured numbers, a trace and a video. Gemini gave me line numbers and a confident guess.</li></ul><p>So on speed and cost, the plain LLM wins easily, and that’s fair: my bugs were all visible in the code. If that’s your kind of bug, a good model reading your source is hard to beat.</p><p>But the difference I felt was trust. Gemini <em>said</em> I could hide off-screen. Autopilot walked there and showed me x=720. Gemini mixed real bugs with design choices, and I had to sort them myself. Autopilot’s list had nothing to throw away, and it found a real bug I didn’t know the demo had.</p><p>And code-reading has a hard ceiling. It can’t catch bugs that only show up in play, like physics, timing, level layout or a door you can clip through. That’s the space Autopilot is built for, and my test didn’t even reach it.</p><p>My result doesn’t confirm Aura’s “43% more accurate, 2x faster” claim, and it doesn’t refute it. That was against other playing agents on 7 Unreal tasks. What I can say is that on a cold run, with a lazy prompt, on Linux, which isn’t even officially supported, it caught everything I hid.</p><h3>Should you try it?</h3><p>My take:</p><ul><li><strong>Yes, if you ship on Unreal, Unity or Godot</strong> and have repeatable QA checks (“can I still open the inventory after a new game?”). Verify and goal prompts with pass/fail exit codes fit straight into CI.</li><li><strong>Use a plain LLM first for cheap code review.</strong> It’s faster and free. Bring Autopilot in for what code can’t prove.</li><li><strong>Expect it to get better per game.</strong> The memory design is the real idea here. My next run starts from a harness that already works.</li><li><strong>Read the benchmark as a start.</strong> Seven tasks is a demo, not a verdict. Run it on your own game.</li></ul><p>The repo is at github.com/ramenvr/autopilot. If you try it on your own project, tell me what it finds. I’d love to hear about a bug that only shows up in play.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZUDKMsjP4mFi92Unet1bQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=081fb9246738" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[HackerShelf: The Books Hacker News Recommends, Rebuilt from Karpathy’s HackerTube with Cursor]]></title>
            <link>https://ai.plainenglish.io/hackershelf-the-books-hacker-news-recommends-rebuilt-from-karpathys-hackertube-with-cursor-7a9cff210bc5?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a9cff210bc5</guid>
            <category><![CDATA[andrej-karpathy]]></category>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[hacker-news]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai-coding]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Tue, 03 Nov 2026 04:31:03 GMT</pubDate>
            <atom:updated>2026-11-03T04:31:03.930Z</atom:updated>
            <content:encoded><![CDATA[<h4>Karpathy built HackerTube in about 2 hours with Claude Code. I rebuilt it for books with Cursor. Here’s my honest time, cost, and everything that broke.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pd-axQuAYSCLdbPbmWECzw.png" /><figcaption><a href="https://ishankdev.github.io/hackershelf/">https://ishankdev.github.io/hackershelf/</a></figcaption></figure><p>On 1 Nov, Andrej Karpathy shared HackerTube (https://hackertube.org/). He had Claude filter 43M Hacker News comments from 20 years down to 6,454 recommended videos, and he said the site took about 2 hours with Opus 5.5 in Claude Code. He also published the data as a public zip.</p><p>As a Lead SWE, my first question was whether I could pull off the same trick with a different topic and a different tool. I picked books, because HN comment threads are basically one long reading list. I picked Cursor, because that’s the tool I use every day.</p><p>The result is HackerShelf: <a href="https://ishankdev.github.io/hackershelf/">https://ishankdev.github.io/hackershelf/</a> <br>(<strong>Source code:</strong> <a href="https://github.com/IshankDev/hackershelf).">https://github.com/IshankDev/hackershelf).</a></p><h3>How Karpathy’s pipeline works</h3><p>His DATA.md lays out the method, and I copied it:</p><ul><li>Scan every HN comment from 2007 to 2026 in the open-index/hacker-news dataset for YouTube links. The comment needs at least 100 characters of its own text.</li><li>Keep a video only if at least 3 different people mentioned it in at least 3 different months.</li><li>Claude Sonnet 5.5 labels each mention with a stance, topics, a reason and a quote.</li><li>Sonnet writes a card for each video, then puts it into 1 to 3 categories.</li><li>Opus 5.5 in Claude Code runs the whole thing.</li></ul><p>He did a few steps by hand: a Google Cloud API key for YouTube metadata, a VPS and a domain.</p><h3>My version: HackerShelf</h3><p>Same idea, but for books:</p><ul><li><strong>Source:</strong> HN comments from Jan 2024 to Aug 2026 from open-index/hacker-news, 9,437,609 comments in total</li><li><strong>Pre-filter:</strong> DuckDB regex for book links (Amazon, Goodreads, Open Library, Bookshop, ISBNs, O’Reilly, Manning, PragProg, No Starch) and phrases like “must read”, “worth a read” and “book called”. This took about 9 seconds and left 11,519 candidates.</li><li><strong>Extraction:</strong> gemini-3.5-flash-lite in 288 batches of 40 found 6,076 book mentions.</li><li><strong>Resolve:</strong> Open Library matched 2,212 works. Using Karpathy’s rule (at least 3 people, at least 3 months), 151 books were left.</li><li><strong>Cards:</strong> one Gemini call per book, so 151 calls</li><li><strong>Site:</strong> a static site on GitHub Pages</li></ul><p>The final shelf has <strong>151 books, 723 mentions and 601 unique recommenders</strong>, sorted into 9 categories.</p><h3>Time: Karpathy’s 2 hours vs. mine</h3><p>Being honest, my total was longer than his.</p><ul><li><strong>Planning and data checks:</strong> 7:47 to 7:50 PM IST</li><li><strong>Cursor build</strong> (pipeline code, the full site and deploy setup): about 40 minutes, 7:56 to 8:37 PM IST</li><li><strong>Gemini extraction:</strong> about 20 minutes, about 8:50 to 9:11 PM IST</li><li><strong>Open Library resolve and filter:</strong> about 44 minutes, 9:24 to 10:08 PM IST, with no LLM. This was the slowest step.</li><li><strong>Cards:</strong> 509 seconds, 10:09 to 10:17 PM IST</li><li><strong>Site data:</strong> under 1 second, at 10:17 PM IST</li></ul><p>From start to finish that’s about <strong>2.5 hours of wall clock</strong> (7:47 to about 10:17 PM IST). That includes my own manual steps: creating the repo, giving Cursor GitHub access and turning on GitHub Pages.</p><p>Cursor’s part, writing the actual code, took about 40 minutes. Most of the remaining time went to calling APIs and waiting on rate limits.</p><h3>Cost</h3><ul><li><strong>Tokens:</strong> 1,968,177 in and 623,943 out</li><li><strong>What I paid:</strong> $0, on Gemini’s free tier. I hit 429s and handled them with backoff and retry.</li><li><strong>Paid-rate equivalent:</strong> at $0.30 in and $2.50 out per 1M tokens, about $0.59 + $1.56 = <strong>about $2.15</strong></li><li><strong>Cursor usage cost:</strong> I don’t know this number yet, so I won’t guess.</li></ul><h3>What went wrong</h3><ul><li><strong>The first Cursor run shipped sample data.</strong> The site worked, but it only had 9 placeholder books. The real data had to come from a separate run of the LLM stage.</li><li><strong>Too many candidates.</strong> The first broad filter matched about 392k comments that might mention a book, which was far too many for a free tier. Tighter link and phrase rules cut that down to 11,519.</li><li><strong>The model paraphrased.</strong> 470 of the extracted “quotes” weren’t actually word for word in the source comment. I didn’t trust them, so I blanked them.</li><li><strong>Open Library isn’t perfect.</strong> One top-10 entry resolved as “1984 (adaptation)” by Michael Dean instead of Orwell’s novel. Fuzzy title matching is the weak link here.</li></ul><h3>Does it hold up?</h3><p>I spot-checked 10 books and 83 mentions against the original HN comments. <strong>83 out of 83</strong> had the right author, and every quote was verbatim in its source comment. For example:</p><ul><li>“I recommend reading The Mom Test, it’s a great book about how to navigate this early validation stage”</li><li>“Robert Nystrom’s Crafting Interpreters [0] is a really fantastic book on the topic.”</li></ul><h3>The top 10 books Hacker News recommends (2024 to 2026)</h3><ol><li><strong>Designing Data-Intensive Applications</strong>, Martin Kleppmann: 13 recommenders, 14 mentions</li><li><strong>Amusing Ourselves to Death</strong>, Neil Postman: 11 recommenders, 11 mentions</li><li><strong>The Making of the Atomic Bomb</strong>, Richard Rhodes: 11 recommenders, 12 mentions</li><li><strong>Permutation City</strong>, Greg Egan: 11 recommenders, 11 mentions</li><li><strong>The Mom Test</strong>, Rob Fitzpatrick: 10 recommenders, 10 mentions</li><li><strong>A Philosophy of Software Design</strong>, John Ousterhout: 9 recommenders, 11 mentions</li><li><strong>Blindsight</strong>, Peter Watts: 11 recommenders, 11 mentions</li><li><strong>Chip War</strong>, Chris Miller: 9 recommenders, 9 mentions</li><li><strong>1984</strong>: 10 recommenders, 10 mentions (Open Library mismatched this one, see above)</li><li><strong>Children of Time</strong>, Adrian Tchaikovsky: 11 recommenders, 11 mentions</li></ol><p>The ranking uses a score that weighs strong recommendations above passing mentions, which is why a book with 9 recommenders can rank above one with 11.</p><h3>What I took away</h3><ul><li>Karpathy’s recipe works beyond videos. The “3 people, 3 months” rule is a surprisingly good filter for noise.</li><li>An AI coding tool can write the code in under an hour. The time goes into data, rate limits and the manual steps.</li><li>Don’t trust LLM-extracted quotes until you check them against the source. 470 were quietly wrong.</li></ul><h3>Credits</h3><p>All credit for the idea and method goes to Andrej Karpathy and HackerTube. The comment data comes from open-index/hacker-news under ODC-By 1.0. The comments themselves are still their authors’ work, and every mention on HackerShelf links back to the original comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*msU01ZhqXaOfp4uY0v4JgA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a9cff210bc5" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/hackershelf-the-books-hacker-news-recommends-rebuilt-from-karpathys-hackertube-with-cursor-7a9cff210bc5">HackerShelf: The Books Hacker News Recommends, Rebuilt from Karpathy’s HackerTube with Cursor</a> was originally published in <a href="https://ai.plainenglish.io">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NVIDIA Nemotron 3.5 Bolt E4B: Running an Offline Voice Agent With Tool Calling on a CPU]]></title>
            <link>https://ai.plainenglish.io/nvidia-nemotron-3-5-bolt-e4b-running-an-offline-voice-agent-with-tool-calling-on-a-cpu-66ab95a5868a?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/66ab95a5868a</guid>
            <category><![CDATA[nemotron]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[calling-function]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Wed, 28 Oct 2026 04:31:04 GMT</pubDate>
            <atom:updated>2026-10-28T04:31:04.226Z</atom:updated>
            <content:encoded><![CDATA[<p>I gave NVIDIA’s new small model five tools, 60 prompts and no GPU. It picked the right tool every time, and a full spoken turn took about 4.5 seconds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FTygsyt7XFsVNH3iyVXrQg.png" /></figure><p>NVIDIA released Nemotron 3.5 Bolt E4B on October 23. The pitch is a small model tuned for agents: it calls tools, follows instructions, and is small enough for laptops, Jetsons and other edge hardware. The model card’s benchmarks are tested on H100s, RTX 5090s and DGX Spark. I wanted to see what happens without any of those.</p><p>So I took the official 4-bit GGUF, ran it on a plain 8-vCPU Xeon VM with 16 GB of RAM and no GPU, and built a small voice assistant around it. It listens to a WAV file, picks a tool, calls it, and speaks the answer back. Everything below is from my own runs. I’ve kept the raw CSVs for every number.</p><h3>What Bolt E4B actually is</h3><p>A few facts from the model card and the files themselves, because the name hides some of them:</p><ul><li><strong>Architecture:</strong> Gemma 4 E4B, post-trained by NVIDIA with reinforcement learning (GRPO via NeMo RL) for tool use, instruction following and multi-turn chat. About 8B total parameters and about 4B “effective” ones.</li><li><strong>License:</strong> OpenMDW-1.1 on NVIDIA’s release. The base Gemma 4 weights are Apache 2.0.</li><li><strong>Context:</strong> 64K tokens. <strong>Text in, text out.</strong> There’s no audio in the model, so a voice agent still needs separate speech-to-text and text-to-speech.</li><li><strong>Weights:</strong> the official repo is nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, with a single Q4_K_M GGUF. NVIDIA also ships a BF16 checkpoint for vLLM and a DFlash speculative-decoding drafter aimed at GPUs.</li><li><strong>Recommended serving:</strong> llama.cpp with --jinja, sampling at temperature 1.0 and top_p 0.95, and a chat-template switch for &quot;thinking&quot; on or off.</li></ul><p>The first surprise: the “4GB” file is <strong>6.18 GB on disk</strong>. I opened the GGUF and summed the tensors:</p><ul><li><strong>Per-layer embedding tables (mostly Q8_0)</strong>: 3.05 GB</li><li><strong>Transformer blocks (Q4_K + Q6_K)</strong>: 2.40 GB</li><li><strong>Token embeddings</strong>: 0.71 GB</li></ul><p>The per-layer embeddings are lookup tables: each token only reads a few rows. That’s what makes the “4GB” label fair at run time, as you’ll see in the memory section. You still need about 6.2 GB of disk space and download time, though.</p><h3>The setup</h3><ul><li><strong>Hardware:</strong> 8 vCPU Intel Xeon (AVX-512, VNNI), 16 GB RAM, no swap, no GPU.</li><li><strong>Runtime:</strong> llama.cpp CPU build b11913, using llama-server as an OpenAI-compatible endpoint.</li><li><strong>Speech-to-text:</strong> faster-whisper, base.en and small.en, int8 on CPU.</li><li><strong>Text-to-speech:</strong> Piper with the en_US-lessac-medium voice.</li></ul><p>The server command was nothing special:</p><pre>llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --alias nemotron-bolt \<br>  --host 127.0.0.1 --port 8000 -c 8192 -np 1 -t 8 --jinja --metrics</pre><p>The model loaded in 2.5 to 6 seconds. My first sanity check was 17 × 23, and it answered 391.</p><h3>Raw speed on 8 CPU cores</h3><p>llama-bench, 8 threads, 3 repetitions:</p><ul><li><strong>Prompt processing (512 tokens)</strong>: 194.6 Â± 32.8</li><li><strong>Generation (128 tokens)</strong>: 10.9 Â± 2.8</li></ul><p>About 11 tokens per second of generation is slow next to a GPU, but it’s enough for an assistant whose replies are one or two sentences. Prefill speed matters more than I expected. My system prompt plus five tool schemas came to about 500 tokens, and the very first request took 6.4 seconds before anything came back. After that, llama.cpp’s prompt cache kept the prefix warm, and only the new user message needed processing.</p><h3>Test 1: tool calling, 20 prompts × 3 runs</h3><p>I defined five mock tools, the kind a voice assistant would have:</p><ul><li>get_weather(city)</li><li>calculate(expression)</li><li>create_event(title, date, time, duration_min)</li><li>set_timer(minutes, label)</li><li>control_light(room, state, brightness)</li></ul><p>The system prompt gave the date (Tuesday 2026–10–27, IST) and asked for short spoken replies. I wrote 20 prompts: three per tool, plus five where the right move is not to call any tool. Those were a joke, a definition, a “thanks”, a trivia question, and “order me a pizza”, which no tool can do.</p><p>Each response was scored on two things: did it pick the right tool (or correctly pick none), and were the arguments exactly right?</p><p><strong>Thinking off</strong> (the mode I’d use for voice), three full runs:</p><ul><li><strong>Right tool, or correctly no tool</strong>: 60 / 60</li><li><strong>Exactly correct arguments</strong>: 57 / 60 (95%)</li><li><strong>Median time to first token</strong>: 0.87â€“1.15 s</li><li><strong>Median end-to-end per prompt</strong>: 2.1â€“3.6 s</li><li><strong>Median generated tokens</strong>: 20</li></ul><p>Some results I liked:</p><ul><li>“Set a timer for 45 seconds” became minutes: 0.75.</li><li>“An hour and a half” became duration_min: 90.</li><li>“Dim the bedroom to 30%” became state: &quot;on&quot;, brightness: 30. It didn&#39;t treat &quot;dim&quot; as &quot;off&quot;.</li><li>“18% GST on 4999” became 4999 * 0.18.</li><li>None of the five no-tool prompts triggered a tool. For the pizza request it said plainly that it couldn’t order food, rather than forcing the request into create_event.</li></ul><p>The miss was the same in all three runs. “Put sprint planning on my calendar for Friday at 11 for an hour” got the right title, time and duration, but the date was <strong>2026–11–06, a week too late</strong>. With today being Tuesday the 27th, “Friday” should be the 30th. Relative-date math is where a 4B model shows its size.</p><p><strong>Thinking on</strong>, one run of the same 20 prompts:</p><ul><li><strong>Right tool</strong>: 20 / 20</li><li><strong>Exactly correct arguments</strong>: 20 / 20</li><li><strong>Median end-to-end per prompt</strong>: 13.1 s</li><li><strong>Median generated tokens</strong>: 150</li></ul><p>With reasoning on, it got Friday right (2026–10–30). That cost about 350 characters of reasoning and 17.6 seconds for that one prompt. The first token still arrives fast, but it’s a reasoning token rather than an answer. Overall that’s roughly <strong>5x slower per turn</strong> for one extra correct argument out of 20. For a voice assistant on CPU, I’d keep thinking off and handle dates in code: pass the model a resolved calendar (“Friday = 2026–10–30”) in the prompt, or let the tool parse “Friday” itself.</p><p>One llama.cpp detail if you’re measuring this yourself: tool-call output streams in late, once the call has been parsed. So TTFT for a tool call is close to the time the whole call is ready. For plain-text replies, the first token showed up in 0.36 to 0.6 seconds.</p><h3>Test 2: a full offline voice turn</h3><p>Next I wired up the whole loop:</p><ol><li>Piper speaks the user’s request into a WAV file, as a stand-in for a microphone.</li><li>faster-whisper transcribes it.</li><li>Bolt picks a tool (thinking off).</li><li>The mock tool runs.</li><li>Bolt turns the tool result into a one-sentence reply.</li><li>Piper speaks the reply into another WAV.</li></ol><p>Six utterances, timed stage by stage. Medians:</p><ul><li><strong>Speech-to-text</strong>: Whisper base.en 0.35 s, Whisper small.en 0.85 s</li><li><strong>LLM: choose tool + args</strong>: Whisper base.en 2.39 s, Whisper small.en 1.85 s</li><li><strong>LLM: spoken reply from tool result</strong>: Whisper base.en 1.48 s, Whisper small.en 1.51 s</li><li><strong>Text-to-speech (Piper)</strong>: Whisper base.en 0.08 s, Whisper small.en 0.07 s</li><li><strong>Whole turn</strong>: Whisper base.en 4.08 s, Whisper small.en 4.58 s</li></ul><p>The LLM is nearly all of the turn time. Piper is basically free: it made a 4-second reply in 70 to 100 ms. Whisper is cheap too. That means most of the latency budget goes to two model calls: one to pick the tool and one to phrase the answer.</p><p>What the transcripts showed:</p><ul><li>base.en heard “Pune” as “Pooner”. Bolt still called get_weather(city=&quot;Pune&quot;) correctly.</li><li>base.en turned “Tell me a one-line joke about compilers” into “Tell me what you want to do.” Bolt sensibly listed its abilities, but the joke never happened. small.en got the sentence right, and Bolt told the joke.</li><li>“Book a dentist appointment tomorrow at four thirty p m” became date: 2026-10-28, time: 16:30, which is correct.</li><li>The 18%-of-4,999 answer was “about 900” in one run and “899.82” in another. At temperature 1.0 the wording varies, so if exact numbers matter, put them in your reply template.</li></ul><p>My takeaway: <strong>small.en is worth its extra half second</strong>, because recognition errors cost far more than ASR latency. Since the model handles fuzzy input well, I’d spend the speed budget on the ASR side.</p><h3>Memory: where the “4GB” name comes from</h3><p>On Linux, llama.cpp memory-maps the GGUF, so top can show a scary number. Right after load, RSS was about 9.6 GB, because the file pages had just been read. Once the system had been running for a while, the server process looked like this:</p><ul><li><strong>Anonymous (actually allocated) memory, 8K context, 1 slot</strong>: 3.8 GB</li><li><strong>RSS after the page cache settled</strong>: 4.1 GB</li><li><strong>Same server with 16K context and 4 slots (anonymous)</strong>: 3.7 GB</li><li><strong>Voice stack (Whisper small.en + Piper) in a separate process</strong>: 0.66 GB</li></ul><p>So about 4 GB of RAM for the model, plus about 0.7 GB for speech. The large embedding tables mostly stay on disk until a token needs a row. That matches the name, and it’s a key reason this model fits small devices. Just plan for the 6.2 GB download and the disk space.</p><h3>Limitations, honestly</h3><ul><li><strong>CPU only.</strong> I didn’t test the DFlash speculative drafter or any GPU, so these numbers are a floor, not a ceiling. NVIDIA’s GPU numbers will look very different.</li><li><strong>Small test set.</strong> 20 prompts × 3 runs is a smoke test, not BFCL. For a fuller picture, NVIDIA reports 67.3 on BFCL-v3 and 74.1 on ToolSandbox for this Q4_K_M build.</li><li><strong>Single tool call per turn.</strong> I didn’t test parallel or chained tool calls.</li><li><strong>Synthetic audio.</strong> My “microphone” was Piper’s own voice, which is clean speech. Real rooms, accents and background noise will make Whisper’s job harder.</li><li><strong>Dates.</strong> With thinking off, it misread “Friday” in every run. Resolve dates in code.</li><li><strong>Variance.</strong> My first run happened while other jobs were finishing, and its median turn was 3.6 s against 2.1 s in the clean third run. Run your own benchmarks more than once.</li></ul><h3>Would I use it?</h3><p>For a local assistant that turns speech into a handful of well-defined actions (timers, lights, calendar, lookups), yes. On a CPU-only VM it picked the right tool 60 out of 60 times, filled the arguments correctly 95% of the time, and completed a whole spoken turn in about four and a half seconds, with no network calls and about 4.7 GB of RAM including speech.</p><p>Where I’d be careful: anything that needs arithmetic on dates or times, long open-ended answers at 11 tokens per second, and noisy audio with a tiny Whisper model. Each of those has a simple fix: resolve dates in code, keep replies short, and use small.en.</p><p>Next I want to run the same loop on a Jetson-class board with the DFlash drafter and see how far that 4.5 seconds drops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jmif56GR6J0ry3iiBUddlg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66ab95a5868a" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/nvidia-nemotron-3-5-bolt-e4b-running-an-offline-voice-agent-with-tool-calling-on-a-cpu-66ab95a5868a">NVIDIA Nemotron 3.5 Bolt E4B: Running an Offline Voice Agent With Tool Calling on a CPU</a> was originally published in <a href="https://ai.plainenglish.io">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude Duet vs Grok Bot vs Muse vs Dots: Always-On AI Agents Compared (2026)]]></title>
            <link>https://ai.plainenglish.io/claude-duet-vs-grok-bot-vs-muse-vs-dots-always-on-ai-agents-compared-2026-887563c2d19e?source=rss-1e910a184252------2</link>
            <guid isPermaLink="false">https://medium.com/p/887563c2d19e</guid>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Ishank choudhary]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 04:31:02 GMT</pubDate>
            <atom:updated>2026-10-25T04:31:02.438Z</atom:updated>
            <content:encoded><![CDATA[<p>Four companies now sell an AI that keeps working after you close the tab. I tested the one I could get my hands on and read every official doc on the other three. Here’s what each one really costs and does.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*32kaX2suE--NJTsH70Gz4g.png" /></figure><p>Over six weeks, every big AI lab has shipped the same idea. You don’t get a chatbot that waits for you to type. You get an agent with its own computer that works on a schedule, connects to your apps, and asks before it does anything risky.</p><p>I’m a Lead Software Engineer, and I already run one of these every day for my writing workflow. So when Anthropic’s Claude Duet launch post passed 41,785 likes and 5.16M views in under a day, I wanted to know how the four actually compare. Hype reels weren’t going to answer that.</p><p><strong>How I tested.</strong> I live in India. I have ChatGPT Go, not Pro, and no Claude subscription. Meta’s Muse isn’t available in my country. So:</p><ul><li><strong>Grok Bot (xAI): tested hands-on.</strong> I’ve used it daily for weeks, and I ran the specific tests below for this post.</li><li><strong>Claude Duet (Anthropic): docs-based, not tested.</strong> I used Anthropic’s product page, launch article, help centre and demo.</li><li><strong>Muse (Meta): docs-based, not tested.</strong> I used Meta’s newsroom post and help centre.</li><li><strong>Dots (OpenAI): docs-based, not tested.</strong> I used OpenAI’s launch post and help centre.</li></ul><p>I checked every price on the official pages on 22 Oct 2026.</p><h3>What “always-on” actually means</h3><p>All four share the same basic design:</p><ol><li><strong>A cloud computer of its own.</strong> It has a browser and sometimes a shell, so the agent can work while your laptop is shut.</li><li><strong>Proactive runs.</strong> Scheduled briefs, monitors and “tell me when” triggers.</li><li><strong>Connectors.</strong> Mail, calendar, docs and code, all through your own accounts.</li><li><strong>An approval gate.</strong> It drafts, and you press the button for anything that sends, pays or deletes.</li></ol><p>They differ on price, where they’re available, and how much of that computer you actually get to use.</p><h3>The comparison</h3><h3>Price (official pages, checked 22 Oct 2026)</h3><ul><li><strong>Claude Duet:</strong> no extra cost on Claude <strong>Pro and Max</strong> during the research preview. It uses your plan’s usage limits, and its background checks count against them. <em>(claude.com, Duet help article)</em></li><li><strong>Grok Bot:</strong> <strong>from $30/mo</strong> (Premium, via X Subscription), with Super at $100 and Ultra at $200. At the 11 Aug 2026 beta launch there was no standalone price. Access came bundled with Cursor Premium Teams ($120/seat), Cursor Ultra ($200) and SuperGrok Heavy ($300), which is where the “$120–300” number in the reels comes from. <em>(x.ai/bot, x.ai news)</em></li><li><strong>Muse:</strong> <strong>free tier</strong> with a weekly limit (card required), <strong>Power $20/mo</strong> and <strong>Maximum $100/mo</strong>. <em>(Meta help centre)</em></li><li><strong>Dots:</strong> needs <strong>ChatGPT Pro</strong>, or the Business Premium/Enterprise beta. Free, Go and Plus are excluded. The cheapest Pro tier is $100, which is <strong>₹10,699/mo</strong> on the Indian App Store. <em>(openai.com, OpenAI help centre)</em></li></ul><h3>Availability</h3><ul><li><strong>Duet:</strong> US-only research preview with a waitlist. Not on Team or Enterprise plans. Works in the iOS/Android apps and on claude.ai.</li><li><strong>Grok Bot:</strong> generally available on paid plans, and I use it from India.</li><li><strong>Muse:</strong> US first, plus Canada and Mexico per Meta’s coverage. <strong>Not available in India.</strong> Works in the app, on the web and in WhatsApp.</li><li><strong>Dots:</strong> launched 29 Sep 2026 with a gradual rollout. Pro is excluded only in the EEA, Switzerland and the UK, so India qualifies on paper.</li></ul><h3>Its own computer</h3><ul><li><strong>Duet:</strong> a cloud VM with its own browser. You can’t yet watch or take over that browser, and it checks in every few hours.</li><li><strong>Grok Bot:</strong> a full persistent Linux box with a shell, a browser desktop you can watch or take over, and files that stay between sessions. <em>(Tested, results below.)</em></li><li><strong>Muse:</strong> the “Muse Secure VM”. Logins happen there, and Meta’s line is that <strong>Muse never sees your passwords</strong>.</li><li><strong>Dots:</strong> its own cloud computer, running GPT-6 Astra.</li></ul><h3>Approvals</h3><ul><li><strong>Duet:</strong> asks before it sends, pays, signs or deletes. It <strong>can make payments you approve</strong> (Anthropic’s example is a $640 permit fee), but it <strong>can’t move money between accounts</strong> or trade securities, even with approval.</li><li><strong>Grok Bot:</strong> outbound email and messages become a draft card, and nothing leaves until I press Send. Risky actions on its computer get an automatic safety review. <em>(Tested, see below.)</em></li><li><strong>Muse:</strong> payments go through one-time Link by Stripe cards.</li><li><strong>Dots:</strong> approval before consequential actions, per OpenAI’s help centre.</li></ul><h3>Connectors</h3><ul><li><strong>Duet:</strong> “more than 6,000 connectors and plugins”.</li><li><strong>Grok Bot:</strong> MCP connectors (Gmail, Notion, GitHub, X and more), plus anything its browser can reach.</li><li><strong>Muse:</strong> Meta apps plus partner integrations (no public count).</li><li><strong>Dots:</strong> 4,000+ apps.</li></ul><h3>Memory</h3><ul><li><strong>Duet:</strong> carries over your Claude memory, skills and apps.</li><li><strong>Grok Bot:</strong> persistent memory and saved skills across chats, plus the files on its box.</li><li><strong>Muse:</strong> personal context across Meta apps (details are thin in the docs).</li><li><strong>Dots:</strong> builds on your ChatGPT account (memory specifics not spelled out in the launch docs).</li></ul><h3>What the reels got wrong</h3><ul><li><strong>“Duet never sees your passwords.”</strong> That’s <strong>Meta’s claim about Muse</strong>. Anthropic says logins run in your own cloud computer “so sensitive details stay private”, which is close but not the same promise.</li><li><strong>“Grok Bot launched at $120–300.”</strong> Those were <strong>bundled plans</strong>, not a standalone price. It now starts at $30/mo.</li><li><strong>“Duet can’t move money.”</strong> Half right. It can’t move money between accounts, but it <strong>can</strong> make a payment you approve.</li><li>Everything else I checked about Duet holds up, including Pro/Max inclusion, the morning brief, meeting prep, follow-ups, 6,000+ connectors and the US-only waitlist. The A/B-test proposal only appears in the demo.</li></ul><h3>Grok Bot, hands-on</h3><p>Every test was read-only or sandboxed. Nothing was sent, posted or deleted.</p><p><strong>Its own computer</strong> — Installed a Python CLI (pip install --user cowsay) in <strong>0.8 s</strong>, then ran it again in a later session. <strong>It persisted.</strong> - Started a <strong>12-minute background job</strong> and walked away. Result: all 12 one-minute steps ran and the job finished on its own (started 09:52, done 10:11 IST). Honest caveat: the box sat idle for about 8 minutes mid-run, so the wall-clock time stretched from 12 to about 19 minutes. <strong>Pass, but don&#39;t count on exact timing.</strong> - Produced a CSV of these results and a PNG chart on the box, ready to hand back in chat. <strong>Pass.</strong></p><p><strong>Connectors (read-only)</strong> — <strong>Gmail:</strong> searched “Medium” over the last 7 days and got back about <strong>201 matching emails</strong> (25 per page) in about 4 s. <strong>Correct.</strong> — <strong>Notion:</strong> searched “Medium” and found my “Medium Topics” page at the top in about 4 s. <strong>Correct.</strong> — <strong>GitHub:</strong> listed my 5 most recently pushed repos, with push dates, in about 4 s. <strong>Correct.</strong> — <strong>X:</strong> read the Claude Duet launch post’s public metrics (41,785 likes, 5,155,927 views, 3,174 reposts, 13,141 bookmarks) in about 3 s. <strong>Correct.</strong></p><p>That’s <strong>4/4 connector reads correct</strong>, each in under 5 s.</p><p><strong>Routines (timing)</strong> — I scheduled a one-shot routine for <strong>10:05 AM IST</strong>. It fired at <strong>10:07:58 IST</strong>, <strong>178 s late</strong>, and wrote its result to a file on its own. <strong>It fired, but with about a 3-minute drift.</strong></p><p><strong>Approval before send</strong> By design, any outbound email or message becomes a draft card in my chat. It shows the recipient, subject and body, and nothing goes out until I press Send. Declining discards it. In weeks of daily use, every outgoing email or message has stopped at a draft card that only I can send. When the agent hits a login or a 2FA prompt, it hands me its screen rather than typing a password, and actions it can’t undo, like deleting things, wait for my explicit yes. I never saw it send anything on its own.</p><h3>Verdict by use case</h3><ul><li><strong>You already pay for Claude Pro/Max and live in the US:</strong> join the <strong>Duet</strong> waitlist. You pay nothing extra, your memory carries over, and it has the biggest connector catalogue. The trade-off is preview limits and no control of its browser.</li><li><strong>You want a real computer you can watch, script and take over, from anywhere:</strong> <strong>Grok Bot</strong>. At $30/mo it’s the cheapest paid entry that’s available outside the US, and the box behaves like a dev machine.</li><li><strong>You want something cheap or free inside WhatsApp and Instagram:</strong> <strong>Muse</strong>, if you’re in a supported country.</li><li><strong>You already pay for ChatGPT Pro:</strong> <strong>Dots</strong> is the natural pick. At ₹10,699/mo it’s hard to justify just for the agent.</li></ul><p>My honest take: these products have converged fast. Approval gates and cloud computers are standard now. The real differences are <strong>where you live, what you already pay for, and how much of the agent’s computer you’re allowed to touch.</strong></p><p><em>Duet, Muse and Dots sections are docs-based, not tested. Prices checked on official pages 22 Oct 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GvXCi0KI8DO5hAQQtbEABQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=887563c2d19e" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/claude-duet-vs-grok-bot-vs-muse-vs-dots-always-on-ai-agents-compared-2026-887563c2d19e">Claude Duet vs Grok Bot vs Muse vs Dots: Always-On AI Agents Compared (2026)</a> was originally published in <a href="https://ai.plainenglish.io">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>