bytesized: a 2,946-byte LLM engine

How small can a program be and still talk?

A C program that runs SmolLM2-135M, a real LLM, and produces exactly the same greedy tokens as Hugging Face transformers. It reads the original model.safetensors and tokenizer.json with no converter. The binary is 2,946 bytes (Linux arm64), small enough to fit in a single QR code (max 2,953 bytes).

Left: decode the QR code, compare sizes, run it (video, real speed). Right: the QR code is the whole program (click it for full size). To get it back:

zbarimg --raw --oneshot -Sbinary qr.png > final && chmod +x final

Most phone camera apps treat QR codes as text and will mangle the binary, so use zbarimg (from zbar) on a computer.

$ ./smol
╭──────────────────────────────────────────────────────────╮
│ ✻ smol                                                   │
│                                                          │
│ engine  2,946 bytes of C                                 │
│ model   SmolLM2-135M, unmodified safetensors             │
│ decode  greedy, matches HF transformers token for token  │
╰──────────────────────────────────────────────────────────╯

> def fibonacci(n):
def fibonacci(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    else:
        return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(10))

  end of text · 6.7s

smol is an optional Python front end. It only runs the binary and draws the screen, and it isn't part of the 2,946 bytes. The engine on its own is ./final "prompt" [n_tokens].

1. Size = bytes of the final binary. The model files are not counted.
2. Correct = greedy decoding, and the first 100 tokens match HF transformers (fp32) exactly. One different token = broken.
3. Model files unmodified. It reads the original model.safetensors and tokenizer.json.
4. Speed = same machine, same prompt, repeated runs, prompt length stated.

You need arm64 Linux. I tested it in OrbStack on an M4 Mac. It should also work on a 64-bit Raspberry Pi 4/5 or AWS Graviton, but I haven't tried those.

# model files (original, from Hugging Face)
mkdir ~/smol && cd ~/smol
for f in model.safetensors tokenizer.json; do
  curl -LO https://huggingface.co/HuggingFaceTB/SmolLM2-135M/resolve/main/$f
done

# run (must be run from the model folder)
/path/to/bin/final "The cat sat" 50
/path/to/smol                    # the little CLI, SMOL_DIR=~/smol by default

• Weights: mmap the safetensors file. The header lists tensors in sorted order, so walk it by position instead of matching names: find the next ., then the next ts (the end of data_offsets). bf16 → fp32 is bits << 16.
• Tokenizer: byte-level BPE using only the vocab. In this tokenizer, merge rule n always creates token id n + 252, so "merge the pair with the lowest id" is exactly BPE. The 48,900 merges are never read.
• Transformer: embedding → 30 × (RMSNorm, attention with RoPE + GQA + KV cache, RMSNorm, SwiGLU MLP) → RMSNorm → tied embedding → argmax. About 40 lines.
• No libc: 5 raw syscalls (openat, lseek, mmap, write, exit). It has its own exp and cos (sin is cos(a - π/2)), computed in double precision so the results round the same way PyTorch's do.
• Self-unpacking: the file is a 112-byte ELF header, a 348-byte assembly unpacker and 2,486 bytes of compressed engine. It unpacks to 3,698 bytes in RAM, flushes the instruction cache and jumps in. The compression is LZ77 plus an LZMA-style range coder, with a separate model for each byte position of the 4-byte arm64 instructions. The program header overlaps the last 8 bytes of the ELF header (hdr.py).

Every row passed the exact-match check before the next change was made.

• ref.py: HF transformers, fp32, greedy, 100 new tokens for 3 prompts → answer_key.json (also stores the top-2 score gap at every step).
• check.py BIN: compares token IDs (from a -DCHECK build) or the decoded text against the answer key.
• tokcheck.py BIN: prompt tokenization vs the HF tokenizer on 12 tricky strings, plus 4 Unicode strings.
• Result: 3/3 prompts × 100 tokens identical. The closest step, in "Once upon a time" at step 94, had " be" 29.1153 vs " learn" 29.1133.

M4, OrbStack Ubuntu VM, 1 thread. Prompt "The cat sat" (3 tokens) + 100 new tokens. 3 runs each; the times include loading.

rv/ is the same engine for standard RISC-V (RV32IMC + Zcb) plus 6 custom 16-bit AI instructions: matvec, rmsnorm, attention (with RoPE and its own KV cache), silu-gate, embed, argmax. The program is 384 bytes: small enough to post as a single tweet in base2048 (rv/tweet.txt, 280 characters by X's own counter).

• Exact: the same checks pass (3 prompts × 100 tokens, ids and text, plus the 12 tokenizer tests).
• Emulated: it runs on rv/emu.c. There is no chip or FPGA yet; Verilog in a simulator is next.
• General instructions: none of them has SmolLM2 built in. The model's sizes come from a 10-byte header.
• Simpler tokenizer: one pass over the vocab in id order plus one whitespace rule. It misses 3 of 823 chunks of real text, where this arm64 engine misses 1.

Details, encodings and every size step: rv/ISA.md.

wasm.c is the same engine as a 3,683-byte WebAssembly module (smol.wasm), so it also runs in a browser or in Node. It has no file access, so the host copies tokenizer.json, model.safetensors and the prompt into its memory at fixed addresses and calls r(prompt_len, n). Text comes back through one import.

• Exact: the same checks pass (3 prompts × 100 ids, the text, the 12 tokenizer tests).
• Size: 3,683 bytes, not compressed (2,079 gzipped). It's about the size of the arm64 engine before it unpacks itself (3,698), since a wasm module can't rewrite its own code.
• Speed: about 12 tok/s in Node 22 on the M4 VM and in the browser. That's faster than the 2,946-byte arm64 build (9.4).

./wasm.sh smol.wasm                      # build (clang with the wasm32 target, wasm-ld, wasm-opt)
cd ~/smol && /path/to/run.js "The cat sat" 50

Try it in your browser (docs/index.html). It downloads the original model files (270 MB) from Hugging Face on the first run. To run the page locally, serve the docs folder, for example python3 -m http.server -d docs.

• The small binary is Linux arm64 only: raw syscalls and an assembly unpacker. The WebAssembly build runs anywhere with a browser or Node.
• The lowest-id BPE shortcut is tested, not proven. Every merge result id = rank + 252 (checked for all 48,900). It matched real BPE on 122,463 distinct pieces (bpe_sim.py).
• The pre-tokenizer treats every byte ≥ 0x80 as a letter. It passed the accent, Japanese and emoji tests, but it isn't the full Unicode regex.
• Special tokens typed into the prompt (<|endoftext|>) are not mapped to their special IDs.
• No error handling. It must run in the model folder. Prompt + output ≤ 1024 tokens.
• It's a base model: it continues text, it doesn't chat.

Inside arm64 Linux with clang, lld and python3:

./pack.sh tiny.c final            # the 2,946-byte binary
./pack.sh tiny.c finalc -DCHECK   # same, but also prints token ids to stderr
./build.sh tiny.c tiny            # unpacked no-libc build (row 9)
clang -O2 -o v0 v0.c -lm          # readable reference
cd ~/smol && python3 /path/to/check.py /path/to/final

• Fit in a QR code. Done: 2,946 bytes, 7 under the limit. The QR code above scans back byte-for-byte and runs.
• Put unpacker code into ELF header fields the kernel ignores.
• A context-mixing model instead of LZ + range coder.
• An x86-64 build.