Skip to content

Add 1,023-byte experimental x86-64 inference engine - #1

Open
cneuralnetwork wants to merge 1 commit into
utsav1033:mainfrom
cneuralnetwork:codex/under-1kb-greedy
Open

cneuralnetwork wants to merge 1 commit into
utsav1033:mainfrom
cneuralnetwork:codex/under-1kb-greedy

Conversation

@cneuralnetwork

Copy link
Copy Markdown

Summary

  • Add a 1,023-byte self-unpacking Linux x86-64 SmolLM2-135M engine.
  • Supply the original tokenizer and safetensors as mmap-able regular files on fd 0 and fd 3, removing pathname opens from the ELF.
  • Include a 1,149-byte descriptor-input variant that retains the checked tokenizer behavior, plus sources, packed binaries, and reproducible build/verification scripts.

Accuracy tradeoff

The 1,023-byte engine uses greedy longest-vocabulary-match tokenization. It matches all 300 reference generated IDs across the three answer-key prompts, but only 13/20 focused tokenizer cases and 236/1,480 texts in a separate deterministic corpus. It always generates 100 tokens and omits the final newline. This is an experimental loose variant, not a general exact replacement.

The 1,149-byte variant passes 20/20 focused tokenizer cases, 300/300 reference generated IDs, all three reference text outputs, and exits cleanly.

Verification

On relaydeck, from a clean checkout of current upstream main:

cd x86
./build_fd.sh
./verify_fd.sh ~/smol

The 1,023 and 1,149 byte figures count the complete packed ELFs. The model, tokenizer, and optional shell wrapper are outside the binary.

@cneuralnetwork
cneuralnetwork marked this pull request as ready for review November 4, 2026 06:47

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant