Rete helps you remember people during conversations on Even Realities G2 glasses. The glasses show a compact person card and relevant saved facts or a grounded answer. The phone shows the live transcript and provides onboarding, identity correction, and a memory vault.
The implementation is consolidated on rehat. See verification status for automated, live-provider, simulator, and remaining physical-device checks.
Based on the official Even Realities ASR template.
-
npm install, then copy.env.exampleto.env.localand fill inOPENAI_API_KEYandMEM0_API_KEY. -
npm run dev(keep it running; it is the token server and the memory server). -
Prove the pipeline before touching the glasses:
node scripts/demo-check.mjs
It runs a scripted conversation through the server and prints each stage: person created, facts extracted, facts confirmed in Mem0. It ends with
PASSor a one-lineFAILthat names the stage and the usual cause. Watch thenpm run devterminal for[memory]/[mem0]lines. -
On the phone, open
http://<your-lan-ip>:5173/vault.html#onboardingand enter your name (memory capture starts only once the wearer has a profile). Both phone pages share a tab bar: Captions, Profile, People, Review. -
npx @evenrealities/evenhub-cli qr --url http://<your-lan-ip>:5173, scan it in the Even App, allow the microphone, and talk. Tap pauses; double-tap saves and exits.
Install dependencies with npm install. Copy .env.example to .env.local and set the server-only keys:
OPENAI_API_KEY=your-openai-api-key
MEM0_API_KEY=your-mem0-api-keyMem0 is the durable source of truth for people and the facts about them, keyed by the person's name; no raw conversation data is stored remotely. The wearer profile is the exception: it lives in a local JSON file at .data/profile.json (override the folder with the optional RETE_DATA_DIR). Keep both keys private and never prefix them with VITE_. Restart the server after changing them. See memory setup for the data boundary and existing-data behavior.
npm run devOpen /vault.html#onboarding to enter your name, aliases, background, current work, interests, and goals. The wearer profile is separate from contacts. The vault also lists people and dated facts, lets you correct names or delete memories, and provides a review queue for uncertain identities. The Captions, Profile, People and Review tabs are at the top of both phone pages on every launch.
The Vite development server supplies both the short-lived transcription token route and the memory API. It listens on the network so the phone can connect; the memory routes have no authentication. Use a trusted network for this prototype.
Keep the development server running. For the simulator:
npm run simulateFor glasses, connect the phone and computer to the same network and generate a QR code:
npx @evenrealities/evenhub-cli qr --url http://<your-lan-ip>:5173Open it in the Even App and allow microphone access. A normal browser alone does not supply the glasses bridge or PCM audio. The manifest requires Even App 2.2.10 and SDK 0.0.16.
/?mock=1 plays a scripted demo with a seeded in-memory vault. /?mock=1&memory=http sends scripted turns to the real memory server. See the demo script for custom transcript injection and the hardware checklist.
- Tap pauses or resumes transcription. Already-sent audio can finish while paused.
- Double-tap saves and ends the conversation, clears the person/context cards, and opens the system exit dialog. The phone keeps the ended conversation's transcript under a "saved" divider; only the live area is cleared. Cancelling restores capture with a fresh conversation and a new Mastra thread. A failed transcription drain retains the original conversation for retry.
- Swipe has no assigned action.
Conversation turns feed the memory agent when a person is identified. The Luna Mastra agent runs once per finalized contact turn: it may identify the contact when they introduce themself by name, loads that person's saved Mem0 memories with its search tool, and saves only new, directly supported facts with its save tool in the same run. The wearer is never the subject of a fact. Phone correction is available when identity is uncertain. Transcription stays on the phone: wearer captions use a darker green than the other speaker's captions, and unknown speech is marked explicitly. Speaker-role accuracy still requires physical-device checks.
People and their facts are stored in Mem0, with each person's facts under that person's name; the wearer profile is stored locally in .data/profile.json. The active server keeps encounter routing in RAM and has no local authoritative memory database, background job queue, or index-only fallback. Unidentified or in-progress speech stays only in RAM until a person is identified and facts are extracted, and is never checkpointed to Mem0; the glasses client can retain unsent requests in its bridge retry queue.
G2 supplies 16 kHz mono signed 16-bit PCM. A stateful resampler converts it to 24 kHz for gpt-live-transcribe. Only a short-lived transcription token reaches the client. Turns commit after speech followed by about 600 ms of quiet, after 15 seconds of continuous audio, or on pause. Stable segment IDs and revisions reconcile partial and final text.
Mastra runs a stateless gpt-6-luna agent with three tools: identify_person, search_memories and save_fact. Search and save hit Mem0 during the agent call, so memories are loaded and written on the fly while the conversation runs; saved facts live under the identified person's name. Mem0 errors are surfaced so a turn can be retried. See memory setup for details.
npm test
npm run buildTests use Node's native TypeScript support (Node 26 verified), with mocked provider calls. Synthetic live-provider checks and simulator checks do not establish real G2 microphone or speaker-role accuracy.
npm run pack builds an .ehpk package. npm run preview, static hosting, and a standalone package do not supply the development server's token or memory API. Production deployment needs an authenticated application server, persistent storage, and the corresponding client URL and manifest network configuration.
| Path | Purpose |
|---|---|
src/asr/ |
Audio resampling, streaming transcription, segment revisions, and drain recovery. |
src/memory/ |
Encounter identity, context, HUD, offline client, and save/exit sequencing. |
src/vault/, vault.html |
Wearer onboarding, people, facts, and identity review. |
server/realtime-token.ts |
Server-only transcription token creation. |
server/memory/ |
Mem0 storage for people and facts, local wearer profile file, and the Mastra memory agent with its Mem0 tools. |
shared/memory-api.ts |
Shared HTTP contract. |
vite.config.ts |
Development APIs and build entries. |