FreeToken — how it fits together.
Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.
The big picture
Picture a kitchen where the cook keeps what is in use on a small counter and fetches the rest from a big pantry. FreeToken works the same way: the graphics card is the counter, and main memory is the pantry.
How FreeToken fits into a company's setup
A coding assistant's request travels down to the computer's hardware. Green is new. After the one-time model download, nothing leaves the company.
Request. AI coding assistant or app: Claude Code, Codex, OpenCode or OpenClaw, or any app built for OpenAI's or Anthropic's services. FreeToken's setup command: repoints the assistant from the AI company to the model on this computer, and removes its Anthropic login. Assistants keep working as before, with no fee per request.
Engine. FreeToken: picks which model pieces sit on the graphics card; remembers what the assistant already sent. 1.5 to 2.3 times as fast as the best other local tool, in the makers' tests. Nobody else has checked this.
Model. A free AI model: chosen from more than 20, such as Qwen3.6 (mid-sized), DeepSeek-V4-Flash (large) or GLM-5.2 (very large). The company picks the model size its computers can hold, and can switch as better free models appear.
Hardware. Computer with an NVIDIA graphics card: RTX 30, 40 or 50 series, or a workstation card. Main memory for the whole model: 32 GB ran the mid-sized model on a laptop; the makers used 180-192 GB for the large model and 512 GB for the very large one. A gaming laptop ran the mid-sized model at 39 tokens a second. A typical cloud assistant writes 33.
FreeToken: technical view
Copy the piece over, or work on it where it is?
Each word the model writes uses a few pieces of the model, called experts, and most are already on the graphics card. Missing ones are either copied over through a link called PCIe, or handled by the main processor.
| Copy it to the graphics card | Work on it in main memory |
|---|
| What happens | The piece travels over the link and the fast graphics card does the work | The main processor works on the piece where it is stored |
| Better when | The link inside the computer is fast | Main memory is fast compared with the link |
| Extra benefit | The piece stays on the card for the next words | Nothing has to travel |
Why not just use an existing local tool such as Ollama or llama.cpp?
The makers tested FreeToken against three local tools: llama.cpp, Ollama and KTransformers. llama.cpp and Ollama fix the pieces on the graphics card at start-up, so most work falls to the slower main processor. They did well on short questions, where llama.cpp started answering fastest.
| Cloud AI service | llama.cpp, Ollama, KTransformers | FreeToken |
|---|
| Where the code goes | ⚠️ To the AI company | ✅ Stays on the computer | ✅ Stays on the computer |
| Speed in coding sessions (RTX 5090, mid-sized model) | Typical Codex: 33 tokens a second | ⚠️ The best of them was 1.8 to 2.3 times slower than FreeToken | ✅ 77 to 83 tokens a second |
| Longest wait for an answer to start | Not measured in the paper | ❌ Each one passed 2.5 minutes in some test: llama.cpp 232 seconds, Ollama 179, KTransformers 946 | ✅ Under 44 seconds in every test |
| The large DeepSeek model (284 billion) | n/a | ⚠️ Ollama could not run it | ✅ 22 to 25 tokens a second |
| The very large GLM-5.2 model on one workstation | n/a | ⚠️ llama.cpp: 7 tokens a second; KTransformers could not run it | ✅ 15 tokens a second |
How fast is it?
Speed is counted in tokens a second. All figures below come from the makers' paper, and nobody else has repeated them. The paper measures speed only, not how good the answers are: that depends on the model the company picks.
| Computer | AI model | Speed | What that means |
|---|
| Gaming laptop (RTX 4060, 8 GB graphics card) | Qwen3.6, mid-sized (35 billion) | 39 tokens a second | A little faster than a typical cloud coding assistant (33) |
| Gaming PC (RTX 5090, 32 GB graphics card) | Qwen3.6, mid-sized (35 billion) | 77 to 83 tokens a second | A few paragraphs in about 4 seconds |
| Gaming PC (RTX 5090, 32 GB graphics card) | DeepSeek-V4-Flash, large (284 billion) | 22 to 25 tokens a second | A few paragraphs in about 13 seconds: fine for an assistant, slower than the cloud |
| Workstation (RTX PRO 6000, 96 GB graphics card) | GLM-5.2, very large (753 billion) | 15 tokens a second | Twice llama.cpp on the same machine |
| Measure | FreeToken | What that means |
|---|
| Speed as a session grows | Stays within 12% of its speed on a single question | Long coding sessions do not slow to a crawl; one other tool lost 31% |
| Longest wait for an answer to start (time to first token) | Under 44 seconds in every test | Assistants do not give up: OpenClaw stops waiting after 120 seconds; other tools hit 179-946 seconds |
| Missing pieces on an RTX 5090 | 16 in 100 for the mid-sized model, 39 in 100 for the large model | Fewer trips to main memory than the other tools' 41-89 in 100 |
Part A: how a coding assistant's request moves
| Who | What happens | In everyday terms |
|---|
| 1. The developer | Repoints Claude Code from the AI company to the model on the company computer, with one command | Keeping the same shop, but changing the supplier |
| 2. Claude Code | Works as before: reads code, runs tools, edits files | The same worker, with an in-house supplier |
| 3. FreeToken | Reads only what is new, even after the assistant deletes old notes | A colleague who remembers the meeting and reads only the latest email |
| 4. The company's computer | Writes the answer; nothing goes to an outside AI company | The work never leaves the building |
Part B: how a computer runs a model bigger than its graphics card
| Who | What happens | In everyday terms |
|---|
| 1. The developer | Runs a one-time speed test of the computer | Timing the walk from counter to pantry |
| 2. FreeToken | Keeps the whole model in main memory, and the pieces in use on the graphics card | The pantry holds everything; the counter holds what is in use |
| 3. FreeToken | Fetches each missing piece, or has the main processor work on it (explained above) | Send the runner, or have the helper make it in the pantry |
| 4. FreeToken | Gives back graphics-card space while running, without a restart | Clearing counter space when someone else needs it |
- Large free models, also called open-weight models, are usually mixture-of-experts models: each word uses only a few of the model's many pieces. DeepSeek-V4-Flash uses 13 billion of its 284 billion parameters for each word.
- On a long request, FreeToken loads the next pieces while the graphics card works. Without this, reading was 19-26% slower.
- The research paper is "FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution" (arXiv, 17 August 2026).