Generative AI WATCH CLOSELY OPPORTUNITY

Strata: open-source software to run a large AI model on one PC

ShiniFrontiers · Source: Niko1221 (independent developer)

Section 1 of 6 FREE PREVIEW

What problem does it solve?

Strata is free, open-source software that runs a large AI model, Qwen3.8-Flash-Next, on an ordinary gaming PC. The model has 125 billion parameters (the numbers it learned in training, a rough measure of its size), so it usually needs a server. Strata was built by an independent developer, Niko1221, with community contributors.

A. A private AI assistant on the company's own PC — for business owners and operations heads

Problem: Staff want to use AI on contracts, reports and internal documents, but sending them to an outside AI service may not be allowed. The AI models small enough for an ordinary PC are usually much less capable than large ones.

What it does: Runs a large model on one PC with a gaming graphics card. It chats and reads documents and pictures. The AI processing happens locally on the PC, so documents do not need to be sent to an external AI provider.

Note: The developer has measured its speed, not its quality against cloud AI services.

B. A local engine for coding agents — for developers and technical leads

Problem: Coding agents are AI tools that read and change a codebase, such as Claude Code or Codex. Used with a cloud model, they send code and requests to an external AI provider, creating potential privacy and usage-cost concerns.

What it does: Gives the same agents a local address to use instead, so their requests can be processed on the PC with no per-use AI fee.

See the difference

The situation	Today	With Strata
A private AI assistant on the company's own PCThe company wants a large, capable AI model in-house	Needs a server with hundreds of gigabytes of graphics memory	Runs on a gaming PC with a 12 GB graphics card and 32-64 GB of RAM
A private AI assistant on the company's own PCA manager wants a long confidential contract summarised	Pasted into an outside AI service, or done by hand	The contract can be processed locally on the company's PC rather than being uploaded to an external AI service
A local engine for coding agentsA developer runs a coding agent on the company's private code	In a cloud setup, code goes to an external provider, with a fee per use	The agent can use the PC instead; no external AI provider or per-use fee is needed
A local engine for coding agentsThe agent reads a large codebase before a task	Every read is sent to the provider and billed	Read on the PC at over 1,000 tokens a second, with no fee

The first row is the project's own claim. The other rows are illustrative situations built on what the project documents; it reports speed, not business results.

What it saves: Per-use AI fees, and the need to send company documents or code to an outside AI provider.
What it costs: The software is free. The company needs a capable PC, about 80 GB of disk, and staff time to set it up and test it.
How ready: Working in the real world

Working and installable on Windows and Linux, with speeds measured by its developer. Built by one developer and contributors; it changes often.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

Join free to read it all →

Section 2 of 6 MEMBERS

How does it work?

Join free to read this section

Strata — how it fits together.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

The big picture

Picture a kitchen: the things used all the time stay on the counter, and the rest waits in the pantry. Strata does the same with the AI model: the parts used most sit on the graphics card, the rest waits in the PC's ordinary memory, and the processor helps out at the same time. That is how a model built for servers fits in a gaming PC.

Where Strata fits in the company's setup

A request travels down from the company's people and tools to the PC's hardware. Green is new. Requests can be processed on the PC, so no AI provider outside the company is needed for this work.

Request. Coding agents: Claude Code, Codex or any app that accepts an OpenAI- or Anthropic-style address; one setting points it at Strata.. Strata chat app: Opens in the browser with Chat, a live Monitor of the graphics card, processor and RAM, and the settings.. Staff need only a browser; a coding agent changes one setting, its base URL. Nothing else to install on their side.

AI access. Strata server: A local address on the company's PC, port 8080, in the OpenAI and Anthropic formats most AI apps accept.. No per-use AI fee. Requests can be processed locally instead of at an external provider.

AI model. Qwen3.8-Flash-Next: A 125-billion-parameter model that usually needs a server, compressed to fit by quantization; Strata's setup downloads about 70 GB.. Up to 94 tokens a second on the developer's 12 GB card, faster than a person reads. Answer quality against cloud services is not measured.

Hardware. Graphics card, 12 GB or more: Keeps the most-used experts; each extra GB of VRAM holds about 700 more, so a bigger card is faster.. 32-64 GB of RAM: Holds all 24,576 experts; the processor computes the ones the card does not hold, at the same time.. About 80 GB of SSD: Holds the model files and a 28.8 GB lookup table, read a few rows per word.. A gaming-class PC instead of an AI server: 64 GB of RAM runs every size. The source gives no hardware prices.

Strata: technical view

Why not just use a cloud AI service?

A cloud AI service is easier to start with and needs no hardware. Strata trades that for running the AI processing on the company's own PC.

	Cloud AI service	Strata
Where documents and code go	⚠️ To the outside provider	✅ Can be processed on the company's PC
Cost per use	❌ A fee per use or per seat	✅ None; the company pays for its own PC
Setup	✅ Sign up and start	❌ A capable PC, a 70 GB download and setup time
People served at once	✅ Many	⚠️ One request at a time by default
Proof of answer quality	Published by the provider	⚠️ The project measured speed, not quality against cloud services

How fast is it?

The developer measured it on a gaming PC with an NVIDIA RTX 5070 (12 GB of VRAM), a Ryzen 5 7600 processor and 64 GB of RAM. Speed is counted in tokens; a token is about three quarters of a word. By the project's own yardstick, 60 tokens a second is faster than a person can read.

Model size	Writes answers	Reads a long document	What that means
Q2_0, most compressed	94 tokens a second	2,650 tokens a second	Fastest; replies appear faster than anyone reads
IQ2_XS, recommended	79 tokens a second	2,090 tokens a second	The installer's recommended size: a little slower, a little smarter
IQ3_S, least compressed	53 tokens a second	1,620 tokens a second	Slowest of the four, close to reading speed; matches the full model on its published tests
Coder, for code	55 tokens a second	2,180 tokens a second	Fits a PC with 32 GB of RAM; weaker outside code

Part A: how a staff question moves

Who	What happens	In everyday terms
1. The staff member	Opens the Strata chat page in a browser on the PC and asks a question or attaches a document	Handing a file to a colleague at the next desk
2. Strata	Reads the whole document, then writes the answer on the same PC	The colleague reads it and replies; the file can stay in the room
3. The staff member	Asks follow-up questions; only the new text is read again	The colleague still remembers the file

Part B: how a coding agent uses it

Who	What happens	In everyday terms
1. The developer	Points the coding agent at Strata's local address instead of the cloud provider	Changing which supplier the orders go to
2. The coding agent	Works as before: reads the code, plans and edits	The same worker, with a different supplier
3. Strata	Answers every request on the PC; no fee per request	The supplier is now in-house

It offers an OpenAI-compatible API at http://127.0.0.1:8080/v1, which most AI apps accept; any API key and any model name work.
Apps built for Anthropic's API use /v1/messages; Codex CLI and other apps using OpenAI's Responses API use /v1/responses.
The context window can be set during setup: how much text the model keeps in mind at once.

Join free → I have an account

Section 3 of 6 MEMBERS

How to set it up, and what it costs

Join free to read this section

Strata — steps and cost.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

Before you start: is it a fit?

	A: private assistant	B: coding agents
Good fit if	Documents must not leave the company	Code must not leave the company
	One person or a small group uses it	AI usage fees for coding agents are growing
	A gaming-class PC is available, or can be bought	The developer already has a PC with a 12 GB or larger graphics card
Who acts on it	Operations head, with someone technical to set it up	Technical lead or a developer
Not yet if	Many staff need answers at the same time	The team relies on the very best answer quality and cannot test locally first

Before you begin

You need	Why
An NVIDIA RTX 20, 30, 40 or 50 series card, or a supported AMD Radeon card, with 12 GB of VRAM or more	Holds the most-used parts of the model
32 GB of RAM or more; 64 GB runs every size	Holds the rest of the model; the RAM decides which size fits
About 80 GB of free disk, ideally an NVMe SSD	The model download is about 70 GB; an SSD makes the first start much faster
Windows 10 or 11, or Linux, with a current graphics driver	The driver is the only thing installed by hand; setup installs the rest
Time and a fast internet connection	The first download takes a while; it resumes if it stops

Part 1: Check the PC and install

Download Strata (the person setting it up) Download https://github.com/Niko1221/Strata/archive/refs/heads/main.zip and unzip it, or clone the repository.
Check whether the PC is suitable (the person setting it up) On Windows, in the Strata folder: Expect: a check of this PC only; nothing is installed.
Install and start (the person setting it up) On Windows double-click START-HERE.bat; on Linux run the line below. Press Enter at each question for the recommended answer: model size, how much text to keep in mind, and whether to read pictures. Expect: the model downloads (about 70 GB), then the Strata app opens in the browser at http://127.0.0.1:8080. The PC may be slow or stop responding for 1-3 minutes while the model loads; wait and do not close the window.
Tune it for this PC (NVIDIA only) (the person setting it up) Optional. Takes about 5-10 minutes; on the developer's PC it made the Coder 7% faster.

Part 2: Use it as a private assistant

Test it on real work (the operations head, with two or three staff) Open http://127.0.0.1:8080, use Chat, and try documents the team works on, including non-confidential ones already summarised another way, to compare. Expect: answers on the PC; the Monitor tab shows the graphics card, processor and RAM at work.
Let others reach it from their own devices (the person setting it up) Only after the test. Always set a key; replace <secret> with a long password.

Part 3: Connect a coding agent

Point the agent at Strata (the developer) In the agent or app, add an OpenAI-compatible provider with the base URL http://127.0.0.1:8080/v1; any API key and model name work. For Claude Code, set: Expect: the agent works as before, with requests answered on the PC.
Compare it on a real task (the technical lead) Give the same task to the agent on Strata and on the current cloud service, and compare the result before moving any work over.

What it costs

Item	Cost
The software	Free (MIT open-source licence)
The model	Free to download; each model carries its own licence, which the company should read before business use
AI usage	None per use; it runs on the company's PC
Hardware	A PC as listed above; the source gives no prices
People	Time to set up and test, and to compare answers with today's tools; the source gives no estimate

Join free → I have an account

	What changes	What it can mean for the business
Privacy	Documents and code are processed on the company's own PC	AI can be used on work the company would not send to an outside provider
Cost	No fee per use	Heavy users, such as coding agents, stop adding to an AI bill
Capability	A large model, not just the small ones that usually fit a PC	Can take on harder questions than a small local model
No lock-in	Works with apps built for the OpenAI or Anthropic format	Can sit alongside a cloud service, or replace it for some work

Section 5 of 6 MEMBERS

What are the limits?

Join free to read this section

Strata — where it falls short.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

What this project can't do yet

Limit	Why it matters	What to do
One developer, own measurements	The speed figures come from the developer's PCs; community results are collected but not independently checked.	Run it on the company's own PC before deciding.
No quality comparison with cloud AI	The sources show speed, not how good answers are compared with a cloud service.	Compare answers on real tasks first (Part 2, step 1).
One request at a time	By default others wait. Setting "parallel": 2 answers two at once, but each answer is slower on a 12 GB card.	Start with one person or a small group.
It takes over the PC	It loads 35-55 GB into RAM and fills the graphics card; the PC can freeze for 1-3 minutes at start.	Use a PC set aside for it, not someone's daily machine.
Local is not the same as secure	Once shared with other devices, it can be reached over the network; chats are kept in each browser's own storage.	Always set an API key when sharing it, and protect the PC like any machine holding company data.
The Coder version is weaker outside code	It keeps half the model's experts, chosen on code: 91% of the full model's score on a coding test, by its authors. Answers in Chinese came out wrong in one report.	Use a full-model size for anything other than code.
Pictures on AMD cards	On Windows, AMD cards cannot read pictures yet; on Linux they take 10-30 seconds each.	Use an NVIDIA card if pictures matter.
A fast-moving project	New versions arrive often and change speed and settings.	Keep the version that works and test before updating.

What it does not do

It runs one model, Qwen3.8-Flash-Next, in several sizes and versions; it is not a general tool for any model.
It does not include hardware or support; the company provides the PC and the time.
It does not run on built-in laptop or desktop graphics; integrated Radeon graphics are listed as not supported.

Join free → I have an account

Strata: open-source software to run a large AI model on one PC

What problem does it solve?

How does it work?

The big picture

Where Strata fits in the company's setup

Why not just use a cloud AI service?

How fast is it?

Part A: how a staff question moves

Part B: how a coding agent uses it

How to set it up, and what it costs

Before you start: is it a fit?

Before you begin

Part 1: Check the PC and install

Part 2: Use it as a private assistant

Part 3: Connect a coding agent

What it costs

What do you gain?

What are the limits?

What this project can't do yet

What it does not do

Sources