AI Operations WATCH CLOSELY OPPORTUNITY

Edge0: free software that runs AI on company laptops

ShiniFrontiers · Source: Edge0-AI

Section 1 of 6 FREE PREVIEW

What problem does it solve?

Edge0 is free, open-source software that runs an AI language model on a laptop instead of in the cloud. Today it works with two models the Edge0 team prepared from open models (Qwen3.6 and Ling 3.0). Staff can draft, summarise and ask questions without sending company text to an outside AI service.

Edge0 — for companies that want AI without sending data out

Problem: Capable AI models are too big for normal laptops, so companies send their text to an outside AI service and pay for every request.

What it does: Keeps the large model on the laptop's storage and loads only the small part each answer needs, so it runs in about 3 GB of memory.

Note: Today a developer must set it up; there is no download-and-go app yet. Both models are labelled "preview".

See the difference

Situation	Today	With Edge0
A lawyer needs a confidential contract summarised	It goes to an outside AI service, or is not summarised with AI at all	Summarised on the lawyer's own laptop; the contract never leaves it. Expect about half a minute of reading first.
A sales team drafts hundreds of customer emails a month	Every draft is a paid request to an outside AI service	Drafted on the team's own laptops, with no per-request fee
A company app already talks to ChatGPT's interface	Moving to a private AI means rewriting the connection	The app points at the laptop instead; Edge0 speaks the same interface

Illustrative examples. Timings are worked out from the project's own measurements on a Mac mini M4 Pro.

What it saves: Per-request AI fees for everyday drafting, summarising and questions, with the text kept on the company's devices.
What it costs: The software and models are free. It needs a Mac with Apple's own M1–M4 chip today, up to 23 GB of disk, and developer time to set up.
How ready: Working in the real world

Working preview: ready on Macs with Apple's own M1–M4 chips now; iPhone, Android and Windows apps released 30 September as source code.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

Join free to read it all →

Section 2 of 6 MEMBERS

How does it work?

Join free to read this section

Edge0 — how it fits together.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

The big picture

Picture a huge library on a small desk. The books stay in the basement, and a librarian fetches only the few pages each question needs, guessing the next request so the pages are ready in time. Edge0 does this with a large AI model: the model stays on the laptop's storage, and only the parts in use sit in memory.

How Edge0 fits on a company laptop

A request travels down the stack and never leaves the device. Green is new.

App. An app on the laptop: for drafting, summarising and questions; an app built for ChatGPT's interface can point here instead. Existing apps switch with small changes: Edge0 speaks the same interface as OpenAI's.

Engine. Edge0: free, open-source engine; predicts which parts of the model each answer needs and loads only those. No per-request AI fee. Quality is about 95% of the full model: 79.2 against 83.2 on five tests.

Model. Edge0's two preview models: built from the open models Qwen3.6 and Ling 3.0; downloaded once from Hugging Face or ModelScope. Writes 15 to 18 tokens a second on the large model: a few paragraphs take about 20 seconds.

Hardware. A Mac with Apple's M1–M4 chip: today; iPhone, Android and Windows apps were released on 30 September as source code. Existing laptops, no new servers; a developer sets it up today.

Edge0: technical view

Why not just use a cloud AI such as ChatGPT?

A cloud AI needs no hardware and offers the provider's best models. But every request leaves the company and is billed. Edge0 trades a little quality and speed for keeping both the data and the cost in-house.

	Cloud AI service	Edge0
Where the text goes	⚠️ To the provider's servers	✅ Stays on the device
Cost	❌ A fee for every request	✅ No per-request fee
Hardware	✅ None needed	⚠️ A Mac with Apple's own M1–M4 chip today; phone and Windows apps are newer
Quality	✅ The provider's best models	⚠️ A little below the full model it is built on (see below)
Speed	Usually fast	⚠️ Slower: see below
Works with existing apps	n/a	✅ Speaks the same connection format as ChatGPT-style apps (OpenAI's chat interface), so they can switch

What the numbers mean

Measure	Large model (35B)	Small model (8B)	What that means
Quality, average of five tests (max 100)	79.2, vs 83.2 for the full model	69.9, vs 72.7	It keeps about 95% of the full model's score: fine for drafts and summaries; check it on demanding work
Writing speed	15 to 18 tokens a second	24 to 25 a second	A 300-token answer, a few paragraphs, takes about 20 seconds on the large model, 12 on the small one
Reading a long document (3,300 tokens)	113 to 140 tokens a second	500 to 1,428 a second	About half a minute before the large model starts answering; a few seconds on the small one
Memory in use	about 2.9 GB	about 1.0 GB	Fits beside other apps on an ordinary laptop
Disk space	about 23 GB	about 4.2 GB	Start with the small model

How a request moves

Who	What happens	In everyday terms
1. The employee	Asks in an app on the laptop or phone	A question at the library desk
2. Edge0	Predicts which parts of the model the answer needs and loads only those	The librarian fetches the right pages from the basement
3. The laptop	Runs the model in about 3 GB of memory	The desk holds only what is open
4. The employee	Gets the answer; the text never leaves the device	The answer is written at the desk, not sent out

Two model sizes, both built on open-weight models (Qwen3.6 and Ling 3.0), squeezed to a quarter of their size (4-bit) by quantization and retrained with LoRA adapters to win back most of the lost quality.
The project's technical report is on arXiv: "The Other Half of the Memory Wall" (16 September 2026).

Join free → I have an account

Section 3 of 6 MEMBERS

How to set it up, and what it costs

Join free to read this section

Edge0 — steps and cost.

Join free to unlock: how it works, a step-by-step setup with its costs, what you gain, and where it falls short, for every Friday idea.

Before you start: is it a fit?

	Edge0
Good fit if	Staff handle confidential text they cannot send to an outside AI
	AI usage fees are growing with volume
	The company already uses Macs with Apple's own M1–M4 chips
Who acts on it	Head of IT or operations, with a developer
Not yet if	The work needs the very best AI quality or instant answers, or the company runs Linux or NVIDIA servers (on the roadmap)

Before you begin

You need	Why
A Mac with Apple's own chip (M1 to M4)	The ready version runs there; the phone and Windows apps are newer
Python 3.10 or later (3.12 recommended)	The programming tools Edge0 needs; the developer installs them
About 4.2 GB of free disk (small model) or 23 GB (large model)	The model is kept on disk and read in pieces
A developer	To install it and connect the company's apps

Part 1: Run it on a Mac

Download the code (the developer)
Install (the developer)
Download the small model first (the developer)
Try it (the developer) Expect: a short demo conversation running entirely on the Mac.

Part 2: Check it on real work

Run real tasks side by side (the team lead, with the developer) Take everyday tasks the team already does with a cloud AI, such as summaries and email drafts, and compare Edge0's answers and waiting time. Expect: a clear view of which tasks it handles well enough.

Part 3: Connect the company's apps

Start the local server (the developer) Expect: a local chat address at http://127.0.0.1:8000/v1/chat/completions, in the same format ChatGPT-style apps use (OpenAI's chat interface).
Point an app at it (the developer) Any app that uses OpenAI's chat interface can switch by changing its address to the local server.
Add phones and Windows PCs (the developer) The iPhone, Android, macOS and Windows apps are in the repository's ios/, android/, macos/ and windows/ folders, each with its own guide. They are new; test them separately.

What it costs

Item	Cost
The software and models	Free (Apache 2.0 open-source licence)
AI usage	None per request. Compare with the company's monthly cloud AI bill for the same work: that is the fee this removes.
Hardware	None new for Macs with Apple's own M1–M4 chips; the large model needs about 23 GB of disk
People	Developer time to install it, test it on real tasks and connect apps

Join free → I have an account

	What changes	What it can mean for the company
Privacy	Confidential text is processed on the device	Less risk of sensitive data leaving the company
Cost	No per-request AI fee	Everyday AI use stops growing the bill
Hardware	A large model runs in about 3 GB of memory	Existing laptops can be used, without new servers
Compatibility	Same chat interface as OpenAI's	Existing apps can switch with small changes

Limit	Why it matters	What to do
Both models are previews	Their makers label them "preview", so behaviour may change.	Test on real tasks before relying on them.
Mac first	The ready Python version runs only on Macs with Apple's own M1–M4 chips; NVIDIA support is planned for late 2026.	Pilot on Macs; test the phone and Windows apps separately.
A little lower quality	About 4 points below its full model on the project's tests.	Use it for everyday drafts and summaries; keep a cloud model for the hardest work.
Slower than the cloud	Long documents take about half a minute before the large model answers; the first request after starting is slower still.	Use the small model where speed matters.
Large download	The large model takes about 23 GB of disk.	Start with the small model (about 4.2 GB).
Long conversations need more memory	Memory grows with the length of the conversation.	Leave memory headroom beyond the 3 GB figure.
A young project	First released on 8 September 2026.	Pin one version and plan for frequent updates.

Edge0: free software that runs AI on company laptops

What problem does it solve?

How does it work?

The big picture

How Edge0 fits on a company laptop

Why not just use a cloud AI such as ChatGPT?

What the numbers mean

How a request moves

How to set it up, and what it costs

Before you start: is it a fit?

Before you begin

Part 1: Run it on a Mac

Part 2: Check it on real work

Part 3: Connect the company's apps

What it costs

What do you gain?

What are the limits?

What this project can't do yet

What it does not do

Sources