OpenWeights / iOS

Open-weight AI.
On your iPhone.

OpenWeights is an experimental iPhone app for running open-weight language models locally. Find compatible models on Hugging Face, download or import weights, and chat on your own device.

Built by Experimental Machines with SwiftUI, llama.cpp, MLX and ExecuTorch.

On-device inference · Measured 7 October 2026

CPU and Metal
on iPhone 16.

Five open-weight models. The same model bytes on each backend. Six-turn conversations that measure how chat feels as the history grows.

Metal started answering sooner and streamed faster in every tested model. LFM2.5 led streaming speed. Gemma started fastest, but often missed the requested reply format.

iPhone 16 · iOS 26.6.2 · llama.cpp · 3 conversations per model/backend · 180 completed turns

01 / Personal iPhone

Four metrics, side by side

Choose a measurement to compare CPU and Metal within each model. Different tokenizers and quantizations limit comparisons between models.

Nominal samples only. Median first text and streaming speed; median of eligible conversation memory peaks.
Model	Backend	M1 First text seconds ↓	M2 Streaming tokens/s ↑	M3 Process peak MiB	M4 Facts correct/scorable	Strict format correct/observed	Nominal turns eligible/planned
Gemma 3 1B IT	CPU	0.759	38.6	793.0	9/9	3/9	18/18
Gemma 3 1B IT	Metal	0.295	45.3	792.9	9/9	3/9	18/18
LFM2.5 1.2B Instruct	CPU	1.070	43.4	720.8	9/9	9/9	18/18
LFM2.5 1.2B Instruct	Metal	0.370	65.3	720.9	9/9	9/9	18/18
Qwen3 1.7B	CPU	1.344	19.0	1778.0	9/9	9/9	18/18
Qwen3 1.7B	Metal	0.438	25.5	385.8	8/8	8/8	17/18
Llama 3.2 3B Instruct	CPU	3.031	14.6	1234.7	8/8	8/8	17/18
Llama 3.2 3B Instruct	Metal	0.745	22.0	401.5	8/8	8/8	17/18
SmolLM3 3B	CPU	2.798	14.0	436.5	7/7	7/7	16/18
SmolLM3 3B	Metal	0.997	23.4	321.3	6/6	6/6	12/18

All configurations completed 3/3 conversations. Eleven warm turns were excluded from headline timing and recall. Memory requires a wholly nominal conversation, leaving 1 to 3 eligible conversations per configuration. Exact sample counts.

Show throughput and memory sample counts

Model	Backend	Throughput turns / 18	Memory conversations / 3
Gemma 3 1B IT	CPU	15/18	3/3
Gemma 3 1B IT	Metal	15/18	3/3
LFM2.5 1.2B Instruct	CPU	15/18	3/3
LFM2.5 1.2B Instruct	Metal	15/18	3/3
Qwen3 1.7B	CPU	15/18	3/3
Qwen3 1.7B	Metal	14/18	2/3
Llama 3.2 3B Instruct	CPU	14/18	2/3
Llama 3.2 3B Instruct	Metal	14/18	2/3
SmolLM3 3B	CPU	13/18	1/3
SmolLM3 3B	Metal	10/18	2/3

Download streaming figure (SVG)Download first-text figure (SVG)Download results (CSV)

02 / Conversation history

Later replies take longer to start

On the sixth turn, the two 3B models took about 11 to 12 seconds to start a CPU reply, versus about 3.6 seconds on Metal. Later prompts are longer and ask different questions. This does not isolate history length alone.

First-text delay by turn, seconds (nominal n)
Model	Backend	Turn 1	Turn 2	Turn 3	Turn 4	Turn 5	Turn 6
Gemma 3 1B IT	CPU	0.357 (n=3)	0.636 (n=3)	0.446 (n=3)	0.875 (n=3)	0.980 (n=3)	2.594 (n=3)
Gemma 3 1B IT	Metal	0.132 (n=3)	0.243 (n=3)	0.063 (n=3)	0.291 (n=3)	0.337 (n=3)	0.896 (n=3)
LFM2.5 1.2B Instruct	CPU	0.479 (n=3)	0.999 (n=3)	0.550 (n=3)	1.157 (n=3)	1.198 (n=3)	3.752 (n=3)
LFM2.5 1.2B Instruct	Metal	0.187 (n=3)	0.320 (n=3)	0.077 (n=3)	0.432 (n=3)	0.465 (n=3)	1.291 (n=3)
Qwen3 1.7B	CPU	0.565 (n=3)	1.320 (n=3)	0.668 (n=3)	1.530 (n=3)	2.283 (n=3)	5.918 (n=3)
Qwen3 1.7B	Metal	0.223 (n=3)	0.435 (n=3)	0.240 (n=3)	0.591 (n=3)	0.693 (n=3)	1.690 (n=2)
Llama 3.2 3B Instruct	CPU	1.307 (n=3)	3.028 (n=3)	1.575 (n=3)	3.255 (n=3)	3.595 (n=3)	12.138 (n=2)
Llama 3.2 3B Instruct	Metal	0.428 (n=3)	0.717 (n=3)	0.174 (n=3)	0.996 (n=3)	1.201 (n=3)	3.565 (n=2)
SmolLM3 3B	CPU	1.416 (n=3)	2.778 (n=3)	1.321 (n=3)	3.279 (n=3)	3.617 (n=3)	11.439 (n=1)
SmolLM3 3B	Metal	0.604 (n=2)	0.927 (n=2)	0.309 (n=2)	0.997 (n=2)	1.202 (n=2)	3.550 (n=2)

Each cell reports the median in seconds and nominal observations (n). Actual generated answers carry forward. Six turns run back-to-back with no simulated user pause.

Download all per-turn measurements

03 / Method & boundaries

What we measured

M1 / First text: Seconds until the first text callback, including prompt preparation and detokenization. Median across eligible turns. Lower is faster.
M2 / Streaming: Generated tokens per second after generation begins. Replies shorter than eight tokens are excluded. Tokens are not equivalent to words across model tokenizers.
M3 / Process footprint: Whole-process memory sampled every 50 ms, including loading. Median of each wholly nominal conversation's maximum. This is not total CPU-plus-GPU device memory.
M4 / Factual recall: Correct/scorable probes at turns 3, 5 and 6. Strict reply format is scored separately. Three repeats of one conversation do not measure general model intelligence.

A synthetic workshop conversation

The fixture introduces four facts: Birch, Porto, 620 credits and vegan. Later turns add repeated logistics text and request fact recall. The longest recorded prompt was 1,594 tokens. Recall uses messages still inside the context window, not persistent memory beyond it.

Matched within each model

CPU and Metal use identical pinned GGUF bytes, the native chat template, retained prefix cache, greedy decoding, thinking disabled, a 2,048-token context and a 64-token reply cap. Qwen3 1.7B uses Q8_0. The other four models use Q4_K_M.

Temperature affects the headline

Only successful complete conversations contribute data. A timing or recall sample must start and end at nominal thermal state. Memory requires all six turns to stay nominal. All 90 raw recall probes were correct. After thermal exclusions, 82/82 eligible probes were correct. Gemma passed only 3/9 strict-format probes per backend.

Device and build

The personal iPhone 16 (iPhone17,3) ran iOS 26.6.2 build 23G90, with Low Power Mode off. Readiness was confirmed as unlocked, unplugged, cool and at least 60% charged. The signed Release build used Xcode 26.2 build 17C52, SDK 26.2 and llama.cpp commit b2e5e9b28b2484fbf94b543432ece638996a8b97. The five-model batch completed in 33.8 minutes including launches, cache verification and cooling. Downloads were separate.

04 / Separate Firebase pilot

The same workflow runs in the cloud

LFM2.5 1.2B completed one six-turn CPU conversation and one Metal conversation on two Firebase iPhone 16 Pro units. Actual OS: iOS 18.3.2 build 22D82. Both fresh 731 MB model downloads passed hash verification. All twelve turns stayed nominal.

Nominal samples only. Median first text and streaming speed; median of eligible conversation memory peaks.
Model	Backend	M1 First text seconds ↓	M2 Streaming tokens/s ↑	M3 Process peak MiB	M4 Facts correct/scorable	Strict format correct/observed	Nominal turns eligible/planned
LFM2.5 1.2B Instruct	CPU	1.173	40.0	716.8	3/3	3/3	6/6
LFM2.5 1.2B Instruct	Metal	0.367	65.3	723.6	3/3	3/3	6/6

One repetition per backend validates delivery. Separate physical phones were assigned, and cloud power and ambient conditions were not controlled. These results are not pooled with the personal iPhone study or used to isolate hardware effects.

Download Firebase pilot (CSV)

Qwen3 0.6B protocol pilot

The initial pilot completed six conversations and 36 turns on the personal iPhone, all nominal. It finalized scoring before the five-model protocol was frozen.

Nominal samples only. Median first text and streaming speed; median of eligible conversation memory peaks.
Model	Backend	M1 First text seconds ↓	M2 Streaming tokens/s ↑	M3 Process peak MiB	M4 Facts correct/scorable	Strict format correct/observed	Nominal turns eligible/planned
Qwen3 0.6B	CPU	0.633	57.8	407.0	9/9	6/9	18/18
Qwen3 0.6B	Metal	0.200	96.3	400.9	9/9	6/9	18/18

Download Qwen pilot (CSV)

05 / Reproduce & inspect

The numbers are downloadable

The public observations contain every measured turn, including thermal exclusions, generated replies, format outcomes and source-report hashes. Device identifiers, host paths, signing material and private archive locations are omitted.

All 228 turn observations (JSON)Exact six-turn workload Model revisions & hashes Device & cohort metadata Measured native source snapshot Public file checksums

Recalculate the published tables

Download the reproduction bundle, extract it and run the command below. Python 3.9 or later is sufficient, with no third-party packages or network access. It recomputes the medians, exclusions and frozen factual scores and checks every published summary and turn-table value.

python3 reproduce.py

Full signed test bundles and provider logs remain in private storage. The public export is sufficient to audit the reported aggregates, with exact measured native source inputs available separately. A new native run requires the model files, pinned native dependency and an appropriately signed iPhone build.

Protocol: openweights-ios-publication-v1 · Measured 7 October 2026 · Published 8 October 2026