Build or Skip

on-device LLM tooling

We said MAYBE on August 5, 2026. Not settled — due August 5, 2027.

Read this with the caveat. We tested the engine that produced this verdict against 292 launches whose outcomes we already knew, and could not show it predicted which survived. Some of its data sources were also dead at the time of scoring. The verdict stays up, dated and unedited, because a record you can quietly revise is not a record — but it is worth less than it looked when it was written.

Demand is real and multi-channel but skews toward infra curiosity rather than proven willingness to pay. Winnable wedges exist, but they sit in the hardest-to-build corner of the stack for a solo founder—this is a plausible BUILD only if the founder has real on-device ML expertise; otherwise lean SKIP.

Was there demand

6out of 10

Multiple channels agree: growing search demand with commercial keywords, five real competitors (some charging/monetizing), and social traction (OWhisper 289 pts, 160-upvote Reddit test). But most demand is developer-curiosity and infra-adjacent rather than an urgent unmet paid need.

Could a builder win it

5out of 10

Clear wedges exist (cross-platform single API, React Native/Expo support, on-device observability) against incumbents that each leave gaps, but the engineering is genuinely hard—quantization, multi-platform native runtimes, mobile fragmentation—and some incumbents (Apple, HuggingFace) are giants.

The case against this verdict

The named wedges are exactly the parts that require deep ML-systems engineering (quantization, native runtimes, cross-platform inference)—the areas where a solo founder is slowest and where well-funded players (Apple, Ollama, HF) are already iterating fast. A thin abstraction layer over other people's runtimes is easily commoditized, and much of the 'demand' is free-tool curiosity, not budget.

Who was already there

  • Apple On-Device LLM (Foundation Models)Apple ecosystem lock-in; limited to iOS/macOS; closed proprietary model; requires developer account; less flexibility for custom models
  • OllamaPrimarily desktop-focused; limited mobile support; steep learning curve for non-technical users; minimal UI/tooling layer
  • LM StudioDesktop-only; no mobile tooling; generic UI; limited API ecosystem; smaller community than Ollama
  • TensorFlow Lite + MediaPipeSteep technical barrier; requires ML expertise; verbose boilerplate; documentation scattered across modules; Android/iOS fragmentation
  • Hugging Face Transformers.jsBrowser/JS only; limited model selection for on-device; poor performance on larger models; immature ecosystem

Other verdicts

What is worth more than this page. The register records what became of 6,266 real launches. No engine has to be right for that to be true.