Build or Skip

tiny on-device LLMs

We said MAYBE on July 26, 2026. Not settled — due July 26, 2027.

Read this with the caveat. We tested the engine that produced this verdict against 292 launches whose outcomes we already knew, and could not show it predicted which survived. Some of its data sources were also dead at the time of scoring. The verdict stays up, dated and unedited, because a record you can quietly revise is not a record — but it is worth less than it looked when it was written.

Evidence shows real but shallow demand for on-device LLM tooling — a few low-engagement launches, some Reddit questions, and growing search — with no one clearly monetizing. The gaps are legitimate but require deep ML engineering, and a no-audience solo founder faces a hard distribution and cold-start problem, so this is a marginal BUILD-leaning-SKIP unless attacked via a content/benchmark wedge first.

Was there demand

5out of 10

Real but modest interest: several Show HN launches and Reddit threads exist and search is growing, but engagement is low (33/15/11 points) and no competitor is clearly charging money for a turnkey product.

Could a builder win it

4out of 10

There are genuine gaps (model-picker/benchmark, turnkey SDK) and competition is fragmented/indie-friendly, but the wedge is deep engineering (running/quantizing models across devices) and a solo founder with no audience has no clear path to first 10 users.

The case against this verdict

The lightest wedge here — a benchmark + model-picker content/tool answering 'best model under 1.5GB for X' — is a proven recurring question that a solo dev can rank for organically, turning weak distribution into an SEO play rather than a cold-start problem. If that ranks, it becomes a natural funnel into a paid SDK or model bundles.

Who was already there

  • Google AI Edge (LiteRT-LM) — Enterprise/framework-focused, steep learning curve, not a turnkey consumer product; docs assume ML engineering skills
  • On-Device LLMs: State of the Union 2026 (v-chandra) — Educational/reference content, not a product or tool; no direct utility for end users
  • Hugging Face (Tiny LLM models hub) — Fragmented model discovery, no unified on-device deployment story, users must self-integrate
  • LocalLLaMA (Reddit community) — Community discussion only, no product; advice is scattered and fast-moving
  • Private AI App experiments (LinkedIn/YouTube creators) — Early-stage, work-in-progress, no polished shipping product; individual efforts

Other verdicts

What is worth more than this page. The register records what became of 6,266 real launches. No engine has to be right for that to be true.