Cheaper AI for Your App: Claude Haiku vs. GPT Nano vs. Gemini vs. On-Device vs. Self-Hosted (October 2026)

We’ve been running a mobile app on the Claude API, and the monthly bill grew faster than the user base. That’s a familiar story: you prototype on the best model, it works, you ship it, and six months later you’re paying frontier prices for requests that mostly summarize a paragraph or classify a note. This post is the comparison we did before deciding what to change.

The short version surprised us. The cheapest fix was not switching providers. It was a one-line model change that cut the bill by more than 95%, followed by two call-pattern fixes that cut what remained in half again. Switching providers, self-hosting, and on-device models all have their place, but they come after that.

All prices below are USD per million tokens, input / output, checked against official pricing pages on October 26, 2026. This market reprices constantly; verify before you budget.

The Baseline: What a Request Costs

To compare options honestly you need a workload, not a price list. Ours is typical of a mobile app with an AI feature:

• 100,000 requests per month
• 1,500 input tokens per request, of which about 1,000 is a fixed system prompt
• 300 output tokens per request

That’s 150M input tokens and 30M output tokens a month. Here’s what that costs on each option, with no caching or batching applied yet:

Option	Input / Output price	Monthly cost
Claude Opus 5.5	$4 / $20	$1,200
Claude Sonnet 5.5	$2 / $10	$600
Claude Haiku 4.5	$1 / $5	$300
Gemini 3.1 Flash-Lite	$0.25 / $1.50	$83
DeepSeek Flash (peak hours)	$0.30 / $1.20	$81
Claude Haiku 5.5	$0.10 / $0.50	$30
GPT-6 Luna	$0.10 / $0.50	$30
Together, Llama 3 8B Lite	$0.14 / $0.14	$25
GPT-5 nano	$0.05 / $0.40	$20
Groq, Llama 3.1 8B	$0.05 / $0.08	$10
Self-hosted L40S on RunPod, always on	~$0.79/hour	~$580 flat
On-device (Apple, Gemini Nano, Gemma)	$0	$0

If you’re on Opus or Sonnet today, the first row to look at is Haiku 5.5. Everything below it on the table saves you at most another $30 a month at this volume, and each of those options costs you something else: a migration, a quality re-evaluation, or a platform constraint.

Option 1: Stay on Claude, Drop a Tier

Anthropic’s budget model changed the math this year. Haiku 4.5 was priced like a premium small model at $1/$5. Haiku 5.5 lists at $0.10/$0.50 for prompts up to 100K tokens, a 10x cut, and sits at the same price as OpenAI’s GPT-6 Luna. Cache reads on Haiku 5.5 are $0.01 per million, and the Batch API halves everything.

One trap: prompts over 100,000 tokens are billed at $0.50/$2.50, five times the base rate, for the whole request. A mobile app shouldn’t be sending prompts that large anyway, but if you stuff a document or a long chat history into context, watch the threshold.

Pros

• Same SDK, same tool-use and structured-output API, same prompt conventions. Changing model is the whole migration.
• Keeps thinking, 1M context, tool calling, and the rest of the current Claude feature set.
• Your evals and prompts were tuned on Claude; the behavioral drift is smaller than a provider switch.

Cons

• A small model is a small model. Multi-step reasoning, nuanced tone, and long-document tasks will degrade. Run your eval set before you flip the switch for everything.
• Haiku 5.5 has no server-side refusal fallback, so if a safety classifier declines a request, you handle the refusal stop reason yourself.

Verdict: Do this first. For most app features (summarize, classify, extract, rewrite, short chat) Haiku 5.5 is indistinguishable from Sonnet to users, and it’s 95% cheaper.

Option 2: Fix How You Call, Not Who You Call

These levers work on every provider and are usually worth more than the provider choice.

Prompt caching. In our workload, 1,000 of the 1,500 input tokens are a fixed system prompt. Cached reads bill at 10% of the input price on Anthropic, OpenAI, and Google (5% on Opus 5.5 and Sonnet 5.5, 2% on DeepSeek). With the system prompt cached, Haiku 5.5 drops from $30 to about $21 a month, and Sonnet 5.5 drops from $600 to about $410. Caching is prefix-based: put the stable content first and anything that varies per request (timestamps, user IDs) after it, or the cache silently never hits. Check cache_read_input_tokens in the usage response; if it’s zero, something in your prefix is changing.

Batch for anything async. Anthropic, OpenAI, and Google all take 50% off for batch jobs. Nightly digests, bulk enrichment, pre-generated content: none of it needs a real-time response.

Cap output and lower effort. Output tokens cost 5x input on most providers. Set max_tokens to what the feature actually needs, ask for terse responses in the prompt, and on Claude’s current models set output_config.effort to low for simple tasks. Thinking tokens bill as output; for a classifier you don’t want the model thinking at all.

Shrink the prompt. Every token of system prompt is paid on every request. Few-shot examples that were needed on a 2024 model are often dead weight on a 2026 model. Measure.

Pros: Provider-agnostic, no quality loss, usually a day of work. Cons: Requires actually reading your usage data. Caching has a minimum prefix size (model-dependent, 512 to 4,096 tokens), so very short system prompts won’t cache.

Verdict: Do this second, and do it regardless of which provider you end up on.

Option 3: Switch to Another Provider’s Budget Tier

If you’ve done the first two and still need to cut, or you want a free tier, this is where the other hosted providers come in.

Provider	Model	Input / Output	Cached input	Notes
OpenAI	GPT-5 nano	$0.05 / $0.40	$0.005	Cheapest hosted frontier-lab model. Flex tier halves it again.
OpenAI	GPT-6 Luna	$0.10 / $0.50	$0.01	Newer and stronger than nano at Haiku 5.5’s price.
Google	Gemini 3.1 Flash-Lite	$0.25 / $1.50	$0.025	Free tier available without a billing account.
Google	Gemini 3.8 Flash	$0.75 / $3.75	$0.075	Promo price; doubles on January 1, 2027.
DeepSeek	DeepSeek Flash	$0.30 / $1.20	$0.006	Half price off-peak (most of the US day). Data processed in China.
xAI	grok-build-0.1	$1.00 / $2.00	$0.20	xAI no longer has a sub-dollar tier.

Two things stand out. First, Anthropic and OpenAI now have matched budget pricing at $0.10/$0.50, so the choice between Haiku 5.5 and GPT-6 Luna is about quality on your task, not cost. Second, Google’s budget tier is no longer the cheapest; Gemini 2.5 Flash-Lite at $0.10/$0.40 is being shut down this month, and its replacements cost 2.5x to 3x more.

Gemini’s special case for mobile. Firebase AI Logic lets a mobile app call Gemini directly without a backend, with the API key protected by Firebase App Check instead of being shipped in the binary. The SDK is free, there’s a free tier that doesn’t require a billing account, and it has first-party Flutter support. App Check becomes mandatory on November 2, 2026. For an app with no backend at all, this is the lowest-friction path to a hosted model.

Pros

• GPT-5 nano is the absolute floor among frontier labs at this volume.
• Gemini’s free tier can cover a small app entirely.
• Multiple providers means leverage and a fallback.

Cons

• A provider switch is a migration: different SDK, different tool-calling shape, different structured-output API, different prompt quirks. Budget a week, not an hour.
• Your evals were built on Claude’s behavior. Re-run all of them.
• Model churn is real on every provider. Gemini 2.5 models shut down this month; OpenAI’s nano/mini lineup has turned over three times in eighteen months. Pin model IDs and watch deprecation notices.
• DeepSeek’s pricing is excellent, but if your users’ data has any residency or compliance requirement, it’s off the table.
• The savings over Haiku 5.5 at our volume are $0 to $10 a month. The engineering cost of the switch eats that for years.

Verdict: Worth it if you need Gemini’s free tier for a zero-budget app, want Firebase’s no-backend path, or have a specific task where another model measurably wins. Not worth it purely on price if you’re already on Haiku 5.5.

Option 4: Open-Weight Models on an Inference Provider

Groq, Together, Fireworks, DeepInfra, and others host open-weight models (Llama, Qwen, Gemma, gpt-oss, DeepSeek) and sell them per token. OpenRouter fronts all of them with a single API and passes through provider pricing plus a 5.5% fee on credit purchases.

Representative prices as of this month:

Provider	Model	Input / Output
Groq	Llama 3.1 8B	$0.05 / $0.08
Together	Llama 3 8B Lite	$0.14 / $0.14
Together	Qwen3.5 9B	$0.17 / $0.25
Together, Groq, Fireworks	gpt-oss-120B	$0.15 / $0.60
Together	Qwen3 235B (A22B)	$0.20 / $0.60

Pros

• The cheapest per-token pricing anywhere, especially on output, which is where frontier labs mark up hardest.
• Groq’s latency is in a class of its own, which matters for interactive features.
• No lock-in to a lab’s roadmap. The same weights are available from several providers, and you can self-host later.
• OpenAI-compatible APIs almost everywhere, so switching between them is a base URL change.

Cons

• An 8B model is noticeably weaker than Haiku 5.5 or GPT-6 Luna on anything requiring judgment. The 120B and 235B models close the gap but cost about the same as the frontier labs’ budget tier, so the price advantage mostly evaporates.
• Tool calling and structured output reliability vary by provider and model. The cheap tier is where it’s flakiest.
• Rate limits and uptime are less predictable than at the big labs. Keep a fallback.
• OpenRouter’s convenience is real but adds a hop and a fee.

Verdict: The right choice for a high-volume, simple task (classification, tagging, short rewrites) where a small model is good enough and output volume is large. Benchmark the specific model on your eval set first.

Option 5: Self-Host

Rent a GPU, run vLLM or similar, serve an open-weight model yourself. On RunPod an L40S (48GB) is about $0.79 an hour on a pod, an A100 80GB about $1.19. Serverless (scale to zero) runs roughly twice that per active hour, with 6 to 12 second cold starts.

The important number is the floor. An always-on L40S is about $580 a month before you serve a single request. At our baseline workload, that’s 20x the Haiku 5.5 bill. The crossover where self-hosting beats per-token pricing is somewhere above several million requests a month, and only if someone on your team wants to own GPU capacity planning, model updates, and on-call.

Pros

• Full control over the model, data, and latency. No provider deprecations, no content policies but your own.
• At very high volume, the cheapest option per token.
• Data never leaves infrastructure you control.

Cons

• A fixed monthly floor that dwarfs the variable cost of a budget API at small or medium scale.
• You own uptime, scaling, security patches, and model upgrades.
• Serverless GPU cold starts are too slow for an interactive mobile feature without a warm pool, which brings back the fixed cost.

Verdict: Not for a mobile app at anything under millions of requests a month. If you’re there, you already know it.

Option 6: On-Device Models

This is the only option where the marginal cost is actually zero, and it’s the one that’s changed most in the last year.

Apple Foundation Models framework. iOS 26 ships a roughly 3B-parameter on-device model with a Swift API: guided generation into typed structs, tool calling, streaming. It’s free, works offline, and user data never leaves the phone. The constraints are hard ones: a 4,096-token context window shared between instructions, prompt, response, and tool definitions; Apple Intelligence-capable hardware only (iPhone 15 Pro and later); and iOS 26 and later. iOS 26.4 added contextSize and tokenCount(for:) so you can budget tokens at runtime instead of guessing.

Gemini Nano via ML Kit. Google’s equivalent on Android. The Prompt API is in beta, runs through AICore, and is free. It performs best on Pixel 10 and supports a short list of recent flagships from Samsung, Xiaomi, and others. The device list is the main limitation; a large share of Android users won’t have it.

Bring your own model. Gemma 3n, Qwen, and Llama small models run on both platforms through llama.cpp, MLC, or MediaPipe’s LLM Inference API. This works on far more devices than the first-party options, at the cost of shipping a 1 to 4GB model with your app (or downloading it on first run), managing memory, and tuning per-device.

Pros

• Zero per-request cost. The only option that scales to infinite users for free.
• Offline, private, no latency to a server, no API key to protect.
• Apple’s framework in particular is well-designed: typed output and tool calling in a few lines of Swift.

Cons

• Quality is a 3B model’s quality. Good for summarizing a note, extracting a date, rewriting a sentence, or classifying intent. Not good for anything that needs real reasoning or world knowledge.
• Device fragmentation. Apple covers roughly the newest three iPhone generations; Gemini Nano covers fewer Android devices than that. You need a cloud fallback anyway.
• A 4K context rules out long documents or long conversations.
• Flutter has no first-party bridge to either framework. You write platform channels to Swift and Kotlin, or use a community plugin and accept its maintenance risk.
• Battery and thermal cost on the user’s device, which they notice.

Verdict: Excellent as the first tier of a hybrid, bad as the only tier. Any feature simple enough for a 3B model should try the device first and fall back to the cloud.

What We’d Actually Do

The options aren’t exclusive. The architecture that minimizes cost without wrecking quality is a router:

enum Tier { device, budget, standard }

Tier pickTier(Task task, {required bool deviceModelAvailable}) {
  if (task.isSimple && task.inputTokens < 3000 && deviceModelAvailable) {
    return Tier.device;          // $0, offline, private
  }
  if (task.needsDeepReasoning) {
    return Tier.standard;        // Sonnet 5.5 for the hard 5%
  }
  return Tier.budget;            // Haiku 5.5 (or GPT-6 Luna / GPT-5 nano)
}

Applied to our baseline, if 40% of requests go on-device, 55% to Haiku 5.5 with the system prompt cached, and 5% to Sonnet 5.5, the monthly bill is about $35 instead of $600 on Sonnet or $1,200 on Opus. The router is maybe fifty lines of code plus the platform channels for the on-device tier.

Two rules whatever you pick:

1. Never ship an API key in the app. Every hosted option above should be called through your backend or, for Gemini, through Firebase AI Logic with App Check. A key in the binary is a key on the internet within a week.
2. Run your evals before and after every change. “Cheaper” only counts if users can’t tell. Twenty representative prompts and a pass/fail rubric is enough to catch a bad downgrade.

Decision Table

If you…	Do this
Are on Opus or Sonnet and haven’t tried a smaller model	Switch to Haiku 5.5. Measure.
Resend a long system prompt every call	Enable prompt caching today.
Have any async workload	Move it to the Batch API.
Need a free tier for a zero-revenue app	Gemini Flash-Lite via Firebase AI Logic.
Run a high-volume simple task and need the absolute floor	GPT-5 nano, or Llama 8B on Groq.
Need offline, private, or zero marginal cost	On-device first, cloud fallback.
Serve millions of requests a month and have an infra team	Consider self-hosting. Otherwise don’t.

Key Takeaways

• Haiku 5.5 at $0.10/$0.50 cuts a Sonnet bill by 95% and an Opus bill by 97.5% with a one-line change. Try it before anything else.
• Prompt caching and batching are provider-agnostic and typically worth another 30% to 50%.
• Anthropic and OpenAI budget tiers are now priced identically; pick on quality for your task. Google’s budget tier got more expensive this year, and xAI has no budget tier at all.
• Open-weight models on Groq or Together are the per-token floor, but only the small ones are meaningfully cheaper than Haiku 5.5, and they’re meaningfully weaker.
• Self-hosting has a fixed floor around $580 a month and only wins at scale most apps never reach.
• On-device models are free and private, but a 3B model with a 4K context, on a fraction of devices. Use them as the first tier of a router, not the only one.

Resources

• Anthropic pricing
• OpenAI API pricing
• Gemini API pricing
• Firebase AI Logic pricing and App Check deadline
• DeepSeek pricing
• xAI models and pricing
• Together AI pricing
• RunPod GPU pricing
• Apple Foundation Models framework
• ML Kit GenAI Prompt API
• Our July 2026 API pricing snapshot