Skip to content
, ,

How to Run an LLM on Your Phone Offline: Android and iPhone Guide (2026)

8–12 minutes
Woman with a long braid winking in a night-time high-rise office, holding a white card that reads PocketPal, with the text Offline AI on your phone

You can run a real AI chatbot on your phone with no internet, no account and no data leaving the device. Free apps like PocketPal AI download a small language model once, and then it works offline: on a plane, in the mountains, or anywhere you do not want your chats on someone else’s server.

This guide shows you exactly how to do it on Android and iPhone in about 10 minutes: which app to use, which model fits your phone’s RAM, the settings that make it fast, what speed to expect, voice chat, and fixes for crashes and slow replies. Model and app details are current as of November 2026.

TL;DR

  • Easiest app: PocketPal AI (free, open source, Android and iPhone). Runs GGUF models with llama.cpp.
  • Pick a model by RAM: 4 GB phone = 0.5 to 1B model, 6 GB = 1 to 2B, 8 GB = 3 to 4B (best balance), 12 GB+ = 4 to 8B.
  • Good picks: Gemma 3 1B (small phones), Qwen3 1.7B (8 GB daily driver), Gemma 4 E2B/E4B, Qwen3 4B or Phi-4-mini (flagships).
  • Use Q4_K_M quantization and a 2,048 to 4,096 token context.
  • Expect about 20 to 40 tokens per second for 1B models on a recent flagship, and 5 to 10 for 7B.
  • Limits: no frontier-level quality, short context, and it gets warm and drains battery on long chats.

Why Run an LLM on Your Phone?

  • Privacy: prompts and documents stay on the device.
  • Offline: works with no signal, on flights and while travelling abroad.
  • Free: no subscription and no usage limits.
  • Always available: no outages, no “server busy”.
  • Learning: a fun way to understand how local AI works before setting it up on a PC.

What You Need

Item Minimum Recommended
RAM 4 GB (tiny models only) 8 GB or more
Free storage 2 GB 5 to 10 GB to try several models
Chip Any recent 64-bit phone Snapdragon 8 Gen 3 / 8 Elite, Dimensity 9400+, Tensor G4+, Apple A17 Pro or newer
Internet Only to download the app and model once Wi-Fi for the download

Your phone shares RAM with the system, so 2 to 4 GB is already in use. Free RAM, not total RAM, decides whether a model loads.

Best Apps for Local AI on a Phone

App Platform Engine Best for
PocketPal AI Android, iPhone, iPad llama.cpp (GGUF) Beginners; any GGUF model from Hugging Face; voice and tools
Google AI Edge Gallery Android 12+, iOS LiteRT Trying Google’s phone-first Gemma models
LM Playground Android llama.cpp, ARM-tuned Fast one-tap loading
Maid Android GGUF runner Advanced users who want more control
MLC Chat Android, iPhone MLC-LLM Curated model list (updates have slowed)
WebLLM Chrome on Android WebGPU Zero install demo in a browser tab

PocketPal is free and MIT-licensed. Chat, models, voices and tools are free with no account. An optional one-time “PocketPal Plus” purchase in the app stores adds hands-free voice conversation and a local API server.

Which Model Fits Your Phone

Rule of thumb: a 4-bit model needs a little more than half its parameter count in GB, plus room for context.

Phone RAM Model size Good picks
4 GB 0.5 to 1B Gemma 3 1B, Qwen3 0.6B, LFM2 small variants
6 GB 1 to 2B Qwen3 1.7B, SmolLM2 1.7B, Gemma 3n E2B
8 GB 3 to 4B (best balance) Qwen3 1.7B as a fast daily driver, Gemma 4 E2B, Qwen3 4B
12 GB 4 to 8B Qwen3 4B, Phi-4-mini (3.8B), Gemma 4 E4B; 7-8B only with short context
16 GB+ 7 to 8B 8B models work but run slower and hotter
Model Size at 4-bit Why pick it
Gemma 3 1B ~0.7 to 1 GB Runs on 4 GB phones, feels instant
Qwen3 1.7B ~1.2 GB Best small all-rounder, optional thinking mode
Qwen3 4B ~2.8 GB Smarter, thinking mode for harder questions
Phi-4-mini (3.8B) ~2.3 GB Strong reasoning for its size
Gemma 4 E2B / E4B Varies Google’s current phone-first models (March 2026)
LFM2 (350M to 2.6B) ~0.3 to 1.6 GB Built for phone CPUs, fast for its size

For a deeper dive into how memory limits work, see our how much VRAM do I need for AI guide; the same maths applies to phone RAM.

Step by Step: PocketPal on Android or iPhone

  1. Install PocketPal AI from Google Play or the App Store (developer: a-ghorbani / PocketPal).
  2. Connect to Wi-Fi for the one-time model download.
  3. Open the menu (☰) > Models.
  4. Pick a model that fits your RAM from the table above and tap Download. Or tap + to search Hugging Face for any GGUF model.
  5. Choose Q4_K_M when asked for a quantization. It is the best balance of size and quality.
  6. Tap Load once the download finishes (or use the chevron next to the chat box).
  7. Turn on airplane mode to prove it works offline.
  8. Start chatting. The first reply is slower while the model loads into memory.

Gated models (like some Gemma releases) need a free Hugging Face access token: create one in your Hugging Face account, then in PocketPal go to Settings > Set Token.

Settings for Speed and Quality

Setting Recommended Why
Quantization Q4_K_M Smallest size without a big quality drop
Context length 2,048 to 4,096 tokens Bigger contexts cause memory spikes and crashes
GPU / NPU acceleration On if offered Metal on iPhone, OpenCL on Snapdragon Adreno, Hexagon NPU on supported Qualcomm chips
Max reply length 256 to 512 tokens Long replies heat the phone and slow down
Temperature 0.6 to 0.8 Balanced; lower for facts, higher for creative writing
Thinking mode (Qwen3) Off for chat, on for maths Thinking uses many more tokens

How Fast Is It? Real Speeds

Model (Q4) Phone Approx. speed Feel
Gemma 3 1B Recent flagship 25 to 40 tok/s Instant for short answers
Llama 3.2 1B iPhone 16 Pro class 25 to 40 tok/s Smooth
~1.5 to 1.7B (Qwen) iPhone 16 Pro class 20 to 30 tok/s Responsive
3 to 4B Flagship 15 to 23 tok/s, then throttles Slows on long replies
7B 12 GB+ flagship 5 to 10 tok/s Usable but sluggish

For comparison, people read at about 5 to 8 words per second, so anything above 10 tokens per second feels fine.

Google reports much higher speeds on NPUs with its own LiteRT builds (for example Gemma 3n E2B at about 28 tokens per second on a Dimensity 9500 NPU), but generic GGUF apps usually run on the CPU or GPU. Use PocketPal’s Benchmark page to test your own phone.

Voice, Pals, Tools and Benchmarks

  • Voice input: dictate with on-device Whisper (whisper.cpp).
  • Text-to-speech: on-device neural voices such as Kokoro, Kitten, Supertonic and Phonon, or your phone’s system voices.
  • Hands-free voice chat: part of PocketPal Plus in store builds.
  • Pals: custom assistants with their own model, system prompt and personality. Browse community Pals on PalsHub.
  • Tools: calculator, date and time, rich HTML, and opt-in web search and page reading.
  • Benchmark: measure tokens per second and memory, and optionally share to the AI Phone Leaderboard.

Built-In Phone AI: Gemini Nano and Apple Intelligence

Many phones already include an on-device model:

  • Gemini Nano on recent Android flagships (through AICore) powers system features. It is not a general chat app and you cannot swap the model. Gemma 4 is the base for Gemini Nano 4. See Pixel 11 AI features and Galaxy AI in One UI 9.
  • Apple Foundation Models: an about 3B on-device model on Apple Intelligence iPhones, used by apps and system features. See our iOS 27 Apple Intelligence guide.

Use these for built-in features; use PocketPal when you want to choose your own model.

Use Your PC’s AI From Your Phone

A phone tops out around 4 to 8B models. If you have a gaming PC, run a bigger model there with Ollama and chat to it from your phone over your home Wi-Fi using a client app or Open WebUI in the browser.

You get 14B to 30B+ quality with the phone as a screen. Start with our Ollama guide and run DeepSeek locally.

What a Phone LLM Is Good (and Bad) At

Good at Bad at
Quick questions and explanations Up-to-date facts (no internet unless you enable search)
Rewriting and grammar fixes Long documents and big files
Short drafts, messages and captions Complex coding and maths
Offline translation of simple text Frontier-level reasoning
Brainstorming and travel help Always-on background assistant (Android kills it)
Private journaling and notes Reliable OCR or vision from small models

Small models can sound confident while being wrong. Double-check facts, health and money information.

Privacy: Is It Really Offline?

With PocketPal, chats and documents stay on the device unless you turn on an online feature: web search and page reading, a remote server, or an online system voice. Other network use (model downloads, PalsHub, purchases, an opt-in leaderboard, feedback) does not attach your chats, according to the project’s README.

For any app, check the store’s Data safety or privacy label, the permissions it asks for, and whether it works in airplane mode. That is the simplest real test.

Troubleshooting

Problem Fix
App crashes when loading a model Model too big for free RAM: close other apps, pick a smaller model or lower quantization
Very slow replies Use a 1 to 2B model, enable GPU, shorten context and max reply length
Phone gets hot Shorter replies, smaller model, remove case, avoid charging while chatting
Download fails Use Wi-Fi, check storage, retry; for gated models set a Hugging Face token
Gibberish output Wrong chat template or broken quant: try another GGUF upload of the same model
Model forgets earlier chat Context too short: start new chats for new topics
Answers stop mid-sentence Raise max reply length a little

My Take

Running AI on your phone is now genuinely useful, not just a party trick. On an 8 GB phone, Qwen3 1.7B or Gemma 4 E2B in PocketPal handles everyday rewriting, quick questions and offline help well.

Keep expectations sensible: for serious work, use a cloud chatbot or your own PC. If you get hooked, the next step is a desktop setup with llama.cpp or Ollama.

FAQ

Can I run ChatGPT offline on my phone?

No, ChatGPT needs the internet. You can run open models like Gemma, Qwen or Phi offline with apps such as PocketPal.

Is PocketPal AI free?

Yes. It is free and open source. An optional one-time Plus purchase adds hands-free voice and a local API server in store builds.

How much RAM do I need to run AI on a phone?

4 GB runs tiny 1B models. 8 GB is the sweet spot for 3 to 4B models. 12 GB or more can run 4 to 8B models with short context.

What is the best AI model for a phone?

For most 8 GB phones, Qwen3 1.7B or Gemma 4 E2B. For 12 GB flagships, Qwen3 4B, Phi-4-mini or Gemma 4 E4B.

Does running AI drain the battery?

Yes, especially with larger models and long replies. Short chats with small models use far less.

Can I run AI on an iPhone?

Yes. PocketPal AI, Google AI Edge Gallery and MLC Chat work on iPhone, and newer iPhones also include Apple’s own on-device model.

Is local phone AI private?

Yes if the app runs fully on-device and you leave online features off. Test it in airplane mode.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Woman with a long braid winking in a night-time high-rise office, holding a white card that reads PocketPal, with the text Offline AI on your phone