Unlimited OCR 2: Run Baidu’s 3B Long-PDF OCR Model Locally (VRAM, GGUF, SGLang, Benchmarks)

Baidu’s Unlimited OCR 2 is a tiny 3B open model that turns whole PDFs, even 100 to 300 pages long, into clean text in one go. It is MIT licensed, already has GGUF and MLX builds, and it is one of the most downloaded OCR models on Hugging Face right now.

The research paper landed on 2 November, with benchmark tables that put it far ahead of much bigger models on long documents. In this guide I cover what it is, the real numbers, the hardware you need, and three ways to run it locally: Python, an SGLang server and a patched llama.cpp.

What Is Unlimited OCR 2?

Unlimited OCR 2 comes from Baidu’s ERNIE team. It is the second version of Unlimited OCR, which launched on 22 June 2026 as an extension of DeepSeek-OCR.

The first version could only take about 40 pages per request. Version 2 removes that ceiling with a new attention design the paper calls Sparse Reference Sliding Window Attention (SR-SWA).

The idea copies how a person transcribes a book. You look at the page you are typing, not every earlier page. SR-SWA caps the active reference tokens at a fixed size (for example 512), so each decoding step costs about the same on page 3 as on page 300.

What changed from version 1

• Page limit: about 40 pages before, effectively unlimited now.
• Speed at 100+ pages: 7.0x faster per step than full attention and 5.6x faster than v1’s R-SWA.
• Memory: peak KV cache falls from 47.00 GB to 1.77 GB at batch 1 with the new decoding mode.
• Throughput: 360 to 1,315 tokens per second at batch 16.
• Single-page quality: about the same (OmniDocBench 93.92 to 94.02).

All figures are from Baidu’s paper (arXiv 2610.39308), so treat them as vendor-reported.

Specs at a Glance

Benchmarks: Long PDFs and Single Pages

Baidu tested three ways: medium documents, very long documents and single pages. The long-document gap is the reason this model is trending.

MPDocBench (up to 30 pages)

MarathonDocBench (103 to 300 pages)

Note: Baidu built MarathonDocBench itself, so expect it to suit its own model. On the neutral single-page OmniDocBench v1.6 it scores 94.02, behind small specialists like TeleOCR (96.91) and OvisOCR2 (96.47).

Sub-scores on MPDocBench are also strong: table structure (T-TableTEDS) 89.54 and reading order edit distance 0.077, both the best in Baidu’s table.

How Much VRAM Do You Need?

Baidu only lists its test GPU, an 80 GB A800. The sizes below come from the file sizes plus typical overhead, so they are my estimates, not official minimums.

Not sure what your card has free? My guide to checking GPU VRAM usage shows the commands for Windows, Linux and Mac.

Try It Online First

Before installing anything, test your own documents in the free Hugging Face Space at huggingface.co/spaces/baidu/Unlimited-OCR-2. Upload a page or PDF and compare the output with what you need.

Baidu also offers it as a paid service on Baidu Cloud, and the model page links Colab and Kaggle notebooks if you want a free cloud GPU.

Warning: Do not upload contracts, IDs or medical files to a public demo. Running locally is the whole point for private documents.

Run It Locally With Python (Transformers)

This is the simplest local route on an NVIDIA GPU. Baidu tested it on Linux with Python 3.10 and pinned versions, and pinning matters because the model uses custom code.

1. Create a clean environment. python3.10 -m venv uocr && source uocr/bin/activate (on Windows: uocr\Scripts\activate).
2. Install the pinned packages.

   pip install torch==2.13.0 torchvision pillow transformers==4.52.1 \
     huggingface-hub==0.36.0 tokenizers==0.21.4 addict==2.4.0 \
     einops==0.8.2 easydict==1.13

3. Turn your PDF into page images. Baidu’s helper uses PyMuPDF at 300 DPI: pip install pymupdf, then save each page as page_1.jpg, page_2.jpg and so on.
4. Load the model.

   import torch
   from transformers import AutoModel, AutoTokenizer
   
   model_name = "baidu/Unlimited-OCR-2"
   tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
   model = AutoModel.from_pretrained(model_name, trust_remote_code=True,
       use_safetensors=True, torch_dtype=torch.bfloat16).eval().cuda()

5. Parse one page (uses detailed tiling, “gundam mode”):

   text = model.infer(tokenizer, image_file="your_doc/page_1.jpg",
       output_path="output/single", save_results=True)

6. Parse a whole document in one request (one global view per page, “base mode”):

   pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]
   text, num_tokens = model.infer_multi(tokenizer, image_files=pages,
       output_path="output/multi", save_results=True)

The first run downloads about 5.9 GB to your Hugging Face cache (~/.cache/huggingface/hub/ on Linux and Mac, C:\Users\<you>\.cache\huggingface\hub\ on Windows).

Tip: Use infer for dense single pages like invoices or forms, and infer_multi when cross-page context matters, such as tables that continue onto the next page.

Serve Whole PDFs With SGLang

For batch jobs or an OpenAI-style API, Baidu recommends SGLang. It needs Baidu’s custom SGLang wheel from the GitHub repo, not the normal pip package.

1. Clone the repo and create a uv environment.

   git clone https://github.com/baidu/Unlimited-OCR && cd Unlimited-OCR
   uv venv --python 3.10 && source .venv/bin/activate
   uv pip install wheel/sglang-0.5.20.dev*.whl
   uv pip install pymupdf==1.27.2.2 fastokens==0.3.2

2. Set the memory variables.

   export SGLANG_UOCR_EANABLE_DOC_LIMIT=1
   export SGLANG_UOCR_DOC_GUNDAM_PAGE_WINDOW=10
   export SGLANG_UOCR_DOC_BASE_PAGE_WINDOW=10
   export SGLANG_UOCR_MAX_PAGE_TOKENS=20000
   export SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64
   export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

3. Start the server.

   python -m sglang.launch_server --model-path baidu/Unlimited-OCR-2 \
     --served-model-name Unlimited-OCR-2 --chat-template unlimited-ocr \
     --port 10000 --mem-fraction-static 0.7 --max-prefill-tokens 2048 \
     --disable-radix-cache

4. Send documents to http://127.0.0.1:10000/v1/chat/completions with the repo’s client script, which handles PDF to 300 DPI images, page tokens and the anti-repeat logit processor.

Note: EANABLE is misspelled in Baidu’s own code, so type it exactly as shown. The Hugging Face page also shows a generic vllm serve snippet, but vLLM support was documented for version 1; use SGLang for v2’s long-document mode.

Run the GGUF With Patched llama.cpp

Community GGUFs from AtomicChat/Unlimited-OCR-2-GGUF and RX5950XT/Unlimited-OCR-2-GGUF make this run on small GPUs and CPUs. The catch: mainline llama.cpp cannot load them, so you build a patched version.

1. Build the patched llama.cpp (drop -DGGML_CUDA=ON without an NVIDIA GPU; Macs use Metal automatically):

   git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp && git checkout b11913
   hf download RX5950XT/Unlimited-OCR-2-GGUF llama.cpp-patch/unlimited-ocr2.patch --local-dir .
   git apply llama.cpp-patch/unlimited-ocr2.patch
   cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j --target llama-mtmd-cli llama-server

2. Download the model and vision file (Q8_0 plus mmproj) from the AtomicChat repo into the same folder.
3. OCR one page from the command line.

   ./build/bin/llama-mtmd-cli -m Unlimited-OCR-2-Q8_0.gguf \
     --mmproj mmproj-Unlimited-OCR-2-F16.gguf --jinja --temp 0 -c 32768 -n 8000 \
     --image page.png -p "<PAGE><__media__>Multi page parsing."

4. Or run a local API server with one slot: ./build/bin/llama-server -m Unlimited-OCR-2-Q8_0.gguf --mmproj mmproj-Unlimited-OCR-2-F16.gguf --jinja -np 1 -c 32768.

The GGUF author measured only single pages: about 2.9% mean character difference from Baidu’s own code, put down to BF16 rounding. Multi-page parsing through llama.cpp is untested. New to quant names? See MLX vs GGUF.

Mac and CPU Options

• Apple Silicon: mlx-community/Unlimited-OCR-2-4bit, -6bit and -8bit appeared on 6 November. The patched llama.cpp build also runs on Metal.
• PyTorch on Mac: GitHub issue #81 reports MPS crashes mid-generation, so avoid the transformers route on Mac for now.
• CPU only: the same issue says CPU works after removing the hard-coded .cuda() call. The Q4_K or Q8_0 GGUF on CPU is the easier path.

Unlimited OCR 2 vs Other OCR Models

Source: all rows from Baidu’s paper tables. Pick Unlimited OCR 2 for long reports, books and scanned archives. Pick a smaller single-page specialist for receipts and forms. For document Q&A rather than transcription, a general vision model like MiniCPM-V 4.7 is the better tool.

Troubleshooting

Output repeats the same line forever

This is issue #82, reported with the llama.cpp server. Use --temp 0 and an output cap such as -n 8000. With SGLang, keep the repo’s no-repeat n-gram processor on (n-gram 35, window 1024 for multi-page, 128 for single).

CUDA out of memory in SGLang

Set SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64 (the default 512 can OOM), keep --mem-fraction-static 0.7 and halve --max-prefill-tokens to 2048. More general fixes are in my CUDA out of memory guide.

“unknown model architecture” in llama.cpp, Ollama or LM Studio

Mainline builds do not support v2 yet. Build llama.cpp b11913 with the unlimited-ocr2.patch as shown above; the listed ollama run hf.co/... command will not work until support is merged.

Tables split badly across page windows

Smaller page windows save memory but weaken cross-page merging. Raise SGLANG_UOCR_DOC_BASE_PAGE_WINDOW if you have VRAM to spare.

Hebrew, Arabic or rotated text comes out wrong

Issues #90, #95 and #79 report a hallucinated token in Hebrew, weak Arabic support and dropped lines on rotated text. Deskew scans first, and test non-Latin scripts before trusting the output.

Limits and Licence

• Licence: MIT for code and weights, so commercial use is fine. The paper itself is CC BY 4.0.
• Self-made benchmark: MarathonDocBench is Baidu’s own; independent long-document tests are not out yet.
• Weak global structure: the paper admits sparse attention loses document-wide context, which hurts heading hierarchies.
• Training data: mostly pseudo-labels from MinerU2.5-Pro and PaddleOCR-VL, which the authors say limits robustness.
• No language list and no official VRAM minimum.
• Serving quirks: a custom SGLang wheel is required, and the llama.cpp patch supports one request at a time.

My Take

A 3B model beating Qwen3.8-Max and Gemini on 30-page documents sounds too good, and the single-page scores show it is not magic. Where it clearly wins is length: memory stays flat while big models fall apart past 100 pages.

For anyone digitising manuals, theses or scanned archives on a home PC, that is the feature that matters. Start with the Hugging Face demo, then use transformers on an 8-16 GB NVIDIA card. I would wait for mainline llama.cpp and Ollama support before building it into a desktop workflow.

Frequently Asked Questions

Is Unlimited OCR 2 free for commercial use?

Yes. The weights and code are MIT licensed, which allows commercial use with the licence notice kept.

How much VRAM does Unlimited OCR 2 need?

Baidu gives no minimum. The BF16 weights are about 5.9 GB, so an 8 GB GPU should handle single pages, and the 2-3 GB GGUF files run on 4-6 GB cards or CPU (estimates).

Can I run Unlimited OCR 2 in Ollama?

Not yet. The GGUFs need a patched llama.cpp build, so Ollama and LM Studio cannot load them until support is merged upstream.

How many pages can it handle at once?

Baidu tested up to 300 pages in MarathonDocBench. SR-SWA keeps per-step cost nearly constant, so the practical limit is your output token budget and time.

Does it output Markdown?

It outputs text with a <PAGE> marker per page, and Baidu’s post-processing script converts that to Markdown with tables.

Is it better than DeepSeek-OCR?

It is built on DeepSeek-OCR and mainly adds long-document parsing. For single pages the gain is small; for 100+ page files it is a different class.

Does it work on a Mac?

Yes, through the MLX 4/6/8-bit builds or the patched llama.cpp with Metal. The PyTorch MPS route currently crashes mid-generation.

Sources

• Hugging Face model card: baidu/Unlimited-OCR-2
• GitHub: baidu/Unlimited-OCR (install, SGLang, changelog, issues #79, #81, #82, #90, #95)
• arXiv 2610.39308: “Unlimited OCR Works 2” (benchmarks, memory, limitations)
• Hugging Face: AtomicChat/Unlimited-OCR-2-GGUF and RX5950XT/Unlimited-OCR-2-GGUF (files, llama.cpp patch)
• Hugging Face: mlx-community/Unlimited-OCR-2 4/6/8-bit