How to Choose an OCR Model
This note covers Joe Barrow’s session in the AI Product Engineering series.
Joe Barrow walks through how to pick an OCR model. He presented a decision matrix with two axes: whether you need plain text blocks or full document structure, and whether you want a hosted API or a self-hosted model:
Big cloud providers like AWS Textract and Google Cloud Vision are cheap with word-level bounding boxes, at $0.60 to $1.50 per 1,000 pages. Document startups like Reducto, Datalab, and Extend are expensive but feature complete, at $5 to $20 per 1,000 pages. Open pipelines like Tesseract and PaddleOCR are cheap and fast, with narrower capabilities. Open VLMs like LightOnOCR, Chandra, and Docling need a GPU to run and are roughly feature complete.
Joe advises that only about 5% of teams should self-host. If you are product focused its often better to use an API.
Even when you use an API, the model you pick still matters, because PDFs break extractors in unexpected ways. A few Joe called out:
- TeX-compiled PDFs contain no space characters. TeX places glyphs at coordinates, so a naive extractor returns one long run of letters with zero spaces.
- Two-column pages carry no reading order. A line-level extractor reads across columns and hands your LLM jumbled text. Cloud providers like Textract fall into this too.
- Line-level output drops semantic structure. Headers, tables, and cell boundaries vanish, so downstream retrieval loses the context.
- Bad scans make cloud APIs invent text. Textract has been observed to return the word “the” a hundred times on a noisy scan.
- Blank pages make VLMs hallucinate boilerplate. An empty page can come back with header text.
- VLM markdown output has no word-level bounding boxes. If your product needs to highlight the source phrase inside the original PDF, VLM output alone might not do it.
- Model licenses have hidden chains. Chandra and Surya are only free for organizations under $2M annual revenue that do not compete with Datalab. GLM-OCR is MIT licensed but pulls in Apache-licensed Paddle layout models, so you inherit both.
If you want to see how bad PDFs get, WTF PDF is a gallery of files that break most extractors.
Joe’s process is short. Pick your two axes, pull 50 to 100 representative pages, run every candidate on the same pages, then inspect the raw outputs and compare failure modes before you commit.
Watch Joe’s session here. His research on open OCR models is also worth reading.
To learn how to compare models on your own documents with evals, see the AI Evals course, a live cohort with hands-on exercises and office hours.