OCR (optical character recognition) is the computer vision task of converting text that appears in an image into machine-readable characters. It turns scanned pages, photographed receipts, screenshots and street signs into text that can be searched, edited, indexed and processed by other software.
In computer vision within artificial intelligence, OCR has evolved from template matching of single characters to deep learning pipelines that detect text regions and recognize whole lines at once. Modern systems also read layout, tables and handwriting, and vision-language models increasingly combine OCR with document understanding in a single step.
How an OCR pipeline works
A modern OCR system works as a sequence of specialized stages. It first cleans the image, then finds where text is, then recognizes the characters, and finally organizes the result into reading order. Each stage can fail independently, which is why diagnosing OCR errors starts by locating the failing stage.
| Stage | Purpose | Common techniques |
|---|---|---|
| Preprocessing | Improve image quality | Deskewing, binarization, denoising, dewarping |
| Text detection | Locate words or lines | Segmentation networks, region proposals |
| Text recognition | Turn line images into characters | CNN plus sequence model, transformers |
| Layout analysis | Find columns, tables, headings | Document layout models |
| Post-processing | Fix likely errors | Language models, dictionaries, validation rules |
Recognition models are usually trained with connectionist temporal classification (CTC) loss or with an attention-based decoder. Both approaches read a whole line as a sequence, which removes the older need to segment individual characters and makes recognition of joined or degraded letters far more robust.
Types of OCR and input conditions
OCR difficulty depends heavily on what kind of text and image the system receives. Printed text on a clean scan is a largely solved problem, while handwriting and text in natural scenes remain substantially harder.
| Variant | Input | Relative difficulty |
|---|---|---|
| Printed document OCR | Scans of typed pages | Low on clean scans |
| Handwriting recognition (HTR) | Handwritten notes and forms | High, varies by writer |
| Scene text recognition | Photos of signs, labels, screens | High, due to angle and lighting |
| Historical document OCR | Old typefaces and degraded paper | High, needs custom training |
| Intelligent character recognition | Hand-printed form fields | Medium |
Image resolution is the most important controllable factor. For document scans, about 300 dpi is the common recommendation, with text height of roughly 20 to 30 pixels per line being comfortable for most recognizers. Below about 200 dpi, small fonts degrade quickly.
Measuring OCR accuracy
OCR quality is measured with edit-distance metrics against a manually verified ground truth. The character error rate (CER) counts the insertions, deletions and substitutions needed to turn the output into the reference, divided by the number of reference characters.
| Metric | What it counts | Why it matters |
|---|---|---|
| Character error rate (CER) | Character edits per reference character | Core recognition quality |
| Word error rate (WER) | Word edits per reference word | Impact on search and reading |
| Field-level accuracy | Correct extracted values per field | Business process outcome |
| Detection precision and recall | Correctly found text regions | Missing or invented text areas |
| Reading order accuracy | Correct sequence of blocks | Multi-column and table documents |
A low CER does not guarantee usable output. One wrong digit in an invoice total is a tiny character error but a serious business error, so production systems also measure field-level accuracy on the values that matter.
OCR compared with related computer vision tasks
OCR shares building blocks with object detection and image segmentation, but its output is a symbolic sequence rather than boxes or masks. The table places it among neighboring approaches.
| Approach | Output | Relation to OCR |
|---|---|---|
| Object detection | Bounding boxes with labels | Used for locating text regions |
| Image segmentation | Pixel-level masks | Used for text and layout masks |
| Vision transformer | Image representations | Backbone of many modern recognizers |
| Document AI extraction | Structured fields and tables | Builds on OCR output and layout |
| Vision-language models | Free-form text answers | Can read text without a separate OCR step |
Vision-language models read complex pages impressively, but they can paraphrase or invent text rather than transcribe it. Where exact transcription and character positions are required, a dedicated OCR engine with bounding-box output remains the safer choice, sometimes combined with a language model for interpretation.
Common OCR problems and how to address them
Most OCR failures trace back to image quality, layout complexity or domain mismatch. Typical problems include:
- Skewed or curved pages: apply deskewing and dewarping before recognition.
- Confusable characters: O and 0, l and 1, or rn and m need validation rules such as checksums and formats.
- Wrong reading order: multi-column pages and tables require layout analysis, not line-by-line reading.
- Unsupported scripts or fonts: models must be trained or fine-tuned on the target language and typeface.
- Overconfident errors: route low-confidence fields to human review instead of trusting every output.
- Sensitive data exposure: scanned documents often contain personal data and need access controls.
A plain-language summary
To a computer, a photo of a page is just colored dots. OCR is the technology that looks at those dots and recognizes the letters and numbers, so the computer can treat the picture as real text.
That is what lets you search inside a scanned contract, copy text from a photo, or have a phone read a menu aloud. The quality of the picture strongly affects the result, so a sharp, straight, well-lit image gives far better text.
An example from practice
An accounts payable department receives about 3,000 supplier invoices a month as scans and photos. Staff previously typed supplier name, invoice number, date and totals into the finance system by hand.
The team deploys an OCR pipeline with deskewing, layout analysis and field extraction, scanning paper invoices at 300 dpi. Validation rules check that line items add up to the total, that tax amounts match the rate, and that the supplier exists in the master data. Fields with low recognition confidence go to a reviewer.
After tuning on several hundred labeled invoices, most invoices pass without manual typing, and reviewers focus only on flagged fields. Arithmetic checks catch digit errors before payment, which is where OCR mistakes would be most costly.