Build an AI Document Processing Pipeline: OCR vs VLM Routing, Review Queues and Cost Per Page (2026)

What is an AI document processing pipeline, architecturally?

An intake stage, a classifier, a router that picks between free text extraction, OCR and a vision model, an extraction step bound to a typed schema, a deterministic validation layer, a confidence gate that feeds a human review queue, and an export. Seven stages, of which only two involve a model and only one is expensive.

Internalise where the money goes before designing any of it. On a three-page invoice, the modelled model cost is about 1.3 cents. The modelled human review cost, at a 22% review rate and 45 seconds per review, is 5.5 cents. Human time is 80% of the bill. That should reorganise your priorities: an afternoon on deterministic validation rules that safely reduce the review rate beats a month optimising extraction tokens.

This is a reference design, how I would build it and what the arithmetic says it costs, grounded in architecting AccioMatrix, an event-driven platform orchestrating multiple LLMs, agents and MCP tooling for 20+ enterprise clients. I have not shipped a document processing pipeline in production. Every figure below is modelled with its inputs printed, and you should substitute your own document mix before quoting any of it.

The workload shape differs from a support or research agent in a way that changes the architecture. Document processing is bursty batch work: an accounts-payable inbox delivers nothing for six hours, then 400 invoices at 09:00 on the first of the month. That is a queue workload with backpressure, per-document idempotency and a dead-letter path, not a request/response API. The general argument for that backbone is in the AI product architecture reference design. The specific consequence: a document must be processable exactly once no matter how many times the same PDF arrives, which means a content hash is part of your primary key.

The last framing point: this is barely an agent. There is no loop, no tool selection and no autonomy in the core path. It is a pipeline with two model calls in it. Teams that reach for an agent framework here inherit the debugging surface of an agent runtime for a workload that is a directed acyclic graph, and they pay for it in every incident. Use a queue and typed stages.

What does the full architecture look like?

Twelve components across four columns: intake and normalisation, classification and routing, three parallel text-acquisition paths, and validation, review and export. The router sits in the middle because it is the only component that touches every page and the only one that can change your bill by a factor of forty.

Intake is more interesting than it sounds. Documents arrive by email attachment, SFTP drop, API upload and scanner, and the same document routinely arrives twice: forwarded by a colleague, re-sent by a supplier, scanned again because the first scan was crooked. The defence is a content hash as part of the document identity, computed on the raw bytes before any processing. Two identical PDFs collapse into one record with two intake events. Two visually identical but byte-different scans do not, which is why a second, weaker duplicate check on extracted invoice number plus vendor plus amount runs after extraction as well.

The classifier runs on the first page only and does two jobs: assign a document type, and detect boundaries in a multi-document scan. The second job matters more than teams expect. A supplier who scans thirty invoices into one PDF and emails it is not an edge case, it is Tuesday, and a pipeline that treats that file as one document produces one confidently wrong record instead of thirty correct ones. Page-boundary detection is a cheap vision call and it belongs before everything else.

The three acquisition paths are the architecture's whole point. Native text extraction is free and instantaneous and covers the majority of accounts-payable volume, because most invoices are software-generated and emailed as born-digital PDFs. OCR handles clean scans at around $1.50 per 1,000 pages. Vision models handle the hard remainder (complex tables, multi-column layouts, handwriting, stamps, poor scans) at about $3.60 per 1,000 pages on a Flash-class model. Build all three behind one interface that returns text plus positional metadata, and the router becomes a small deterministic function rather than an architectural commitment.

Everything after extraction is deliberately unglamorous. Validation is arithmetic. The confidence gate is a set of field-level rules. The review queue is a UI with a source crop next to each flagged value. The export is an idempotent write keyed on document hash. The journal is a Postgres table with one row per page and one per human correction. That last table is the most valuable asset the system produces, a labelled dataset of exactly the fields your extraction gets wrong.

How do you decide between OCR and a vision model?

Per page, deterministically, from three measurable properties: whether the page already has a usable text layer, whether it contains a table whose structure carries meaning, and how clean the scan is. Never per project, and never by defaulting everything to the most capable path. That decision is where teams pay forty times more than they need to.

Start with the free path because it is the largest. A born-digital PDF carries an embedded text layer, and reading it costs nothing, takes milliseconds and is character-perfect by construction. The check is mechanical: extract the text layer, count characters per page, and compare against the page area. A page with a few hundred characters spread over an A4 area is a scan with an accidental text fragment; a page with three thousand characters is a real text layer. In accounts payable this path typically covers the majority of volume, because most invoices are software-generated and emailed.

The second decision is whether structure carries meaning. Traditional OCR returns characters and positions; it does not reliably return the fact that this number belongs to that row and that column. For a simple remittance advice that is fine. For a twelve-line invoice table with merged cells, a continuation page and a discount column, a character stream produces line items that are individually plausible and collectively wrong: the worst kind of extraction error, because it passes a schema check and fails an arithmetic one. That is exactly the case for a vision model, which reads layout and content in the same pass.

The third input is scan quality, worth measuring rather than assuming. Skew, resolution, contrast and the ratio of low-confidence OCR tokens are all cheap to compute, and a page that scores badly should escalate to the vision path rather than produce a text stream the extraction model then confidently misreads. Same shape as any routing problem: a policy table and a deterministic function, argued in full in LLM routing and caching, cost per request.

One note on prebuilt document models. Azure prebuilt models at around $10 per 1,000 pages, Google Document AI in the $10-$30 band and AWS Textract Forms plus Tables at around $65 are all more expensive than a Flash-class vision model at roughly $3.60. They earn their price only when their output schema is closer to what you need than your own prompt and schema, and when you value the vendor's accuracy commitment over the flexibility. For a bespoke schema, which most real AP pipelines have because your ERP has opinions, the vision path is usually cheaper and more controllable.

export type Route = "native_text" | "ocr" | "vlm" | "human_triage";

export type PageSignals = {
  /** characters recovered from the embedded PDF text layer */
  textLayerChars: number;
  /** page area in square inches, from the media box */
  areaSqIn: number;
  /** ruling lines / cell candidates detected by a cheap CV pass */
  tableCellCandidates: number;
  /** 0-1, from a fast skew + contrast + resolution heuristic */
  scanQuality: number;
  /** classifier confidence that this page belongs to the assigned doc type */
  typeConfidence: number;
  /** true when the doc type declares that table structure is load-bearing */
  structureIsLoadBearing: boolean;
};

/** Below this, an apparent text layer is stray metadata, not real text. */
const MIN_CHARS_PER_SQ_IN = 4.5;

export function routePage(s: PageSignals): { route: Route; reason: string } {
  if (s.typeConfidence < 0.7) {
    return { route: "human_triage", reason: "unclassified_document" };
  }

  const density = s.textLayerChars / Math.max(s.areaSqIn, 1);
  const hasRealTextLayer = density >= MIN_CHARS_PER_SQ_IN;

  // Structure beats characters. A perfectly OCR'd table is still the wrong
  // shape, and a text layer does not tell you which cell a number lives in.
  if (s.structureIsLoadBearing && s.tableCellCandidates >= 12) {
    return { route: "vlm", reason: "load_bearing_table_structure" };
  }

  if (hasRealTextLayer) {
    return { route: "native_text", reason: "embedded_text_layer" };
  }

  if (s.scanQuality < 0.55) {
    return { route: "vlm", reason: "poor_scan_quality" };
  }

  return { route: "ocr", reason: "clean_scan_simple_layout" };
}

/** Modelled unit economics, Aug 2026 list prices. Keep this next to the router
 *  so nobody changes a threshold without seeing what it costs. */
export const ROUTE_COST_PER_PAGE_USD: Record<Route, number> = {
  native_text: 0.0,
  ocr: 0.0015,     // ~$1.50 per 1,000 pages, basic OCR
  vlm: 0.0036,     // ~1,600 image + 1,200 prompt tokens in, 400 out, Flash-class
  human_triage: 0.25,
};

What happens to one document, end to end?

Hash and dedupe, classify and split, route each page, acquire text on the cheapest sufficient path, one extraction call bound to a typed schema, deterministic validation, a confidence gate, and either an idempotent export or a place in a review queue. Roughly six seconds for a three-page born-digital invoice, two model calls, about 1.3 cents.

The single most important structural decision: extraction runs once over the whole document, not once per page. A three-page invoice has a header on page one, line items spanning pages one and two, and totals on page three. Extract page by page and you get three partial records that then need reassembling by logic that gets the continuation cases wrong. One call, whole document, one typed record. If the document is genuinely too long for a single context, split on the semantic boundary the classifier found rather than on page count.

Validation runs before the confidence gate and is entirely deterministic. Line items sum to subtotal. Subtotal plus tax equals total. Tax rate is one your jurisdiction permits. Invoice date precedes due date. Currency is one this vendor bills in. The vendor exists in your master data. Every one is a pure function, costs nothing, and catches a class of error no self-reported confidence score will. In my experience building extraction systems, the arithmetic checks find more real errors than anything the model says about itself.

Export is idempotent on the document hash. A retried export must not create a second payable, and the mechanism is the same as any irreversible tool call: write the export record with its key before dispatching, and on a unique-constraint violation read back the prior receipt instead of calling again. The full pattern with the crash-window reconciliation branch is in the agent loop in production.

Note what does not appear in the sequence: an agent. No loop, no tool selection, no planning. Two model calls, some deterministic functions, a queue and a human. If your document pipeline has an agent runtime in it, ask what the loop is for. Occasionally there is a good answer, such as an agent that fetches a purchase order to reconcile against, but usually a framework was chosen before the problem was described.

import { z } from "zod";

const Located = <T extends z.ZodTypeAny>(inner: T) =>
  z.object({
    value: inner,
    page: z.number().int().min(1),
    bbox: z.tuple([z.number(), z.number(), z.number(), z.number()]),
  });

export const LineItemSchema = z.object({
  description: Located(z.string().min(1)),
  quantity: Located(z.number().positive()),
  unit_price_minor: Located(z.number().int().nonnegative()),
  line_total_minor: Located(z.number().int()),
});

export const InvoiceSchema = z.object({
  vendor_name: Located(z.string().min(1)),
  vendor_tax_id: Located(z.string()).nullable(),
  invoice_number: Located(z.string().min(1)),
  invoice_date: Located(z.string().date()),
  due_date: Located(z.string().date()).nullable(),
  currency: Located(z.string().length(3)),
  po_number: Located(z.string()).nullable(),
  line_items: z.array(LineItemSchema).min(1),
  subtotal_minor: Located(z.number().int()),
  tax_minor: Located(z.number().int().nonnegative()),
  total_minor: Located(z.number().int()),
});

export type Invoice = z.infer<typeof InvoiceSchema>;

export type Flag = { field: string; rule: string; detail: string };

/** Pure arithmetic. No model, no cost, and it finds more real extraction
 *  errors than any self-reported confidence score. */
export function validate(inv: Invoice, vendorIds: Set<string>): Flag[] {
  const flags: Flag[] = [];
  const v = <T>(x: { value: T }) => x.value;

  for (const [i, li] of inv.line_items.entries()) {
    const expected = Math.round(v(li.quantity) * v(li.unit_price_minor));
    if (Math.abs(expected - v(li.line_total_minor)) > 1) {
      flags.push({
        field: "line_items[" + i + "]",
        rule: "line_total_mismatch",
        detail: "qty x unit = " + expected + ", extracted " + v(li.line_total_minor),
      });
    }
  }

  const summed = inv.line_items.reduce((a, li) => a + v(li.line_total_minor), 0);
  if (Math.abs(summed - v(inv.subtotal_minor)) > 1) {
    flags.push({ field: "subtotal_minor", rule: "lines_do_not_sum", detail: "sum " + summed });
  }

  if (v(inv.subtotal_minor) + v(inv.tax_minor) !== v(inv.total_minor)) {
    flags.push({ field: "total_minor", rule: "total_mismatch", detail: "subtotal + tax != total" });
  }

  if (inv.due_date && v(inv.due_date) < v(inv.invoice_date)) {
    flags.push({ field: "due_date", rule: "due_before_issue", detail: v(inv.due_date) });
  }

  if (!vendorIds.has(v(inv.vendor_name).toLowerCase().trim())) {
    flags.push({ field: "vendor_name", rule: "unknown_vendor", detail: v(inv.vendor_name) });
  }

  return flags;
}

How should the human review queue work?

Field-level rather than document-level, with a source crop shown next to every flagged value, keyboard-driven, and ordered by financial exposure rather than arrival time. A reviewer should be confirming or correcting four fields in forty-five seconds, not reading a PDF and retyping an invoice.

The economics justify unusual care here. At a 22% review rate and 45 seconds per document, review is 80% of the pipeline's cost. Two design decisions move that number more than any model change. First, the source crop: show the extracted value beside a tight image crop of exactly where it came from and a reviewer confirms in two seconds; show a value and a full-page PDF viewer and the same confirmation takes twenty. The bounding box in the extraction schema exists entirely to make this possible, which is why it is a required field, not a nice-to-have.

Second, review fields, not documents. A document with one flagged quantity does not need a human to re-read the vendor name, the dates and eleven other line items. Present the flags, let everything else through, and record which fields were touched. That cuts per-document review time and, more importantly, produces a corrections log precise about which field failed rather than which document was wrong.

Third, order the queue by exposure rather than arrival, the easiest decision to skip. A flagged $180,000 invoice and a flagged $42 one are not equally urgent, and a queue that drains in arrival order occasionally leaves the large one until the end of the day. Sort by amount, by approaching payment terms, and by vendor risk, and put a hard SLA on the top band.

One thing to resist: routing to review on model-reported confidence alone. Self-reported confidence tracks how typical the document looks, not whether the number is right. A beautifully laid out invoice with a transposed digit scores high. Deterministic validation failures, master-data mismatches, out-of-distribution vendor templates and value thresholds are all better triggers, and all of them are free.

What does one page actually cost?

About 0.45 cents per page in models on a three-page born-digital invoice, and about 2.3 cents per page all-in once human review is included at a 22% review rate. Extraction is 1.3 cents per document; review is 5.5. The models are not the bill.

Break the document down. Intake and hashing are $0.0002. Classification on page one is $0.0017. Routing is free. Text acquisition on the native path is free, or $0.0045 for three OCR pages, or $0.0108 for three vision-model pages. One extraction call over roughly 10,200 input tokens and 600 output tokens on a Flash-class model is $0.0099. Validation is free. A second-pass re-extraction on the 12% of documents that fail a check is $0.0012 amortised. Journal storage is $0.0003. Model subtotal: $0.0134 on the native path.

Then review. Twenty-two per cent of documents go to a human for 45 seconds at $20/hour fully loaded: $0.25 per reviewed document, $0.055 amortised across all documents. That is 80% of a $0.0684 total. As an engineering priority: cutting the review rate from 22% to 12% saves $0.025 per document, nearly twice the entire model spend. No amount of prompt optimisation competes with a validation rule that safely auto-approves a document class.

Against manual keying the comparison is not close. Four minutes of data entry per invoice at $20/hour is $1.33. At 33,000 documents a month, roughly 100,000 pages, that is about $44,000 of keying against about $2,260 of pipeline cost, a modelled saving of roughly $41,700 a month. Even at a pessimistic 40% review rate the pipeline lands near $0.11 a document and the comparison still holds by an order of magnitude.

Two caveats on that saving. First, the manual baseline assumes people currently key every invoice, often only partly true: many AP teams already have partial automation, so the real delta is smaller. Second, the review rate on your document mix is unknowable in advance. A portfolio of forty regular suppliers with stable templates behaves nothing like a long tail of one-off vendors. Run 500 real documents through a two-week pilot and measure the review rate before anyone builds a business case, because that one number moves the answer by a factor of three.

What breaks in production, and how would you know?

Ten things, and the ones that cost real money all produce a well-formed record. A transposed digit in a total, a line-item table that shifted a column, a duplicate invoice paid twice, and a wrong vendor match all pass schema validation and arrive in your ERP looking exactly like correct data.

Duplicate payment deserves its own paragraph because it is the failure with a direct cash cost and a reconciliation tail. Two defences, both needed. The content hash catches byte-identical resubmissions, covering forwarded emails and repeated SFTP drops. It does not catch the same invoice rescanned, or sent as a PDF once and an image the next time, so a second check on extracted vendor plus invoice number plus amount, run after extraction and before export, is mandatory. Neither check is a model.

The quiet, systemic failure is a vendor template change. A supplier redesigns their invoice, extraction starts putting the discount column where the quantity used to be, and every invoice from that vendor is subtly wrong until someone notices. The detection signal is not an error rate. It is a per-vendor correction rate that jumped. Group your corrections log by vendor and alert on the delta, and this becomes a Tuesday morning ticket instead of a quarterly write-off.

Retention and privacy are real here in a way they are not in most AI products, because invoices and contracts contain personal data, bank details and commercially sensitive terms. Two rules to settle before the first line of ingestion code: originals live in one place with a retention policy enforced by a job rather than a document, and page images must not leak into model-provider training or long-lived trace storage. Check the data-handling terms of the specific API tier you use, not the marketing page, and record which tier each document was processed under in the journal.

Four signals catch most of the rest: review rate by document type, per-vendor correction rate, escaped-error rate measured by deliberately sampling auto-approved documents, and cost per document with a price snapshot. The third requires sending one to two per cent of clean documents to a human anyway, which feels wasteful and is the only honest way to know your accuracy. Everything else is a number you assumed.

“Auto-approved documents are by definition never checked. Unless you deliberately send one or two per cent of them to a human anyway, your escaped-error rate is not a measurement. It is a number you assumed and then put on a slide.”— The reason clean-document sampling belongs in v1

What does it take to build, and what should you skip?

About seven engineer-weeks for a two-person team to a production v1, plus two to three engineer-days a month of maintenance. Skip fine-tuning, skip a document-specific vector database, skip fine-grained document typing, and skip anything that promises to remove the human entirely. What you cannot skip is the router, the deterministic validators, the corrections log and clean-document sampling.

Week one is intake, hashing, rendering and the journal: the plumbing that makes documents processable exactly once and debuggable afterwards. Week two is the classifier and the splitter, and the splitter is the part with real value: multi-document scans are common and getting them wrong loses records rather than corrupting them. Week three is the router and all three acquisition adapters behind one interface, less work than it sounds because two of the three are thin wrappers around existing services.

Week four is extraction with the typed schema and the located-field structure. Week five is validation, twenty or so arithmetic, referential and temporal rules, the cheapest week in the build measured by cost avoided. Weeks six and seven are the review UI, the corrections log, the export path with idempotency, and the sampling job. The review UI is the piece most likely to be handed to whoever is free, and it should not be: it directly controls 80% of the running cost.

On skipping: do not fine-tune a model in v1. Few-shot examples drawn from your corrections log give most of the benefit at none of the operational cost, and the corrections log does not exist on day one anyway. Do not build a vector database over documents unless you have a retrieval use case beyond extraction. The pipeline described here never searches, it processes. If you add document search later, RAG over private documents covers the permission-aware version, the only version worth building over contracts.

And be sceptical of straight-through-processing targets above about 85% for a heterogeneous document mix. The last band is long-tail by construction: unusual vendors, damaged scans, handwritten annotations, one-off formats. Chasing them with engineering has poor returns compared with making review fast, which is why the review UI deserves a full week. For the general scoping discipline for compressing a build like this, building an MVP in days with AI covers it, and the ongoing architecture work is what AI product development exists for.