Before touching blocks or confidence scores, it’s worth placing Textract precisely between “a scanner” and “a data entry clerk.”

A scanner, or basic optical character recognition software, converts an image into a flat string of characters — it can tell you the pixels form the word “Total” and the word “$1,240.00” somewhere on the page, but it has no idea those two pieces of text are related to each other as a label and its value. A human data entry clerk reading the same invoice instantly understands that relationship, along with the layout logic of tables, checkboxes, and signature blocks, because a human brings visual and contextual reasoning to the page, not just character recognition. Amazon Textract exists to bring that second kind of understanding — structural, relational, layout-aware — to a machine, at a speed and scale no clerk could match, and without requiring a computer vision team to build and maintain that understanding from scratch.

AWS launched Textract in 2019, extending the same “managed AI, no ML expertise required” philosophy behind Rekognition (images) and Comprehend (text understanding) into the specific domain of document structure. Two specialized analyzers ship alongside Textract’s general document analysis capability: AnalyzeExpense, purpose-built for invoices and receipts, and AnalyzeID, purpose-built for identity documents like driver’s licenses and passports — both covered in the family comparison chapter ahead, since reaching for general document analysis when a specialized analyzer already exists for your exact document type is a common early inefficiency.

The commercial case for Textract is worth stating plainly, because “document processing” can sound like a back-office footnote until it’s tied to actual numbers. A mortgage lender processing loan applications manually might need a human to spend twenty to thirty minutes keying data out of a stack of income statements, bank records, and tax forms per application. A pipeline that extracts the same structured data automatically in seconds doesn’t just save labor cost — it collapses a multi-day underwriting queue into same-day turnaround, which is a genuine competitive difference in lending. That shift, from “a human types this in” to “a human reviews only what the model flagged as uncertain,” is the entire reason document intelligence services like Textract exist, and it’s why the intermediate questions in this guide — how accurate, how fast, what happens with a messy scan — matter more than the basic question of “can it read text.”

There’s a subtler reason this topic rewards intermediate-level study rather than a quick surface tour. Once a first synchronous call successfully returns text from a clean, single-page scan, the interesting engineering problems stop being “how do I call this API” and start being “how do I build a pipeline that behaves correctly across ten thousand real-world documents that are never as clean as my test scan was.” Real documents arrive skewed, photographed at an angle under office fluorescent lighting, faxed through three generations of machines, or scanned with a coffee ring on page four. A production-grade Textract pipeline is defined far more by how it handles that long tail of imperfect input than by how it handles the ideal case, and that’s exactly the kind of judgment this guide is built to develop.

Four ideas — the Block object model, confidence scores, geometry, and synchronous versus asynchronous processing — govern almost every Textract design decision.

Every piece of structure Textract detects — a line of text, a word, a table cell, a form field — is returned as a Block, a JSON object with a type (LINE, WORD, TABLE, CELL, KEY_VALUE_SET, SELECTION_ELEMENT, and others), a unique ID, and a set of Relationships linking it to other blocks. A LINE block relates to the WORD blocks that make it up through a CHILD relationship; a KEY_VALUE_SET block representing a form field relates to its label through one relationship type and its value through another. Reading Textract output fluently means understanding that you are not parsing flat text — you are walking a graph of connected blocks, and the actual business value (a table’s rows and columns, a form’s labeled fields) only emerges once you traverse those relationships correctly.

Just as with Comprehend, every detection carries a confidence score from 0 to 100 (Textract uses a 0–100 scale rather than 0–1), reflecting how certain the model is about that specific block — a word, a table cell’s contents, a form field pairing. A blurry fax or a handwritten note will produce noticeably lower confidence scores than a clean, born-digital PDF, and production pipelines should treat that score as a first-class signal, not an afterthought, routing low-confidence extractions to human review rather than trusting them blindly.

Geometry is the third core idea: every block includes a bounding box (and, for finer precision, a polygon) expressed in normalized coordinates — values between 0 and 1 relative to the page’s width and height — describing exactly where on the page that block was found. This matters practically for building review interfaces that highlight the exact region a low-confidence extraction came from, letting a human reviewer glance at one highlighted area rather than re-reading an entire page to verify one field.

Finally, the synchronous versus asynchronous split shapes almost every architecture decision. Synchronous operations process a single-page document (or explicitly the first page of a multi-page PDF) and return results in the same API call, within seconds. Asynchronous operations, submitted against a document already sitting in S3, handle multi-page PDFs (up to 3,000 pages) and TIFF files, running as a background job that must be polled or notified via Amazon SNS on completion. Confusing these two — assuming a synchronous call will process an entire multi-page PDF — is one of the single most common and costly mistakes in Textract pipeline design, covered in detail in the anti-patterns chapter ahead.

A distinction worth mastering alongside these four ideas is the difference between a SELECTION_ELEMENT and a regular text-bearing block. Checkboxes and radio buttons don’t contain readable characters, so Textract represents them as SELECTION_ELEMENT blocks carrying a SelectionStatus attribute — SELECTED or NOT_SELECTED — rather than extracted text, with their own confidence score reflecting how certain the model is about the mark’s state rather than about any character recognition. Parsing logic that expects every block to contain readable text will simply skip or mishandle these elements, which is a subtle but common source of missing data on forms that rely heavily on checkboxes, such as tax forms or medical intake questionnaires.

A working Textract system is really three layers: the document source, the Textract service boundary, and whatever consumes the resulting block graph.

Document sources are either passed directly as image bytes in a synchronous request (for a single scanned page or photo) or referenced by S3 location for asynchronous jobs handling multi-page PDFs and TIFFs. A common ingestion pattern has documents land in an S3 “inbox” bucket — uploaded by a user, a fax gateway, or an email attachment pipeline — which then triggers the Textract processing step automatically.

The Textract service boundary is where the actual computer vision and layout analysis happens: detecting text regions, classifying whether a region is a table, a form field, or plain paragraph text, and building the Block relationship graph that captures those structural findings. Which specific analysis runs depends on the FeatureTypes you request when calling AnalyzeDocument or its asynchronous equivalent — TABLES, FORMS, QUERIES, and SIGNATURES are each opt-in, and requesting only what you actually need both reduces cost and keeps the output easier to parse.

Output consumers are whatever processes the resulting JSON block graph: application code parsing synchronous results immediately, a Lambda function triggered by an SNS notification when an asynchronous job completes, or a downstream system like Amazon Comprehend analyzing the extracted text further, or a human review interface built on Amazon Augmented AI (A2I) surfacing low-confidence extractions for verification.

flowchart LR
    subgraph Sources
      A1[Single Page Image]
      A2[Multi-Page PDF in S3]
      A3[Invoice / Receipt / ID Doc]
    end
    A1 -->|Sync: AnalyzeDocument| TXT[Amazon Textract]
    A2 -->|Async: StartDocumentAnalysis| TXT
    A3 -->|AnalyzeExpense / AnalyzeID| TXT
    TXT -->|SNS Notification| LAM[Lambda Consumer]
    TXT -->|Block JSON| APP[Application Logic]
    LAM --> A2I[Amazon A2I Human Review]
    LAM --> CMP[Amazon Comprehend]
    A2I --> DB[(Structured Data Store)]
    CMP --> DB
        

One architectural detail worth internalizing early: Textract itself does not interpret meaning beyond structure. It can tell you a KEY_VALUE_SET pairs the label “Date of Birth” with the value “03/14/1985,” but it does not know that value represents a birth date in any semantic sense beyond the label text extracted alongside it — that interpretation, and any validation logic built on top of it, belongs in your own application layer, often in combination with Comprehend for further text understanding once Textract has done the structural extraction.

It’s also worth being explicit about how output is packaged differently between the two processing modes, since this affects downstream parsing code directly. A synchronous call returns the full block graph inline in the API response, ready to parse immediately. An asynchronous job, by contrast, writes its output as one or more paginated JSON files to the S3 location you specify, and for very large documents the results may be split across multiple pages that your consumer logic needs to fetch and stitch back together using the NextToken pattern common across many AWS APIs — a detail that trips up teams building their first asynchronous consumer, who sometimes assume the first page of results is the complete result.

Understanding the stages between a submitted image and a structured block graph explains most of Textract’s behavior and its occasional surprises.

Processing begins with layout detection: the model segments the page into regions — paragraphs, tables, form fields, images, signature areas — before any text recognition happens at all. This layout-first approach is what separates Textract from character-by-character OCR: it decides “this rectangular region is a table” as a distinct step from reading what’s inside each cell, which is why Textract can correctly reconstruct a table’s row and column structure even when the underlying grid lines are faint, inconsistent, or entirely absent from the scanned image.

Once regions are segmented, text recognition runs within each region, converting pixels to characters and words, each tagged with its own confidence score and bounding geometry. For table regions, a separate structural model identifies row and column boundaries and assigns each detected block of text to a specific CELL block, complete with row index and column index attributes — this is what lets you reconstruct a table programmatically as a two-dimensional grid rather than a flat list of disconnected text fragments. For form regions, the model pairs detected labels with their corresponding values using visual and spatial cues — proximity, alignment, common form conventions like a colon or adjacent checkbox — producing the KEY_VALUE_SET relationships described in the previous chapter.

The Queries feature works differently from the other feature types and is worth understanding on its own terms. Rather than extracting every table and form field indiscriminately, Queries lets you ask a natural-language question directly — “What is the invoice total?” or “What is the patient’s date of birth?” — and Textract returns the specific answer along with its confidence and location, using the same underlying layout and language understanding but targeted at exactly the fields you specified rather than everything on the page. This is particularly valuable for documents with inconsistent layouts across different vendors or sources, where building rigid extraction rules for every possible label wording would be brittle, but asking a natural-language question generalizes across label variations like “Total Due,” “Amount Owed,” and “Balance.”

It’s worth appreciating why this generalization is possible at all. The Queries model was trained to associate a natural-language question with the visual and textual context most likely to contain its answer, rather than performing an exact string match against a known label. This is the same reason a human reviewer doesn’t need every invoice to use identical wording to find the total — they recognize “the number near the bottom labeled something like total or balance due” as a pattern, and Queries approximates that same flexible pattern recognition rather than a rigid lookup table. This flexibility comes with a trade-off worth remembering: Queries can occasionally return a plausible but incorrect answer when a document’s layout is genuinely ambiguous, such as an invoice listing both a subtotal and a grand total in visually similar positions, which is exactly the kind of case a confidence threshold and spot-check review process should catch.

Tracing one document from upload to structured, reviewed data ties the previous chapters together into a single timeline.

The step most teams underestimate is the fourth-to-fifth transition: Textract returns a rich, correct graph of blocks, but nothing in that graph tells you “this document is done” in a business sense until your own logic decides what confidence threshold is acceptable for each specific field. A missing signature block, for example, is not an error Textract will raise — it’s a fact your application logic has to notice and act on, since Textract’s job ends at accurate structural extraction, not business rule enforcement.

It’s worth walking through a concrete example to make this lifecycle less abstract. Imagine a two-page insurance claim form arriving as a scanned PDF. It’s routed to the asynchronous API because it has more than one page, requested with both TABLES and FORMS feature types since it contains a claims table and several labeled fields. Layout segmentation identifies the table on page one and roughly a dozen form fields spread across both pages. Extraction runs, producing a claim-number field at 99% confidence and a hand-initialed acknowledgment box at 61% confidence. An SNS notification fires the moment the job completes, a Lambda function parses the block graph, and the low-confidence acknowledgment field is routed to a human reviewer through A2I while the high-confidence fields flow directly into the claims system — the entire six-step lifecycle compressed into a few seconds of automated processing plus a much smaller pocket of targeted human attention.

The Family

You may likeAmazon FSx→

6Textract’s Capabilities and Related Services

At the intermediate level, choosing the right analyzer — or the right sibling service entirely — matters as much as calling any single API correctly.

A useful rule of thumb: reach for DetectDocumentText when you only need raw text and structure genuinely doesn’t matter, since it’s the cheapest and fastest option. Reach for AnalyzeDocument with TABLES and FORMS when the document is a general business form with labeled fields and tabular sections but doesn’t fit a specialized analyzer. Reach for AnalyzeExpense specifically for invoices and receipts — it already understands vendor names, line items, tax amounts, and totals as named concepts, sparing you the work of mapping generic KEY_VALUE_SET pairs onto those business concepts yourself. Reach for AnalyzeID for identity document verification flows, where it returns clearly named fields like DATE_OF_BIRTH and DOCUMENT_NUMBER rather than requiring you to guess which form label corresponds to which identity attribute.

It’s also worth clearly separating Textract from two adjacent AWS services it’s frequently paired or confused with. Amazon Rekognition handles general image analysis — objects, scenes, faces — and can detect text in images too, but its text detection is a much shallower capability aimed at things like street signs or product labels in photos, not structured document analysis; using Rekognition where you actually need table or form extraction will not produce usable structured output. Amazon Comprehend, covered in a companion piece, picks up exactly where Textract leaves off: once Textract has extracted the raw text and structure from a document, Comprehend can analyze that extracted text for sentiment, entities, or custom classification — the two services are designed to be chained, not to duplicate each other’s job.

Choosing between AnalyzeDocument’s feature types deserves one more layer of nuance beyond the table above. Requesting TABLES and FORMS together on the same call is common and efficient when a document genuinely contains both, since Textract processes both structures in a single pass rather than requiring two separate calls. QUERIES, by contrast, is typically combined selectively rather than requested by default on every document, since defining useful queries requires knowing in advance what specific answers a document type should contain — a capability best reserved for document types your pipeline already understands well, rather than an exploratory first pass on an unfamiliar document format, where TABLES and FORMS alone will usually surface more of the document’s actual structure to inspect.

Textract removes the heaviest lifting of production document processing, but it doesn’t remove every judgment call.

Scaling Textract is mostly about choosing the right processing mode for document volume and page count, and respecting the quotas that govern each.

Synchronous operations are subject to transactions-per-second (TPS) quotas per account and Region, separately tracked for each operation — DetectDocumentText, AnalyzeDocument, AnalyzeExpense, and AnalyzeID each have their own independent ceiling. These are soft limits AWS will raise on request for legitimate production traffic, but a design assuming unlimited synchronous throughput during a launch or a seasonal peak (tax season for a document-heavy financial workload, for instance) is a common and avoidable source of throttling incidents.

Asynchronous jobs scale along a different axis entirely: rather than a per-second call rate, the relevant constraints are concurrent job limits per account and the per-job page ceiling of 3,000 pages. A pipeline processing thousands of multi-page loan application packets per day needs to think in terms of how many jobs can run concurrently, not how many API calls per second it’s making, and should queue incoming documents rather than submitting every job the instant a document arrives if that would exceed the concurrent job quota.

Throughput for a single asynchronous job scales roughly with page count and document complexity — a 50-page contract with dense tables on every page takes noticeably longer to process than a 50-page document that’s mostly plain paragraph text, since table and form structure detection is computationally heavier than raw text recognition. Teams processing high volumes at predictable times of day (end-of-month invoice batches, for example) often find it worthwhile to pre-request a concurrent job quota increase ahead of that known peak rather than discovering the current limit mid-batch.

A further scaling consideration specific to Textract is that quotas apply separately across the specialized analyzers too — AnalyzeExpense and AnalyzeID each carry their own TPS ceiling independent of general AnalyzeDocument calls, which means a pipeline combining invoice processing and identity verification in the same account needs to track and, if necessary, request increases for each quota independently rather than assuming a single combined Textract limit governs all traffic. Fan-out design is also worth considering deliberately at scale: rather than one Lambda function submitting jobs and separately polling for results, many high-volume pipelines split submission and result-handling into two decoupled stages connected by SQS or SNS, so a temporary slowdown in result processing never causes new document submissions to back up or time out.

Textract’s own infrastructure reliability is AWS’s responsibility; the reliability of the pipeline wrapped around it is yours to design.

Textract’s service infrastructure runs across multiple Availability Zones within a Region, meaning a single data center issue does not interrupt synchronous calls or running asynchronous jobs. The reliability gap that remains at the application layer centers on two things: retrying throttled synchronous calls with exponential backoff, and correctly handling the asynchronous job lifecycle, including jobs that fail partway through or that complete with a partial-success status on certain pages.

A specific reliability pattern worth adopting for asynchronous jobs is subscribing to the Amazon SNS topic Textract publishes job-completion notifications to, rather than polling GetDocumentAnalysis or GetDocumentTextDetection on a fixed timer. SNS notifications arrive the moment a job finishes, letting a downstream Lambda react within seconds, and they scale naturally to any number of concurrent jobs without the polling overhead of repeatedly checking status for jobs that haven’t finished yet. Production pipelines should also always check the JobStatus field explicitly — SUCCEEDED, FAILED, or PARTIAL_SUCCESS — rather than assuming an SNS notification alone means the extraction fully succeeded, since a PARTIAL_SUCCESS status on a multi-page document means some pages processed correctly while others encountered an issue worth investigating individually.

Idempotency deserves specific attention in Textract pipelines, since a Lambda function retried after a transient failure could otherwise submit the same document for analysis twice, doubling cost and potentially creating duplicate downstream records. A common safeguard is using the ClientRequestToken parameter available on the asynchronous start operations, which lets Textract recognize a retried submission with the same token as a duplicate of an already-running or completed job rather than starting a brand new one — a small configuration detail that prevents a subtle but real cost and data-integrity issue at scale.

Because Textract routinely processes some of the most sensitive documents an organization holds — tax forms, medical records, identity documents — its security model deserves close attention.

Identity and access is governed by IAM policies scoped to specific API actions and, for asynchronous jobs, to specific S3 input and output bucket paths, so a service submitting documents for analysis can be denied any ability to read another team’s processed output or submit jobs against unrelated buckets.

Encryption at rest applies to asynchronous job output written to S3, encryptable with AWS KMS using either an AWS-managed or customer-managed key. Encryption in transit is enforced by default across all Textract API calls over HTTPS/TLS. For network isolation, VPC endpoints via AWS PrivateLink let document analysis calls originate from inside a private VPC without traversing the public internet — a frequent requirement given how often Textract handles regulated categories of documents like financial statements or protected health information.

For workflows requiring human verification of low-confidence extractions, Amazon Augmented AI (A2I) deserves specific mention as the AWS-native way to build that review step without standing up custom infrastructure: it routes flagged extractions to a defined human workforce (private, vendor-managed, or Amazon Mechanical Turk), presents the original document region alongside Textract’s extracted value using the geometry data described earlier, and records the human’s correction back into your pipeline — a pattern that keeps sensitive document content within AWS-managed infrastructure throughout the review process rather than exporting it to a separate, less controlled system.

Data residency and retention are worth addressing explicitly for regulated workloads. Textract does not persist the documents you submit for analysis beyond the processing needed to return results, though asynchronous job output written to S3 persists exactly as long as your own bucket lifecycle policy dictates — meaning retention of extracted data, and the compliance obligations tied to that retention, are decisions made in your own storage layer rather than imposed by Textract itself. Teams handling documents under regulations with strict data residency requirements should also confirm which AWS Region their Textract calls run in, since document content and derived output remain within the Region where the API call was made, consistent with AWS’s broader Regional data residency model.

Textract exposes its health through CloudWatch, and a handful of signals matter far more than the rest for an intermediate-level pipeline.

Beyond the headline CloudWatch metrics, a practice worth building early is tracking the distribution of confidence scores across processed documents over time, not just per-document pass/fail outcomes. A gradual downward drift in average confidence scores for a specific document type — say, receipts from one particular vendor whose print quality has degraded — is a leading indicator of rising review workload well before it shows up as a spike in human-reviewed cases. CloudTrail, as with every AWS service, logs every management-plane API call, which matters during a security review or a cost audit when it’s unclear who submitted a large batch of jobs or configured a specific S3 output location.

It’s also worth instrumenting the human-review side of the pipeline, not just the Textract call itself. Tracking what percentage of documents require any human review at all, broken down by document type and source, turns an otherwise invisible operational cost into a measurable one — a finding that 40% of documents from one particular intake channel require review, against 5% from another, is exactly the kind of signal that justifies investing in better scan quality or a more targeted query set for that specific source, rather than treating review workload as a uniform, unavoidable tax across every document equally.

Textract’s real value shows up in how naturally it chains into broader AWS document workflows rather than as a standalone extraction tool.

The most common real-time integration pairs Textract with AWS Lambda triggered directly by an S3 upload event: the moment a document lands in an inbox bucket, a Lambda function calls the appropriate synchronous or asynchronous Textract operation, with no server infrastructure to manage and automatic scaling to match upload volume. For asynchronous, multi-step workflows — submit a job, wait for SNS completion, route low-confidence fields to A2I, then write final results to a database — AWS Step Functions is the natural orchestration layer, turning what would otherwise be a fragile chain of Lambda functions and manual state tracking into a single observable, retryable workflow.

Chaining Textract’s output into Amazon Comprehend for further text understanding is another frequent pattern — once Textract has extracted the raw narrative text from a document (a doctor’s clinical notes, a legal contract’s free-text clauses), Comprehend can run entity recognition, sentiment, or custom classification on that extracted text, something Textract itself was never built to do. For search and discovery use cases, Amazon Kendra can ingest Textract’s structured output to make scanned document archives searchable in ways a raw PDF never could be, since Kendra can index the actual extracted text and metadata rather than an opaque image file.

A last integration pattern worth naming explicitly involves feedback loops for continuous quality improvement. When Amazon A2I routes a field to human review and a reviewer corrects it, that correction is valuable signal beyond just fixing that one document — pipelines that log corrections systematically (which field types get corrected most often, which document sources produce the most review flags) build an evidence base for where to invest in preprocessing improvements, additional query definitions, or stricter confidence thresholds, turning what might otherwise be a purely reactive review process into a continuous improvement loop for the pipeline as a whole.

A few recurring shapes show up again and again in Textract-based systems — some worth copying, some worth avoiding.

The confidence-gated review pattern mirrors the same shape seen across AWS’s AI services: high-confidence extractions flow through automatically, while low-confidence fields route to a human reviewer via Amazon A2I rather than being silently trusted. The extract-then-enrich pattern chains Textract’s structural extraction into Comprehend or a custom validation service, treating Textract as the layer that turns pixels into text and structure, and a second layer as the one that turns that text into validated business meaning. A third recurring shape, the classify-then-route pattern, runs a lightweight document classification step first — often using Comprehend’s custom classification on a quick text sample, or simple metadata like file naming conventions — to determine which specific Textract feature types or specialized analyzer a given document should be sent to, avoiding the anti-pattern of requesting every feature type on every document regardless of fit.

Most Textract production issues trace back to one of a small set of recurring, avoidable mistakes.

Two further habits separate teams running Textract smoothly for years from those that get surprised repeatedly. The first is periodically sampling a percentage of high-confidence extractions for manual spot-checking even when volumes are high, since a systematic error affecting one document type or one source vendor can otherwise go undetected for a long time if it happens to produce consistently high confidence scores despite being subtly wrong. The second is documenting confidence thresholds per field type with the same rigor recommended for any NLP or ML pipeline — a threshold chosen for a critical field like a loan amount should be more conservative, and its rationale more clearly recorded, than a threshold chosen for a cosmetic field like a document’s page header.

A third habit worth building in is treating document source quality as a variable to actively manage rather than a fixed constraint to simply tolerate. If a specific intake channel — a particular fax gateway, a specific mobile scanning app, a legacy photocopier — consistently produces lower-quality scans that drag down confidence scores and increase review volume, that’s often a solvable upstream problem, whether through basic image preprocessing (deskewing, contrast normalization) applied before submission, or through working with that specific source to improve scan quality at capture time, rather than accepting permanently elevated review workload as an unavoidable cost of that channel.

Seeing how document intelligence shows up in production systems makes the abstract capabilities concrete.