Amazon QLDB – Beyond the Basics
A working engineer's guide to how QLDB's journal, cryptographic verification, and optimistic concurrency actually behave once you get past "it's an immutable database."
If you already know that QLDB is “an immutable, cryptographically verifiable ledger database,” this article picks up from there. We’ll spend our time on how the journal actually produces its cryptographic guarantees, what optimistic concurrency control means for how you write application code, how streams turn the journal into a live event feed, and what to weigh before you commit a system’s audit trail to it. One note before we start: AWS has been moving customers off QLDB toward alternatives such as Amazon Aurora PostgreSQL with cryptographic verification extensions and the open-source Amazon QLDB ledger project — if you’re evaluating QLDB for a new system today, check the current AWS guidance on service availability before you commit, since the concepts below remain useful for understanding ledger-style databases either way.
AWS has announced the discontinuation of Amazon QLDB for new customers, with migration guidance pointing toward Aurora PostgreSQL-based ledger patterns. Existing QLDB deployments and the concepts in this guide remain relevant for understanding journal-based, cryptographically verifiable data models — verify current service availability on the AWS site before architecting a new system around it.
A fast recap of what problem QLDB was built to solve, framed for someone past the introductory pitch.
QLDB is a purpose-built ledger database: every change to every document is appended to an immutable, cryptographically chained journal, and the “tables” you query with PartiQL are really just a materialized, indexed view over that journal’s history. The pitch that matters at the intermediate level isn’t “immutable data” by itself — plenty of databases can be made append-only with discipline. It’s that QLDB gives you a built-in, mathematically verifiable proof that a document’s history has not been altered, without you having to build and operate that proof system yourself.
This distinguishes QLDB from two adjacent categories: general-purpose relational databases with manual audit tables (where nothing stops a privileged user from editing the audit log itself), and blockchain frameworks (which add decentralized consensus overhead that most single-organization systems don’t actually need).
An audit table bolted onto RDS
A trigger-populated history table is only as trustworthy as the permissions protecting it — anyone with sufficient database privileges can rewrite it undetected.
A private blockchain network
QLDB has a single centrally-owned journal, not a distributed ledger requiring consensus across independent parties — it solves verifiability, not decentralized trust.
The vocabulary intermediate QLDB work depends on — journal, revisions, and the document model.
Journal — the append-only, sequential log that is the actual source of truth for a QLDB ledger. Every insert, update, and delete is recorded here first, permanently, in the order it was committed.
Document Revision — QLDB never overwrites a document in place. An “update” creates a new revision of the document with an incremented version number and a pointer back to its predecessor, so the full history of every document is always retrievable.
Amazon Ion — QLDB’s underlying data format, a superset of JSON that adds richer types (timestamps, precise decimals, binary blobs) while remaining structurally JSON-compatible; every document you store is, at the storage layer, an Ion value.
PartiQL — the SQL-compatible query language QLDB exposes over its document-oriented storage. It supports familiar SELECT, INSERT, UPDATE, and DELETE syntax while operating over semi-structured, nested Ion documents rather than fixed relational rows.
Tables — logical, named partitions of the journal that behave like relational tables for query purposes, but are really just filtered, indexed views over the subset of journal entries tagged to that table.
Think of the journal as a bank’s paper ledger book, written in ink, page after page, never torn out or erased. A “table” is like a card-index summary someone maintains for quick lookup — convenient for finding the current balance fast, but the actual proof of every transaction that ever happened still lives in the ink-written pages behind it.
How the journal, indexed storage, and streams layer fit together.
- Journal Storage — the durable, append-only, replicated log; the authoritative record every other component derives from.
- Indexed Storage — a queryable, materialized representation of the current and historical document state, built from the journal, that PartiQL actually executes against for performance.
- QLDB Streams — a continuous, near-real-time export of journal changes to an Amazon Kinesis Data Stream, letting other systems react to every committed revision as it happens.
- Ledger Permissions Mode —
STANDARDmode enforces IAM-based, table- and PartiQL-statement-level permissions; the olderALLOW_ALLmode grants any authenticated principal full access and is generally discouraged for new ledgers.
flowchart TB App["Application
QLDB Driver"] --> API["QLDB Session API"] API --> Journal["Journal Storage
(append-only, replicated)"] Journal --> Indexed["Indexed Storage
(queryable tables)"] Journal --> Streams["QLDB Streams"] Streams --> Kinesis["Amazon Kinesis Data Stream"] Kinesis --> Consumers["Downstream Consumers
Lambda / Analytics / Search"] API --> Indexed
The cryptography behind “you can mathematically prove this data wasn’t tampered with.”
Every block of committed transactions in the journal is hashed, and those hashes are combined pairwise up through a Merkle tree until they produce a single root hash called a digest. Because each new block’s hash incorporates the hash of everything before it, changing even one byte of historical data would change every hash above it in the tree — making tampering mathematically detectable rather than merely policy-forbidden.
Picture a tall stack of sealed envelopes, where each envelope’s wax seal is stamped with a summary of every envelope below it. If someone secretly swaps the contents of an envelope near the bottom, the seal on every envelope above it stops matching what it should be — you don’t have to open every envelope to know something changed, you just have to check the seals.
QLDB lets you request a digest of the ledger at any point in time, and separately request a proof for a specific document revision. Using the digest, the proof, and the revision’s own hash, you (or an independent auditor, with no access to the live system at all) can recompute the Merkle path and confirm the revision genuinely belongs to that verified history — this is the mechanism that makes QLDB’s “cryptographically verifiable” claim something you can actually check, not just something AWS asserts.
What happens between calling a write and it being durably, verifiably committed — and why conflicts are normal, not exceptional.
QLDB uses optimistic concurrency control (OCC) rather than locking. A transaction reads the documents it needs, computes its changes, and attempts to commit; QLDB checks whether any of the documents the transaction read have been modified by another transaction since it started. If so, the commit is rejected with an OccConflictException, and the application is expected to retry the transaction from the beginning.
sequenceDiagram
participant App as Application
participant QLDB as QLDB Session
App->>QLDB: Begin transaction
App->>QLDB: SELECT balance FROM Accounts WHERE id = 'A1'
QLDB-->>App: balance = 500
App->>QLDB: UPDATE Accounts SET balance = 400 WHERE id = 'A1'
App->>QLDB: Commit
alt No conflicting write since read
QLDB-->>App: Commit succeeds, new revision appended
else Another transaction modified A1 first
QLDB-->>App: OccConflictException — retry transaction
end
Because OCC conflicts are an expected, routine outcome rather than an error condition, application code that talks to QLDB should always wrap transactions in retry logic. The official QLDB drivers build this retry behavior in by default — bypassing it and handling transactions manually is a common source of subtle production bugs.
Turning the cryptographic theory from Chapter 4 into an operational habit.
A typical verification workflow looks like this: periodically request and securely store a ledger digest (for example, exporting it to a separate, access-controlled location outside the QLDB ledger itself). Later — during an audit, a dispute, or a compliance review — retrieve the specific document revision in question along with its proof, and independently recompute whether that revision’s hash resolves correctly against the previously stored digest.
The value of this workflow depends entirely on storing digests somewhere independent of the ledger’s own administrators. A digest that only ever lives inside the same AWS account with the same permissions as the ledger offers weaker guarantees than one exported to a separate trust boundary — the cryptography only protects you as well as your operational discipline around where the digest itself lives.
Turning an immutable journal into a live feed other systems can react to.
QLDB Streams delivers every committed revision to a Kinesis Data Stream in near real time, in the order it was committed. This is what lets QLDB act as a system of record while other systems stay in sync without polling: a Lambda function can react to every new revision, a search index can be updated incrementally, or an analytics pipeline can build a running materialized view without ever querying the ledger directly for bulk exports.
Common Streams Pattern: Search Index Sync
A Lambda consumer reads each streamed revision and upserts a corresponding document into a search service (such as OpenSearch), keeping a fast, flexible search layer continuously current with the ledger’s verified history without the search layer itself needing to be the source of truth.
What actually limits throughput in a ledger database, and why it isn’t the same as a typical OLTP system.
Because every write is a serialized append to a single ledger’s journal, and OCC conflicts increase under high contention on the same documents, QLDB’s write scalability is bounded less by raw storage throughput and more by how much your write pattern contends over the same rows. A workload that spreads writes across many independent documents scales comfortably; a workload with many concurrent writers repeatedly updating the same few “hot” documents will see a rising rate of OCC retries under load.
Reads against indexed storage scale independently of write contention, since they’re served from the materialized, indexed view rather than replaying the journal — but query performance still depends heavily on whether you’ve created indexes on the fields your PartiQL queries filter on, exactly as with a conventional database.
What QLDB manages for you, and what durability guarantee that actually buys.
QLDB automatically replicates journal data across multiple Availability Zones within a Region, and as a fully managed service, AWS handles the underlying replication, scaling, and failure recovery without requiring you to provision or manage database instances, unlike a self-managed relational database on EC2.
What You Get For Free
- Multi-AZ durability without manual replica configuration
- No instance sizing, patching, or failover orchestration to manage
- Journal durability guarantees backed by AWS’s operational SLAs
What Remains Your Responsibility
- Designing document and table structure to avoid hot-document write contention
- Building and testing OCC retry logic into your application
- Storing digests independently for meaningful audit-grade verification
Access control layered on top of an already-tamper-evident data model.
- IAM-based access control — in
STANDARDpermissions mode, IAM policies can restrict access down to specific PartiQL statement types and specific tables, rather than granting blanket ledger access. - Encryption at rest — every ledger is encrypted using a KMS key, either AWS-owned by default or a customer-managed key for organizations needing direct control over key rotation and revocation.
- VPC endpoints — QLDB supports private connectivity via VPC endpoints, so traffic between your application and the ledger never needs to traverse the public internet.
- Encryption in transit — all API calls to QLDB are made over TLS by default through the standard AWS SDK/API request signing process.
An append-only journal stops silent tampering with history — it does not, by itself, stop an over-privileged principal from appending damaging new revisions. IAM permissions and least-privilege table access remain just as necessary as with any other database.
The signals that tell you whether your ledger is healthy under load.
| Metric / Signal | What It Tells You | Watch For |
|---|---|---|
| OccConflictExceptions (rate) | How often concurrent transactions collide on the same documents | A rising rate under steady load signals hot-document contention needing a data-model rethink |
| Read/Write IOs | Throughput consumed against the ledger | Sustained high IOs alongside growing latency suggests you’re approaching practical throughput limits |
| Session request latency | Round-trip time for QLDB API calls | Rising latency alongside stable IOs often points to inefficient PartiQL queries missing indexes |
| Streams consumer lag | How far behind a Kinesis consumer is from the live journal | Growing lag means downstream systems (search, analytics) are serving increasingly stale data |
CloudTrail logs every QLDB API call for administrative auditing, complementing — but not replacing — the cryptographic, data-level verification the journal itself provides.
How QLDB is provisioned and reached from application and analytics tooling.
Ledgers, tables, and indexes are typically provisioned through Infrastructure as Code so that permissions mode, KMS key configuration, and table/index definitions stay consistent and auditable across environments — a natural fit given that consistency of configuration matters as much as consistency of data in a compliance-oriented system.
Lambda & ECS via the QLDB Driver
Official drivers handle session pooling and built-in OCC retry logic, which matters more here than with a typical database client given how central retries are to the transaction model.
Streams into Data Lakes
QLDB Streams feeding Kinesis Data Firehose is a common way to land verified, time-ordered ledger history into S3 for large-scale historical analysis outside the ledger itself.
What ledger-style modeling looks like when done well — and the mistake that undermines the whole point.
Event-sourced document design — modeling each meaningful business event (a payment, a status change, an approval) as its own document revision rather than mutating a single summary field repeatedly plays to QLDB’s strengths, since the full history of “what happened and when” becomes the natural query surface rather than something you have to reconstruct after the fact.
Narrow, single-purpose tables — keeping tables scoped to one well-defined entity type, with indexes on the fields you actually filter by, keeps both query performance and OCC contention predictable as write volume grows.
Pattern
Using QLDB as a general-purpose OLTP database for high-contention, high-throughput workloads unrelated to auditability.
Why It Fails
OCC-based concurrency is well suited to workloads with low-to-moderate contention on any single document; a system with many writers constantly updating the same hot rows (think a real-time inventory counter under heavy load) will spend an increasing share of its throughput on retries rather than committed work.
What To Do Instead
Reserve QLDB for the subset of data that genuinely needs a verifiable history — the audit trail, the compliance record, the chain of custody — and keep high-contention, high-throughput operational state in a database designed for that access pattern, syncing the two via streams where they need to stay related.
Advantages
- Built-in, independently verifiable cryptographic proof of data history
- Full document revision history retained automatically, with no custom audit-table logic
- Fully managed durability and Multi-AZ replication with no instance management
- Real-time journal-to-stream integration for reactive downstream systems
Disadvantages & Trade-offs
- OCC-based concurrency makes high-contention, high-throughput workloads a poor fit
- Immutability is a one-way door — there is no built-in mechanism to truly delete a revision, which has real implications for data-retention and right-to-erasure requirements
- Verification is only as strong as the operational discipline around independently storing digests
- Narrower ecosystem and tooling than a mainstream relational or NoSQL database
Always build retry logic around OCC conflicts
Treat OccConflictException as an expected outcome to handle, not an error to surface to the end user — the official drivers automate this, so use them rather than rolling your own transaction handling.
Design documents to minimize write contention
Splitting a frequently-updated aggregate into smaller, independently-updated documents reduces how often unrelated writers collide on the same revision.
Store digests outside the ledger’s own trust boundary
A digest kept in the same account with the same administrative access as the ledger provides a weaker audit guarantee than one exported and retained independently.
Index the fields your PartiQL queries actually filter on
QLDB does not automatically index every field — query performance against indexed storage depends on deliberate index design, exactly as it would in a relational system.
Reserve QLDB for genuinely audit-critical data
Not every table in a system needs a cryptographically verifiable history — apply it selectively to the data where that guarantee actually matters, and keep unrelated high-churn state elsewhere.
The categories of system where a verifiable ledger earns its complexity.
Supply chain provenance
Tracking a product’s chain of custody from manufacture to delivery is a natural fit for an append-only, verifiable revision history where every handoff is its own recorded event.
Financial transaction records
Systems that must prove, after the fact, that a transaction record has not been altered since it was written benefit directly from QLDB’s cryptographic digest-and-proof model rather than relying purely on access-control policy.
Regulatory and compliance audit trails
Industries facing strict audit requirements — insurance claims history, healthcare record changes, government benefits processing — use ledger-style databases specifically so an external auditor can verify history independently rather than trusting the operator’s word for it.
DELETE statement removes a document from active table queries, but the journal retains the full revision history, including the deletion event itself, as part of the ledger’s permanent record — true erasure of history is fundamentally at odds with the immutability guarantee.Carry This Forward
- The journal, not the queryable table, is QLDB’s real source of truth — tables are a materialized view over it.
- Cryptographic verification works via a Merkle tree of hashes; a digest plus a proof lets anyone independently confirm a revision’s history is untampered.
- QLDB uses optimistic concurrency control — conflicts are a normal, expected outcome that application code must retry, not an exceptional failure.
- Write throughput is bounded more by document-level contention than by raw storage limits; hot documents create rising OCC retry rates.
- Verification guarantees are only as strong as where you store your digests — keep them outside the ledger’s own trust boundary.
- Immutability means there is no true delete — factor that into data-retention and right-to-erasure planning up front.
- Reserve QLDB for the genuinely audit-critical subset of your data, and keep high-contention operational state elsewhere.

