A window is a slice of time. Windowing is the act of grouping events into those slices so they can be counted, summed or compared.

Let us start with the building blocks. An event is a small record that says something happened, such as “Priya paid 40 dollars at 10:03:12”. A stream is an ordered, never-ending sequence of events. Apache Kafka is a platform that stores and delivers these streams reliably to many programs at once. Kafka stores events in named logs called topics, and each topic is split into partitions so that many machines can share the work.

Now the key definition. Windowing means dividing a stream into finite groups of events based on time, then running a calculation on each group. The group is called a window. Each window has a start time, an end time, and a result. When you ask “how many logins per minute?”, every minute is one window, and the count inside it is that window’s result.

An important clarification: Kafka brokers do not do windowing

Many learners assume the Kafka server itself creates windows. It does not. The Kafka brokers are responsible for storing events durably and handing them out in order. They know nothing about “five-minute buckets”. Windowing is performed by stream processing applications that read from Kafka. The most common tool is Kafka Streams, a Java library that lives inside your own application. Another popular option is ksqlDB, which lets you write windowed queries in a SQL-like language. Big engines such as Apache Flink and Spark also read from Kafka and offer their own windowing features, with slightly different rules.

In this article, unless stated otherwise, we describe the behavior of Kafka Streams, because it is the reference implementation most people mean when they say “windowing in Kafka”.

Without windows, a stream aggregation grows forever, uses more memory every day, and answers questions nobody asked.

Suppose you write a program that counts payments per card. Without windows, the count for every card simply climbs forever. After a year, the number tells you the card’s lifetime activity, which is fine for a bank statement but useless for spotting fraud. Fraud shows up as bursts: twenty purchases in three minutes. To see a burst you must look at a short slice of time. Windows give you that slice.

Three problems windows solve

Typical jobs that use windows

Before any window can be drawn, the system must know what time each event happened. This is trickier than it sounds.

A window is built from timestamps, so the meaning of “time” matters a great deal. Kafka Streams recognizes three notions of time, and mixing them up is the number one source of confusing bugs.

Event time versus processing time

Picture a mobile game. A player scores a goal at 10:00:05 while riding in a subway tunnel with no signal. The phone uploads the event at 10:07:30 after leaving the tunnel. The event time is 10:00:05, because that is when the goal happened. The processing time is about 10:07:30, because that is when your program saw it. If you built windows on processing time, this goal would be counted in the wrong minute, and the leaderboard for 10:00 would be too low. Event time gives the correct answer.

Kafka Streams normally uses event time. Every Kafka record carries a timestamp. By default, that timestamp is the time the producer created the record. A topic can instead be configured so the broker stamps each record when it is appended, which is the ingestion-time behavior. If your timestamp lives inside the message body, for example a field named “transactionTime”, you can supply a custom timestamp extractor so Kafka Streams reads the right value.

Stream time: the application’s internal clock

Here is a concept unique to Kafka Streams: stream time. Stream time is the largest event timestamp the application has seen so far, tracked separately for each task. It only moves forward. If the newest record seen carries a timestamp of 10:15, stream time is 10:15, even if the wall clock says it is already 11:00 because the topic was quiet or a backlog was being replayed.

Why does this matter? Because Kafka Streams decides when a window is finished by looking at stream time, not the wall clock. A window that ends at 10:05 is considered closed only when stream time has moved beyond 10:05 plus the grace period. This design makes results reproducible: if you replay the same data tomorrow, you get the same windows and the same answers, no matter how fast or slow the replay runs.

The simplest and most popular window: equal-sized time slices placed one after another, like the days on a calendar.

A tumbling window has a single setting: its size. If the size is five minutes, the windows are 10:00 to 10:05, then 10:05 to 10:10, then 10:10 to 10:15, and so on. Each window starts exactly when the previous one ends. There are no gaps and no overlaps, so every event belongs to exactly one window. The start of a window is included and the end is excluded, so an event stamped exactly 10:05:00 falls into the 10:05 to 10:10 window.

Kafka Streams aligns windows to the epoch, which is midnight on 1 January 1970 in UTC. In practice this means five-minute windows always begin at :00, :05, :10 and so on, no matter when your application started. Two different applications computing five-minute windows will therefore agree on the boundaries. This alignment is also why days in a tumbling-window setup follow UTC unless you handle time zones yourself.

flowchart LR
  subgraph W1["Window 10:00 to 10:05"]
    e1["Payment at 10:01"]
    e2["Payment at 10:03"]
  end
  subgraph W2["Window 10:05 to 10:10"]
    e3["Payment at 10:06"]
  end
  subgraph W3["Window 10:10 to 10:15"]
    e4["Payment at 10:12"]
    e5["Payment at 10:14"]
  end
  W1 --> W2
  W2 --> W3

The diagram shows five payments. The first two land in the first window, giving a count of two. The third lands alone in the second window, giving a count of one. The last two land together in the third window, giving a count of two. Nothing is counted twice and nothing is missed.

When tumbling windows are the right choice

A hopping window has a size and a smaller “hop”. Windows overlap, so one event can appear in several of them.

A hopping window has two settings. The size says how long each window lasts. The advance (also called the hop) says how far the start moves forward to create the next window. If the size is ten minutes and the advance is five minutes, you get windows of 10:00 to 10:10, then 10:05 to 10:15, then 10:10 to 10:20. Each window starts five minutes after the previous one, yet each still looks ten minutes wide, so neighbors share five minutes of time.

Because of the overlap, a single event belongs to more than one window. The number of windows is simply the size divided by the advance. With size ten and advance five, each event lands in two windows. With size sixty minutes and advance one minute, each event lands in sixty windows. That last setup is popular for “last hour, updated every minute” dashboards, but it multiplies the work sixty times, so choose the hop with care.

flowchart TB
  E["Login event at minute 7"] --> A["Window minute 0 to 10"]
  E --> B["Window minute 5 to 15"]
  A --> RA["Count for window A updated"]
  B --> RB["Count for window B updated"]

Why teams choose hopping windows

Hopping windows solve the boundary problem from the previous chapter, at least partly. A burst that crosses one edge is very likely to sit fully inside another window. They also give smoother charts because the answer changes gradually. The price is extra storage and compute, plus the fact that totals across windows no longer add up, since the same event is counted several times. You cannot sum hourly hopping counts to get a daily count.

Sliding windows do not follow a fixed clock schedule. They are defined by a maximum time distance between events.

Tumbling and hopping windows use fixed edges laid out in advance. Sliding windows, added to Kafka Streams in version 2.7, work differently. You give them one setting: the maximum time difference between records. Two records belong to the same window if their timestamps are no further apart than that difference. Both the start and the end of a sliding window are inclusive, unlike tumbling and hopping windows where the end is excluded.

Behind the scenes, Kafka Streams creates windows around the arriving records so that every distinct group of records that fit within the time difference gets its own result. It does not create empty windows for quiet periods, and it does not produce identical duplicate results, which keeps it far cheaper than a hopping window with a tiny advance.

Where sliding windows shine

Session windows have no fixed length. A session keeps growing as long as events keep arriving, and ends after a quiet gap.

A session window is defined by an inactivity gap. Suppose the gap is thirty minutes. As long as a user’s events arrive less than thirty minutes apart, they belong to the same session. When thirty minutes pass with no event for that user, the session ends. The next event begins a new session. One user’s session might last two minutes; another’s might last three hours. The window length adapts to human behavior.

Session windows are always calculated per key. If the key is the user ID, each user has their own independent sessions. Two users can be in the middle of a session at the same time, and one user’s silence never closes another user’s session.

flowchart LR
  c1["Click 10:00"] --> c2["Click 10:10"]
  c2 --> c3["Click 10:15"]
  c3 --> s1["Session 1 ends after 30 minute gap"]
  s1 --> c4["Click 11:00"]
  c4 --> c5["Click 11:05"]
  c5 --> s2["Session 2 ends after 30 minute gap"]

Sessions can merge

Here is the subtle part. Because events can arrive out of order, a late event might fill a gap between two sessions and join them together. Imagine session one ended at 10:15 and session two began at 10:40, with a gap of 25 minutes, but the gap limit is 30. If both sessions already existed when this is discovered, they were separate only because of how the events happened to arrive. Kafka Streams handles this by merging the two sessions into one larger session. The old session results are retracted (a deletion marker is emitted downstream) and the new merged result is emitted. Anything that consumes session results must be ready for this “update by replacement”.

Common uses

The four window types answer four different kinds of questions. Match the shape of the window to the shape of the question.

A simple decision path

A windowed application is a small team of parts: topics that carry data, tasks that process it, and stores that remember the running totals.

Let us walk through the pieces. Everything begins with producers, the applications that write events into Kafka. Events land in an input topic, which is divided into partitions. Your Kafka Streams application reads from that topic. The application can be run as several copies, called instances, on different machines. Kafka splits the partitions among the instances so that the work is shared.

flowchart LR
  P["Producers"] --> LB["Load Balancer and API Gateway"]
  LB --> IN["Input Topic with Partitions"]
  IN --> A1["Streams App Instance 1"]
  IN --> A2["Streams App Instance 2"]
  A1 --> S1["Window State Store 1"]
  A2 --> S2["Window State Store 2"]
  S1 --> CL["Changelog Topic"]
  S2 --> CL
  A1 --> OUT["Output Topic"]
  A2 --> OUT
  OUT --> D["Dashboards and Alerts"]
  OUT --> DB["Serving Database"]

The components in plain words

How the store keeps track of windows

The state store does not hold “a window” as one object. It holds many small entries, each identified by three pieces: the key, the window start time, and the aggregated value. For example: “card 4711, window starting 10:05, count 3”. When a new event arrives, the application finds the entry for that key and window, updates the value, and writes it back. This is why the number of entries equals the number of active keys multiplied by the number of open windows per key.

When results flow downstream, the key is wrapped so that it carries the window boundaries along with the original key. Consumers therefore know both who the result belongs to and which time slice it covers. If a consumer wants a plain key, it converts the wrapped key back to the original in a later step.

Follow a single payment event from the moment it enters the application until its window result leaves it.

flowchart TD
  R["Record arrives with timestamp"] --> ST["Update stream time to the maximum seen"]
  ST --> Q{"Is window end plus grace behind stream time"}
  Q -->|"No, window still open"| U["Update aggregate in state store"]
  Q -->|"Yes, window closed"| D["Drop record and count it in a metric"]
  U --> F["Forward updated result downstream"]
  U --> CH["Write change to changelog topic"]

Real events arrive late. The grace period is how long a window waits for stragglers before it is sealed.

In a perfect world, events reach Kafka in the order they happened. In the real world, networks stall, phones lose signal, retries happen, and batch uploads arrive hours later. An event is out of order when its timestamp is older than one that arrived before it. An event is late when it arrives after its window would normally be considered finished.

Kafka Streams handles this with a grace period, a setting you choose for every window definition. The rule is simple: a window remains open for updates until stream time passes the window end plus the grace period. Consider a five-minute window ending at 10:05 with a grace of two minutes. An event stamped 10:04 that shows up when stream time is 10:06 is still accepted, because 10:06 is before 10:07. The same event arriving when stream time is 10:08 is rejected, because the window closed at 10:07.

What happens to events that are too late?

Records that arrive after their window closes are dropped. Kafka Streams does not crash or block. It increments a “dropped records” metric, so you can see how often it happens. That is your early warning. If the dropped-records rate is high, your grace period is shorter than the real-world delay, and your numbers are undercounting.

A window has a result long before it closes. Understanding continuous updates versus final results prevents nasty surprises.

By default, Kafka Streams behaves as if every window were a live scoreboard. Each time an event updates a window, a new result for that window can be emitted. If a five-minute window sees a hundred events, downstream consumers may see many successive versions of the count as it climbs. This is called the continuous refinement model. It gives low latency, because you see numbers immediately, but it means the same window and key appear many times in the output.

Two things reduce the noise. First, Kafka Streams keeps a small record cache in memory in front of the state store, so that several quick updates to the same key and window are merged into one before being forwarded. Second, the cache is flushed on a schedule known as the commit interval. The default is about thirty seconds under normal settings and much shorter when exactly-once processing is enabled. A shorter interval means fresher results and more output traffic; a longer one means fewer, larger updates.

Final results only

Some downstream systems cannot cope with repeated updates. A billing system, for example, must receive one final number per customer per window. For those cases you can ask Kafka Streams to hold results back until a window is closed, meaning stream time has passed its end plus grace. Older releases do this with a suppression step. Newer releases include a built-in option to emit results only when the window closes. In ksqlDB the same idea appears as an “emit final” mode.

Reading windowed results later

Window state stores can also be queried directly through the interactive queries feature of Kafka Streams, so an application can answer “what is the current count for this key in the current window?” without going through the output topic. The store keeps windows for a retention period that must be at least the window size plus grace. If you want to query older windows, you must extend the retention, which costs extra disk space.

Windows are not only for counting. They also decide which events from two different streams belong together.

Imagine two streams. One holds “ad shown” events. The other holds “ad clicked” events. To measure click-through you must match each click with the ad that was shown to the same user. But a click can only sensibly be connected to a display that happened shortly before it, perhaps within thirty minutes. A click a month later is a different story. A stream-stream join uses a time window to express this: two records match only if they share the same key and their timestamps are within the allowed window of each other.

The join window can be symmetric or lopsided. You can say “match a click with any display up to thirty minutes before it and up to zero minutes after it”. Kafka Streams keeps recent events from both streams in window stores so that a record arriving on one side can find its partner from the other side. Just like aggregations, joins use a grace period for late records, and old entries are removed once they pass the retention limit.

Machines crash. A good windowed design survives that without losing or double counting a single event.

Window state lives on local disks, so what happens when a machine dies? The answer is the changelog topic. Every update to a window store is also written to a compacted Kafka topic. When a task moves to another instance, the new owner replays the changelog to rebuild the local store, then resumes reading input from where the old owner left off. Because both the changelog and the input offsets are stored in Kafka, nothing is lost.

flowchart TD
  F["Instance crashes"] --> RB["Group rebalance reassigns tasks"]
  RB --> ST{"Standby replica available"}
  ST -->|"Yes"| FAST["Promote standby and catch up briefly"]
  ST -->|"No"| SLOW["Restore store from changelog topic"]
  FAST --> RES["Resume processing from committed offsets"]
  SLOW --> RES

Rebuilding a large store from scratch can take many minutes. To shorten that time, you can ask for standby replicas: warm copies of the state kept up to date on other instances. When the primary fails, a standby takes over almost immediately.

Exactly-once processing

If a crash happens after a window was updated but before the application recorded that it had read the event, the event could be processed twice, doubling the count. Kafka Streams can prevent this with exactly-once processing. When it is enabled, reading the input, updating the state, and writing the output are committed together as one atomic transaction. Either all three happen or none do. For counts that feed billing or compliance reports, this setting is well worth its small cost in throughput.

Windowing is powerful, yet every choice trades something for something else. Know the trades before you commit.

The four big trade-offs

A well-designed windowed application can handle hundreds of thousands of events per second. The secret is understanding what limits it.

Scaling in Kafka Streams follows one simple rule: parallelism equals the number of input partitions. Each partition is handled by one task, and each task is handled by one thread at a time. If your input topic has twelve partitions, you can usefully run up to twelve threads spread over any number of instances. Adding a thirteenth thread brings no benefit, because there is no task left for it. Choose the partition count at design time with growth in mind, since increasing it later changes how keys map to partitions.

What makes windowing expensive

Practical tuning levers

Databases, caching and load balancing around windows

Windowed results are rarely the end of the road. They often flow into a fast database or cache so that user-facing services can read them in milliseconds. Kafka Connect is a common way to copy the output topic into such a store. Load balancers and API gateways matter at the front door: they spread incoming events across producer instances and reject bad or unauthenticated traffic before it can pollute your timestamps. A healthy front end keeps the stream clean, which makes every window downstream more trustworthy.

A dashboard nobody can trust is worse than no dashboard. Reliability means the numbers stay right even when machines fail.

Deployments deserve special care. Restarting all instances at once forces every task to move and every store to restore. A gentle rolling restart, one instance at a time and with warm standbys ready, keeps the application producing results throughout. Newer versions of Kafka use an incremental rebalancing process that moves only the tasks that must move, further reducing disruption.

Windowed state contains real user activity. Protect it like the sensitive data it is.

You cannot fix what you cannot see. A handful of signals reveal almost every windowing problem.

Set alerts on lag and on the dropped-records rate first; they catch most trouble early. Log rebalances and restores with timestamps so that you can connect a dip in results with a specific restart. Finally, keep a small set of known-answer test streams. If a synthetic stream that should produce exactly ten events per minute suddenly reports nine, you know something in the pipeline changed.

A few proven shapes solve most windowing needs. A few tempting shortcuts cause most of the outages.

Patterns that work

Anti-patterns to avoid

A short checklist gathered from many production systems. Follow it and you avoid most windowing pain.

Mistakes seen again and again

The same five events produce four different answers depending on the window you choose. Seeing the numbers side by side makes the differences stick.

Suppose a payment company watches one card, number 4711. The card makes purchases at these times: 10:01, 10:03, 10:04, 10:07 and 10:52. The business question is “how active is this card?” Let us answer it with each window type and see how the story changes.

With tumbling windows of five minutes

The windows are 10:00 to 10:05, 10:05 to 10:10, and 10:50 to 10:55. The first window contains three purchases (10:01, 10:03, 10:04). The second contains one (10:07). The third contains one (10:52). The answers are three, one and one. Notice that the activity between 10:03 and 10:07 was really four purchases in about four minutes, but the boundary at 10:05 split them into three and one.

With hopping windows of ten minutes advancing every five

Now the windows are 10:00 to 10:10, 10:05 to 10:15, and later ones near 10:50. The window from 10:00 to 10:10 contains four purchases. The window from 10:05 to 10:15 contains only the 10:07 purchase. The 10:45 to 10:55 and 10:50 to 11:00 windows each contain the 10:52 purchase. The maximum count is now four, which is closer to the truth, and the burst is visible. The cost is that the 10:07 purchase was counted twice, once in each overlapping window.

With sliding windows and a six-minute difference

The sliding window asks, for every position, which purchases fit within six minutes of each other. The purchases at 10:01 and 10:07 are exactly six minutes apart, so all four early purchases (10:01, 10:03, 10:04 and 10:07) fit in one window, giving a count of four. The 10:52 purchase is far away, so it forms its own window with a count of one. Had the limit been five minutes, the largest group would be three, because 10:01 and 10:07 would no longer fit together. The important point is that the answer comes from the data itself and does not depend on where a clock boundary happened to fall.

With session windows and a ten-minute gap

The purchases at 10:01, 10:03, 10:04 and 10:07 are each less than ten minutes apart, so they form one session that runs from 10:01 to 10:07 with four purchases. The purchase at 10:52 is forty-five minutes after the last one, which is more than the gap, so it begins a second session with one purchase. Session windows tell you there were two visits, one busy and one quiet, which is a different and often more natural view of behavior.

Window state lives on disk and in memory. A rough calculation before launch prevents nasty surprises after it.

The basic formula is easy to remember: state entries equal the number of active keys multiplied by the number of windows kept open per key. Then multiply by the average size of one entry, including the key, the window boundaries and the aggregated value. The result is the size of the state store across the whole application, which is then divided among the instances.

Consider a payment company with five million active cards. If it uses five-minute tumbling windows with a two-minute grace period, most cards have only one or two open windows at a time, since a window stays open until its end plus grace. Suppose each entry takes about one hundred bytes. The estimate is five million times two times one hundred bytes, which is roughly one gigabyte. That is comfortable on a single machine, and easily shared across several.

Now suppose the team switches to hourly windows that advance every minute, so each event lands in sixty windows, and each card holds around sixty open windows. The estimate becomes five million times sixty times one hundred bytes, which is about thirty gigabytes. The same business idea now needs thirty times more storage, and every event triggers sixty updates. This is why the ratio of size to advance is the first number to check in a design review.

Do not forget retention and history

Retention adds to the total. If you keep windows for a day so that dashboards can look back, then the number of stored windows per key is a full day divided by the window size, not just the few that are open. Keeping five-minute windows for a day means 288 windows per key. With five million keys and one hundred bytes each, that is about 144 gigabytes. Many teams are surprised to find that retention, not the open windows, dominates their disk usage.

Time-based logic is famously hard to test. The trick is to control the clock instead of waiting for it.

Because Kafka Streams follows event time, tests do not need to wait five real minutes to see a five-minute window close. A test can feed the topology a handful of events with chosen timestamps, then feed one more event with a much later timestamp to push stream time forward and close the earlier windows. The library provides a test driver for exactly this purpose, and it runs entirely in memory without any Kafka cluster.

Debugging a wrong number

When a window count looks wrong, ask four questions in order. First, are the timestamps what you think they are? Print a few and compare them with the real times. Second, is the window boundary where you expect, remembering the alignment to UTC? Third, is the dropped-records metric rising, meaning events are arriving after their window closed? Fourth, is the consumer of the output seeing several refinements of the same window, and treating each as a new one? These four causes explain the large majority of confusing results.

Business people think in local days and months. Kafka windows think in fixed slices of UTC. Bridging the two needs care.

A window of twenty-four hours is not the same as a calendar day in every place. In a region that observes daylight saving, one day a year has twenty-three hours and another has twenty-five. A month has anywhere from twenty-eight to thirty-one days. Fixed-length windows cannot represent these calendar ideas exactly, because they always cover the same number of milliseconds.

Teams usually pick one of three approaches. The first is to accept UTC days and label the reports clearly. This is simple and consistent, and it suits global products. The second is to shift each timestamp by the local offset before windowing, so that local midnight becomes a window boundary. This works well for one region but must be handled carefully around daylight saving changes. The third is to compute hourly windows and combine them into local days in the reporting layer, where a database or query engine understands calendars properly. This third option is the most flexible and is a common choice for products that serve several time zones.

Windowing sits behind many everyday digital experiences. Here is how the ideas in this guide show up in well-known kinds of companies.

Apache Kafka was created at LinkedIn to move activity events, and it is now used across industries by companies of every size. Public engineering talks and blog posts describe patterns like the ones below. The details of each company’s internal systems differ and change over time, so treat these as illustrations of the technique rather than exact blueprints.

Several tools can window Kafka data. They share ideas but differ in details.

The biggest conceptual difference is how “time has moved on” is decided. Kafka Streams uses stream time, the largest timestamp seen by a task. Flink and Spark use watermarks, which are markers that flow through the pipeline saying “no event older than this is expected any more”. The goal is the same: decide when a window can be sealed. Kafka Streams is the lightest option to deploy, since it needs no separate cluster, while dedicated engines offer richer features for very large or very complex jobs.

Quick answers to the questions that come up most when people first use windowing.

Windowing is how Kafka-based applications turn endless events into answers tied to time.