Event time is the timestamp of the moment an event actually occurred at its source, such as when a sensor took a reading, a user clicked a button or a payment was authorized. In streaming data, it is contrasted with processing time, the moment a system handles the event, and with ingestion time, the moment the event entered the pipeline.
Processing by event time makes results reflect what happened in the real world rather than when data happened to arrive. Because events can be delayed or arrive out of order, event-time processing requires mechanisms such as watermarks, windows and rules for late data.
How event time differs from other time domains
Streaming systems distinguish several time domains, and the gap between them, often called event-time skew, is the core reason event time is hard. Network delays, mobile devices going offline, buffering in a message broker and retries can all separate when an event happened from when it is processed.
The table below compares the three time domains used in stream processing.
| Time domain | Assigned by | Deterministic on replay | Typical use |
|---|---|---|---|
| Event time | The source, when the event occurs | Yes | Correct business aggregations |
| Ingestion time | The broker or entry point | Mostly | Approximation when sources lack clocks |
| Processing time | The operator’s local clock | No | Low-latency monitoring, timeouts |
The key property is determinism. Reprocessing a stored event stream by event time produces the same results every time, while processing-time results change with load, outages and replay speed. That makes event time essential for backfills, audits and reproducible analytics.
Watermarks and windows in event-time processing
Event-time processing groups events into windows based on their timestamps and relies on watermarks to decide when a window is complete. A watermark is a moving assertion that no more events with timestamps earlier than T are expected; when it passes the end of a window, the window can emit its result.
Watermarks are heuristics. A common strategy sets the watermark to the maximum observed timestamp minus a bounded delay, for example a few seconds for server events or several minutes for mobile clients. Too short a delay drops valid events as late; too long a delay increases latency.
| Window type | How it groups events | Typical use |
|---|---|---|
| Tumbling window | Fixed, non-overlapping intervals | Counts per minute or per hour |
| Sliding window | Fixed length, overlapping by a step | Moving averages and rolling rates |
| Session window | Activity separated by a gap of inactivity | User sessions, device bursts |
| Global window with triggers | All events, emitted on custom conditions | Running totals with periodic updates |
Frameworks such as Apache Flink, Apache Beam and Kafka Streams implement these concepts with slightly different vocabulary. Beam expresses them as the questions of what, where, when and how: what is computed, where in event time, when results are emitted and how later refinements relate to earlier ones.
Handling late and out-of-order events
An event is late when it arrives after the watermark has passed its timestamp. Every event-time pipeline needs an explicit late-data policy, because some delays will always exceed the watermark heuristic, especially with mobile, IoT or cross-region sources.
| Strategy | Behavior | Trade-off |
|---|---|---|
| Drop late events | Ignore events behind the watermark | Simple but loses data |
| Allowed lateness | Keep window state open for an extra period | More memory, updated results |
| Side output | Route late events to a separate stream | Preserves data for later correction |
| Retractions and updates | Emit corrected results downstream | Consumers must handle changes |
| Batch reconciliation | Recompute periods later from storage | Accurate but delayed |
The choice depends on how consumers use the output. A real-time dashboard can usually tolerate small corrections, while a billing system needs either a generous allowed lateness or a reconciliation process that guarantees completeness before invoices are produced.
Event time in relation to the streaming stack
Event time is carried by the data, not by the infrastructure. A message broker such as Kafka stores records in order of arrival within each partition and attaches its own timestamp, which may be the producer’s create time or the broker’s log-append time depending on configuration.
Ordering in an event stream is therefore arrival order, not event-time order. Partitioning by key keeps events for one entity together, but events from different partitions interleave unpredictably, so stream processors must merge watermarks across partitions, typically by taking the minimum. An idle partition can stall that minimum, which is why frameworks offer idleness detection.
Event-time processing also interacts with backpressure. When downstream operators slow down, processing time drifts further from event time, but event-time results remain correct; only their latency grows.
Pitfalls and common mistakes
Most event-time problems come from unreliable timestamps or from misjudging lateness. The table below lists typical symptoms and their fixes.
| Symptom | Likely cause | Fix |
|---|---|---|
| Windows never close | Idle partition holds back the watermark | Enable idleness handling |
| Many events counted as late | Watermark delay too short | Measure skew and widen the bound |
| Events dated in the future | Unsynchronized device clocks | Validate and clamp timestamps |
| Results shift by hours | Time zone or local-time formatting | Store timestamps in UTC |
| Totals differ from batch reports | Late data dropped silently | Add side outputs and reconciliation |
Timestamps should be recorded as close to the source as possible, in UTC, with sufficient precision, typically milliseconds. Assigning event time at ingestion for convenience quietly converts an event-time pipeline into an ingestion-time one.
In plain words
Picture letters sent by post. Each letter has the date it was written, and a separate date when it reaches your mailbox. Event time is the date the letter was written, not the day it arrived.
Data systems face the same issue. A phone may record a purchase while offline and send it an hour later. If the system counted it by arrival, the sale would land in the wrong hour. Using event time keeps every record in the moment it truly happened, even when it shows up late.
A real-world example
A ride-hailing platform computes driver earnings and trip counts in five-minute tumbling windows by event time. Driver phones often lose connectivity in tunnels and underground garages, so some trip events arrive up to ten minutes late. The pipeline uses a watermark with a two-minute bounded delay for fast dashboards and an allowed lateness of 30 minutes that updates affected windows.
Events later than 30 minutes go to a side output, and a nightly batch job reconciles the day’s totals from storage. When a regional network outage delays thousands of events by twenty minutes, the live dashboard shows a temporary dip, then corrects itself as windows update. Daily earnings statements match the reconciled totals exactly, because every trip is counted in the interval when it actually happened.