Structured logging is the practice of writing log events as machine-readable records with named fields, typically JSON objects or key-value pairs, rather than as free-form text sentences. In monitoring & logs for servers, structured logging turns each event into data that log aggregation systems can filter, count and correlate reliably without fragile parsing rules.
A traditional line such as “User 42 failed login from 10.0.0.5” becomes a record with fields like event, user_id, source_ip and outcome. The information is the same, but the structured form can be queried precisely, for example to count failed logins per source address across thousands of servers.
How structured logging represents an event
A structured log event is a set of fields with consistent names and types. Some fields appear in every event, such as timestamp, log level and service name, while others depend on the event, such as the HTTP status code of a request or the duration of a database query.
| Field | Example value | Purpose |
|---|---|---|
| timestamp | 2026-03-14T09:21:07.512Z | Ordering events across servers, in UTC |
| level | error | Filtering by log level |
| service | checkout-api | Identifying the source application |
| message | payment declined | Short human-readable summary |
| trace_id | 4bf92f3577b34da6 | Linking events of one request across services |
| duration_ms | 182 | Measuring latency as a number |
The essential shift is that context travels as data, not as prose. Instead of building a message string, code passes fields to the logger, and the logger serializes them. A human-readable message is still useful, but it should be constant per event type so that events can be grouped by it.
Common structured log formats
JSON lines, one JSON object per line, are the most widely used format because nearly every log shipper and aggregation platform parses them natively. Other formats remain common where readability or legacy compatibility matters.
| Format | Example | Strengths | Weaknesses |
|---|---|---|---|
| JSON lines | {“level”:”info”,”user_id”:42} | Nested data, universal parsing | Harder to read by eye |
| logfmt | level=info user_id=42 | Compact, readable | No standard nesting |
| Syslog structured data | RFC 5424 SD elements | Fits existing syslog pipelines | Verbose, limited tooling |
| Binary formats | systemd journal fields | Efficient indexing on the host | Needs specific tools to read |
Many teams adopt a shared field schema, such as the Elastic Common Schema or OpenTelemetry semantic conventions, so that the same concept always has the same field name. Without it, one service logs user_id, another userId and a third uid, and cross-service queries break.
Structured logging compared with plain text logs
Plain text logs are easy to write and hard to analyze, while structured logs need more discipline up front and pay off at scale. Text logs require regular expressions or grok patterns to extract values, and these patterns break whenever a developer rewords a message.
| Aspect | Plain text logs | Structured logs |
|---|---|---|
| Parsing | Regex per message format | Native field extraction |
| Querying | Full-text search | Exact filters and aggregations |
| Robustness to change | Breaks when wording changes | Stable while field names stay |
| Human reading | Natural in a terminal | Needs formatting tools |
| Metrics from logs | Fragile and approximate | Direct counts and percentiles |
Structured logging also bridges logs with other telemetry. Adding a trace ID and span ID to each event connects log lines with distributed traces, and numeric fields allow quick metrics without separate instrumentation. Access logs and error logs from web servers can be switched to JSON output in most servers, which removes an entire parsing stage.
Implementing structured logging on servers
Implementation starts in the application, with a logging library that accepts fields, and continues through the pipeline that ships, stores and indexes events. Applications should write to standard output or a local file, and a shipper forwards events to a central system such as an ELK stack or Splunk.
| Decision | Recommended choice | Reason |
|---|---|---|
| Time format | ISO 8601 in UTC with milliseconds | Unambiguous ordering across zones |
| Field naming | One shared schema for all services | Cross-service queries work |
| Context propagation | Request ID and trace ID on every event | Correlation of distributed requests |
| Sensitive data | Redaction at the logger | Passwords and tokens never leave the host |
| Cardinality | Avoid unbounded field names | Keeps indexes small and fast |
Most modern logging libraries support contextual loggers: a request handler binds fields such as request ID and user ID once, and every event logged during that request carries them automatically. This removes repetitive code and ensures consistent correlation fields.
Pitfalls of structured logging
Structured logs make data easy to collect, which makes it easy to collect too much. Volume and storage costs grow quickly when every request logs large objects.
- Logging whole objects such as request bodies exposes personal data and inflates storage.
- Dynamic field names like one field per user ID explode index mappings.
- Inconsistent types, with a field logged as a number in one service and a string in another, cause indexing errors.
- Interpolating values into the message instead of separate fields loses the benefit of structure.
- Skipping log rotation for local JSON files fills disks just as text logs do.
A practical rule is to treat log fields like an API contract: document them, review changes and avoid renaming fields that dashboards and alerts depend on.
Understanding it simply
Imagine two ways of keeping a diary. In one, you write free sentences. In the other, you fill in a form with boxes for date, place, people and cost. Structured logging is the form version of a server’s diary.
Sentences are pleasant to read but slow to search. A form lets you ask exact questions, such as how many errors happened yesterday on one server. The key benefit is that computers can answer questions about logs instantly.
Applied in practice
A software company runs twenty microservices that all wrote plain text logs in different styles. When a customer reports a failed payment, engineers spend about an hour searching each service’s logs by timestamp and guessing which lines belong together.
The team switches every service to JSON lines with a shared schema, adds a request ID generated at the API gateway and passes it to every downstream call. Logs are shipped to a central log aggregation cluster, and passwords and card numbers are redacted in the logger configuration.
Now an engineer pastes one request ID into the search field and sees the complete path of the failed payment across six services in seconds. The team also builds an alert on the count of payment_declined events per minute. Investigation time drops from about an hour to a few minutes, without adding any new monitoring agent.