OpenTelemetry is an open-source, vendor-neutral observability framework that defines how applications and infrastructure produce, describe and transmit telemetry: traces, metrics and logs. It provides APIs, language SDKs, automatic instrumentation, a wire protocol called OTLP and a standalone Collector, so that telemetry can be generated once and sent to any compatible backend.
In cloud observability, OpenTelemetry addresses a long-standing problem: each monitoring vendor used to require its own agents and libraries, which made switching tools expensive and left gaps between services. OpenTelemetry, a Cloud Native Computing Foundation project formed in 2019 by merging OpenTracing and OpenCensus, separates instrumentation from the analysis backend, making telemetry portable across vendors and open-source tools.
Components of OpenTelemetry
OpenTelemetry is a set of specifications and implementations rather than a single product. It does not store or visualize data; that remains the job of a backend such as a tracing system, a metrics database or a commercial observability platform. The table below lists the main building blocks.
| Component | Role | Where it runs |
|---|---|---|
| API | Stable interfaces for creating spans, metrics and log records | Inside application code and libraries |
| SDK | Implements sampling, batching, resource detection and export | Inside the application process |
| Instrumentation libraries | Automatic telemetry for frameworks, HTTP clients, databases | Inside the application process |
| OTLP | Protocol for transmitting telemetry over gRPC or HTTP | Between processes and services |
| Collector | Receives, processes and exports telemetry | Agent on a host or central gateway |
| Semantic conventions | Standard attribute names such as http.request.method | Shared vocabulary across all signals |
The split between API and SDK is deliberate. A library author can instrument code against the lightweight API without forcing any configuration on users; if no SDK is installed, the calls do nothing. The application owner then installs the SDK and decides where data goes.
How telemetry flows through OpenTelemetry
A request entering a service creates a span, a timed operation with attributes, events and a status. When that service calls another one, the SDK injects the trace context into the outgoing request, usually as the W3C traceparent header. The receiving service extracts it and creates child spans, so all spans of one request share a trace ID. This mechanism, called context propagation, is what makes distributed tracing possible.
Telemetry then leaves the process, typically through an OTLP exporter, on default ports 4317 for gRPC and 4318 for HTTP. Many deployments send it first to a local or central Collector, which runs configurable pipelines made of three stages.
| Stage | Purpose | Typical examples |
|---|---|---|
| Receivers | Accept incoming data | OTLP, Prometheus scraping, host metrics, log files |
| Processors | Transform, filter or sample | Batching, memory limiting, attribute redaction, tail sampling |
| Exporters | Send data to backends | OTLP to a vendor, Prometheus remote write, object storage |
Every signal also carries a resource, a set of attributes describing the source, such as service name, version, host, container and cloud region. Consistent resource attributes are what later allow a backend to correlate traces, metrics and logs from the same service.
OpenTelemetry compared with related approaches
OpenTelemetry overlaps with several older or narrower tools. It is best understood as the instrumentation and transport layer that can feed many of them.
| Approach | Scope | Relationship to OpenTelemetry |
|---|---|---|
| Vendor agents | Proprietary collection for one platform | Increasingly replaced or wrapped by OpenTelemetry |
| Prometheus | Metrics collection and storage | Collector can scrape and export Prometheus metrics |
| Log shippers | Forwarding log files | Collector can read files; SDK can emit structured logs |
| OpenTracing and OpenCensus | Earlier tracing and metrics standards | Merged into OpenTelemetry and retired |
| eBPF-based tools | Kernel-level telemetry without code changes | Can export data in OpenTelemetry formats |
The project matures signal by signal. Tracing and metrics are stable in the major languages, the logs data model and bridges to existing logging libraries are widely usable, and newer signals such as continuous profiling are still evolving. Maturity also differs between language SDKs, so checking the status for a specific language is part of planning.
Adopting OpenTelemetry in practice
A common path starts with automatic instrumentation, which uses agents or library hooks to capture HTTP, gRPC, database and messaging calls without code changes. Teams then add manual spans around important business operations and attach domain attributes, such as order type or tenant tier, that no automatic tool can know.
- Set service.name and deployment environment consistently, because backends group everything by them.
- Route telemetry through a Collector so that backends, sampling and redaction can change without redeploying applications.
- Use tail-based sampling in the Collector to keep all error and slow traces while sampling routine ones.
- Follow semantic conventions instead of inventing attribute names, so dashboards work across services.
- Remove personal data and secrets with redaction processors before export.
Pitfalls and limitations
The most expensive mistake is high-cardinality attributes on metrics, such as user IDs or full URLs, which multiply the number of time series and can overload a metrics backend. Such details belong on spans or logs. Another common gap is broken context propagation across message queues, proxies or asynchronous jobs, which splits one logical request into several unrelated traces.
Teams should also budget for overhead and volume. Instrumentation adds small CPU and memory costs to each service, and unsampled tracing in busy systems can produce more data than the analysis backend can economically store. OpenTelemetry standardizes collection, but it does not decide what is worth collecting; that remains an engineering decision tied to service level objectives.
In plain words
Imagine a factory where every machine reports its status in a different language. Each inspection company then needs its own translators. OpenTelemetry is a shared language for software. It describes how long each step took, how many requests arrived and what went wrong.
Because the reports follow one standard, a company can change its monitoring tool without rewiring all its software. It can also follow a single customer request as it passes through dozens of services.
A real-world example
An online travel agency runs about 60 microservices in Kubernetes and uses two monitoring products acquired at different times. Engineers cannot follow a booking from the search page to the payment provider, because each product sees only part of the system. The team installs OpenTelemetry auto-instrumentation in its Java and Node.js services and deploys a Collector as an agent on every node, plus a central gateway.
The gateway applies tail-based sampling, keeping every trace with an error or a duration above two seconds and 5 percent of the rest. Within a few weeks, a slow booking trace reveals a hotel availability call retried three times inside a loop. Fixing it cuts median booking time by about a third, and when the company later consolidates on one vendor, it changes only the Collector exporter configuration.