OpenTelemetry: Unified observability

In short: An open standard with libraries for producing traces, metrics and logs from applications in a uniform way and sending them to any analysis tool.

In more detail: OpenTelemetry (OTel for short) is a project of the Cloud Native Computing Foundation. Previously every vendor had its own libraries, and switching meant reworking the code. OTel defines common APIs, SDKs and a data format (OTLP), so applications are instrumented once and the data can go to Jaeger, Prometheus, Grafana, Datadog or others.

In Depth

The three signals

• Traces: the path of a request through several services, divided into spans (individual work steps with duration and properties). A trace ID links all spans.
• Metrics: numeric series such as requests per second or memory use.
• Logs: text events that can be linked with the trace ID.

Building blocks

Instrumentation produces data, automatically (agents for Java, Node.js, Python and others capture HTTP calls and database accesses) or manually via the API. The SDK collects and processes it. The collector is a separate service that receives, filters, enriches and forwards data. Context travels between services through headers such as traceparent (W3C Trace Context).

Example of a manual span (JavaScript)

import { trace } from '@opentelemetry/api';
const tracer = trace.getTracer('shop');
await tracer.startActiveSpan('calculate-shipping', async (span) => {
  span.setAttribute('country', 'DE');
  // ... work ...
  span.end();
});

Notes

Traces are expensive if you store everything: sampling is usual, i.e. only part of the requests or all failed and slow ones. Take care not to write personal data or secrets into attributes. The project is mature to differing degrees for the various signals; traces and metrics are regarded as stable.

See also: Observability, Prometheus and metrics, Logging