Docs

Docs

Datadog Integration

Learn how to export Observability Kit metrics and traces to Datadog.

Datadog is a commercial service for collecting and viewing telemetry data, with support for metrics, traces, and logs. Datadog doesn’t accept the OpenTelemetry Protocol (OTLP) directly, so an OpenTelemetry collector is used as a proxy: the kit’s metrics and traces are exported to the collector over OTLP, and the collector’s Datadog exporter converts and forwards them to Datadog’s API.

Account Setup

To set up an account, go to the Datadog website and click Free Trial. Select the appropriate region for storing data, fill in the remaining details, and follow the instructions.

Completing sign-up involves installing one of Datadog’s agents to send some data. That agent isn’t required for this integration, so it can be removed immediately afterwards. An easy way to run it once is the Docker option: create a container, send some data, then stop and remove the container.

OpenTelemetry Collector Setup

Create a collector.yaml configuration file that receives OTLP and exports to Datadog:

Source code
collector.yaml
receivers:
  otlp:
    protocols:
      http:
      grpc:

processors:
  batch:

exporters:
  datadog:
    api:
      site: DD_SITE
      key: DD_API_KEY

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [datadog]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [datadog]

Replace the following placeholders:

DD_SITE

The domain of the Datadog region you registered for. Read it from the browser’s address bar when logged into the Datadog UI — for example datadoghq.com for US1, or datadoghq.eu for the EU region.

DD_API_KEY

An API key for authenticating against the Datadog API. To create one, open the Datadog web UI, go to Organization Settings then API Keys, and create a key.

Run the collector either by downloading the binary or with Docker:

Download the binary from the latest release on GitHub. Download the otelcol_contrib_ variant — it includes the Datadog exporter — for your operating system.

Then run the collector:

Source code
terminal
otelcol-contrib --config collector.yaml
terminal
terminal
terminal

Configure the Application

Add Actuator with the OTLP metrics registry, and a tracing bridge with the OTLP exporter:

Source code
pom.xml
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-otlp</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-otlp</artifactId>
</dependency>

Point both pipelines at the local collector:

Source code
application.properties
spring.application.name=vaadin

# Metrics over OTLP to the collector
management.otlp.metrics.export.url=http://localhost:4318/v1/metrics

# Traces over OTLP to the collector
management.otlp.tracing.endpoint=http://localhost:4318/v1/traces
management.tracing.sampling.probability=1.0

If the collector runs on another host, change the URLs accordingly.

Logging Setup

Datadog doesn’t ingest logs over OTLP. The following work-around sends logs to the Datadog logs API and correlates them with the kit’s traces.

Logback Configuration

This method uses a Logback TCP appender to send logs to Datadog while adding the metadata needed to correlate logs with traces.

Add a TCP appender for Logback:

Source code
XML
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>7.4</version>
</dependency>

Micrometer Tracing places the current traceId and spanId in the logging MDC. To correlate logs with traces, convert those to the dd.trace_id and dd.span_id fields Datadog expects. Create a DataDogContextProvider class:

Source code
Java
package com.example.application;

import ch.qos.logback.classic.spi.ILoggingEvent;
import com.fasterxml.jackson.core.JsonGenerator;
import net.logstash.logback.composite.AbstractJsonProvider;

import java.io.IOException;
import java.util.Map;

public class DataDogContextProvider extends AbstractJsonProvider<ILoggingEvent> {
    @Override
    public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException {
        Map<String, String> mdc = event.getMDCPropertyMap();

        if (mdc.containsKey("traceId")) {
            String traceId = mdc.get("traceId");
            String low64 = traceId.substring(traceId.length() - 16);
            long ddTraceId = Long.parseUnsignedLong(low64, 16);
            generator.writeStringField("dd.trace_id", Long.toUnsignedString(ddTraceId));
        }

        if (mdc.containsKey("spanId")) {
            String spanId = mdc.get("spanId");
            long ddSpanId = Long.parseUnsignedLong(spanId, 16);
            generator.writeStringField("dd.span_id", Long.toUnsignedString(ddSpanId));
        }
    }
}

This reads the traceId and spanId from the MDC and writes the corresponding Datadog IDs into the JSON output.

Next, create a custom encoder that registers the provider. It extends LogstashEncoder, which produces JSON that the Datadog logging endpoint can process:

Source code
Java
package com.example.application;

import ch.qos.logback.classic.spi.ILoggingEvent;
import net.logstash.logback.LogstashFormatter;
import net.logstash.logback.composite.AbstractCompositeJsonFormatter;
import net.logstash.logback.encoder.LogstashEncoder;

public class DataDogLogstashEncoder extends LogstashEncoder {
    @Override
    protected AbstractCompositeJsonFormatter<ILoggingEvent> createFormatter() {
        AbstractCompositeJsonFormatter<ILoggingEvent> formatter = super.createFormatter();
        ((LogstashFormatter) formatter).addProvider(new DataDogContextProvider());
        return formatter;
    }
}

Add a TCP appender that uses the encoder to your Logback configuration:

Source code
XML
<appender name="JsonTcp" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>tcp-intake.logs.${DD_SITE}:443</destination>
    <keepAliveDuration>20 seconds</keepAliveDuration>
    <encoder class="com.example.application.DataDogLogstashEncoder">
        <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
            <layout class="ch.qos.logback.classic.PatternLayout">
                <pattern>${DD_API_KEY} %mdc{keyThatDoesNotExist}</pattern>
            </layout>
        </prefix>
    </encoder>
    <ssl />
</appender>

The configuration references two variables, which use the same values as the collector configuration:

DD_SITE

The domain of your Datadog region.

DD_API_KEY

An API key for authenticating against the Datadog API.

Pass them when running the application, for example:

Source code
terminal
java -DDD_SITE=datadoghq.eu -DDD_API_KEY=1234ABCD -jar myapp.jar

Finally, register the appender for the root logger:

Source code
XML
<root level="info">
    <appender-ref ref="JsonTcp" />
    <!-- ...other appenders -->
</root>

Viewing Data

Viewing Traces

In the Datadog web UI, select APM then Traces. The view graphs the number of requests, failed requests, and latency, along with a list of the latest traces.

The search bar filters by tags, which are span attribute values. Click a trace to see the tags recorded for its spans; click a tag to filter on that value. Clicking a trace ID opens a side panel with detailed information such as nested spans, attributes, and related logs.

See the Datadog Trace Explorer documentation for more.

Viewing Metrics

In the Datadog web UI, select Metrics then Explorer. Enter one of the metrics collected by the kit — for example vaadin.sessions.active — to view it.

See the Datadog Metrics Explorer documentation for more.

Viewing Logs

Log integration is limited to the Logback logging framework, as described above.

In the Datadog web UI, select Logs. The view lists recent log entries, which can be filtered by text or log level.

See the Datadog Logs Explorer documentation for more.

Updated –