OTEL Collector deployment

This guide describes how to deploy and configure the OpenTelemetry (OTEL) Collector for Ripple Custody.

Before you deploy the OTEL Collector, you need:

• A running Ripple Custody deployment.
• An understanding of telemetry concepts such as OpenTelemetry, OTLP, traces, metrics, and logs.
• Familiarity with the official OTEL Collector Helm charts.

The Ripple Custody Helm charts include a bundled OTEL Collector instance that is disabled by default:

opentelemetry-collector:
  enabled: false

You can deploy OTEL Collector in two configurations:

1. A single dedicated OTEL Collector instance for both trusted and untrusted components.
2. Two separate dedicated instances: one for trusted components and one for untrusted components.

For more information on trusted and untrusted components, see Security model.

For more information on OTEL Collector deployment, see:

• Kubernetes OpenTelemetry documentation
• The official OTEL Collector Helm charts

To enable the bundled OTEL Collector for testing, set the enabled field to true:

opentelemetry-collector:
  enabled: true

This deploys an OTEL Collector pod alongside the Ripple Custody deployment and sets the environment variables that let pods discover the OTEL Collector service.

The default resource values may be too low for a typical deployment. Run load testing at the peak usage you expect, observe resource usage, and set values accordingly.

opentelemetry-collector:
  resources:
    requests:
      cpu: 4
      memory: 4Gi
    limits:
      cpu: 8
      memory: 16Gi

Add third-party contributors as parameters in the OTEL Collector configuration. The OTEL Collector Contrib GitHub repository contains three main categories:

• Third-party receivers
• Third-party processors
• Third-party exporters

Configuration recommendations:

• Receivers: Set the appropriate network parameters for the protocol, such as TCP or UDP.
• Processors: Configure log and trace formatting, enrichment, and trace throttling.
• Exporters: Configure exporters to send telemetry data from OTEL Collector to another OTEL receiver or backend.

opentelemetry-collector:
  ports:
    syslog:
      protocol: TCP     # or UDP

By default, OTEL Collector listens on the OTLP ports, 4317 for gRPC and 4318 for HTTP or HTTPS, but not for the rSyslog protocol.

The Ripple Custody Helm charts configure OTEL Collector to listen on port 54526 with the TCP protocol for rSyslog telemetry data.

opentelemetry-collector:
  ports:
    syslog:
      enabled: true
      containerPort: 54526
      servicePort: 54526
      hostPort: 54526
      protocol: TCP

By default, OTEL Collector only receives telemetry data sent by telemetry network protocols, such as OTLP or rSyslog, from Ripple Custody components. OTEL Collector can also read and export host telemetry data, such as stdout logs and host metrics.

opentelemetry-collector:
  enabled: true

  presets:
    logsCollection:
      enabled: true

For more information, see OTEL Collector presets.

You must update the default pipelines configuration for your environment. OTEL Collector executes each pipeline in three steps: receivers, processors, and exporters.

There are four pipelines:

• logs/syslog - Syslog-received logs
• logs - OTLP-received logs
• traces - Distributed traces
• metrics - Application and system metrics

opentelemetry-collector:
  config:

    service:
      pipelines:
        logs/syslog:
          receivers: [syslog]
          processors: [batch]
          exporters: [logging]
        logs:
          receivers: [otlp]
          processors: [batch]
          exporters: [logging]
        traces:
          receivers: [otlp]
          processors: [tail_sampling]
          exporters: [logging]
        metrics:
          receivers: [otlp]
          processors: [batch]
          exporters: [logging]

The OTEL Collector Contrib GitHub repository provides public implementations and documentation for OTEL Collector receivers, processors, and exporters.

In OpenTelemetry, a processor sits between the receiver and exporter in the data pipeline. Processors modify telemetry data such as traces, metrics, and logs. Common tasks include filtering, batching, and data enrichment.

The Helm charts provide a default processor configuration named tail_sampling for handling traces:

opentelemetry-collector:
  config:

    processors:
      tail_sampling:
        decision_wait: 30s
        num_traces: 50000
        expected_new_traces_per_sec: 50
        policies: [...]

The tail_sampling processor defines policies that specify rules for selecting which traces to keep for analysis. Examples include capturing domain-related traces, authenticated requests, and forced logs. Policies also filter traces based on criteria such as specific HTTP targets, sources, payloads, latency, and status codes. The samples-some policy is a probabilistic sampling policy that keeps a percentage of traces to reduce storage and processing costs while providing a representative sample.

Exporters transmit telemetry data from OTEL Collector to external systems or storage. They package and send telemetry data to destinations such as monitoring platforms, logging systems, or analytics tools.

opentelemetry-collector:
  config:

    exporters:

      elasticsearch/trace:
        endpoints: ['http://my-elastic-search-host:9200']
        traces_index: trace_index
      elasticsearch/log:
        endpoints: ['http://my-elastic-search-host:9200']
        logs_index: log_index
        sending_queue:
          enabled: true
          num_consumers: 20
          queue_size: 1000
      elasticsearch/syslog:
        endpoints: ['http://my-elastic-search-host:9200']
        logs_index: syslog_index
        sending_queue:
          enabled: true
          num_consumers: 20
          queue_size: 1000

      pipelines:
        logs/syslog:
          exporters: [elasticsearch/syslog]
        logs:
          exporters: [elasticsearch/log]
        traces:
          exporters: [elasticsearch/trace]

• Telemetry overview
• Telemetry configuration
• Security model