# OTEL Collector deployment This guide describes how to deploy and configure the OpenTelemetry (OTEL) Collector for Ripple Custody. This guide applies to on-premise deployments only. For telemetry concepts, see [Telemetry overview](/products/custody/operations-and-maintenance/monitoring/telemetry). For Ripple Custody Helm chart telemetry settings, see [Telemetry configuration](/products/custody/deployment/reference/telemetry-configuration). ## Prerequisites Before you deploy the OTEL Collector, you need: - A running Ripple Custody deployment. - An understanding of telemetry concepts such as OpenTelemetry, OTLP, traces, metrics, and logs. - Familiarity with the official OTEL Collector Helm charts. ## Deployment model The Ripple Custody Helm charts include a bundled OTEL Collector instance that is disabled by default: ```yaml opentelemetry-collector: enabled: false ``` The bundled OTEL Collector instance is for testing purposes only. Do not use it for production workloads. For production, deploy a dedicated and segregated OTEL Collector instance. You can deploy OTEL Collector in two configurations: 1. A single dedicated OTEL Collector instance for both trusted and untrusted components. 2. Two separate dedicated instances: one for trusted components and one for untrusted components. For more information on trusted and untrusted components, see [Security model](/products/custody/overview/security-model). For more information on OTEL Collector deployment, see: - Kubernetes OpenTelemetry documentation - The official OTEL Collector Helm charts The YAML snippets on this page use the `opentelemetry-collector` section from the OTEL Collector Helm `values.yaml` file, not the Ripple Custody Helm configuration. ## Enable the collector To enable the bundled OTEL Collector for testing, set the `enabled` field to `true`: ```yaml opentelemetry-collector: enabled: true ``` This deploys an OTEL Collector pod alongside the Ripple Custody deployment and sets the environment variables that let pods discover the OTEL Collector service. ## Configure resources The default resource values may be too low for a typical deployment. Run load testing at the peak usage you expect, observe resource usage, and set values accordingly. | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `resources.requests.cpu` | string | - | Minimum CPU allocation for the OTEL Collector pod | | `resources.requests.memory` | string | - | Minimum memory allocation for the OTEL Collector pod | | `resources.limits.cpu` | string | - | Maximum CPU allocation for the OTEL Collector pod | | `resources.limits.memory` | string | - | Maximum memory allocation for the OTEL Collector pod | ```yaml opentelemetry-collector: resources: requests: cpu: 4 memory: 4Gi limits: cpu: 8 memory: 16Gi ``` Resource values must be high enough that the `opentelemetry-collector` pod does not enter a `Pending` or `CrashLoopBackOff` Kubernetes state. The higher the number of users, the greater the volume of telemetry data generated, and the higher the resource consumption for OTEL Collector. ## Configure third-party contributors Add third-party contributors as parameters in the OTEL Collector configuration. The OTEL Collector Contrib GitHub repository contains three main categories: - Third-party receivers - Third-party processors - Third-party exporters Configuration recommendations: - **Receivers**: Set the appropriate network parameters for the protocol, such as TCP or UDP. - **Processors**: Configure log and trace formatting, enrichment, and trace throttling. - **Exporters**: Configure exporters to send telemetry data from OTEL Collector to another OTEL receiver or backend. ```yaml opentelemetry-collector: ports: syslog: protocol: TCP # or UDP ``` Contributions from telemetry vendors such as Microsoft, AWS, Elasticsearch, and Prometheus can be found in OTEL Collector, but not all vendors are present. If a telemetry vendor contribution exists, it might not include every feature from that vendor. ## Telemetry data reception By default, OTEL Collector listens on the OTLP ports, `4317` for gRPC and `4318` for HTTP or HTTPS, but not for the rSyslog protocol. The Ripple Custody Helm charts configure OTEL Collector to listen on port `54526` with the TCP protocol for rSyslog telemetry data. ### Syslog port configuration | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `ports.syslog.enabled` | boolean | `true` | Enable the syslog receiver port | | `ports.syslog.containerPort` | integer | `54526` | Container port for syslog reception | | `ports.syslog.servicePort` | integer | `54526` | Kubernetes service port for syslog | | `ports.syslog.hostPort` | integer | `54526` | Host port for syslog reception | | `ports.syslog.protocol` | string | `TCP` | Network protocol, `TCP` or `UDP` | ```yaml opentelemetry-collector: ports: syslog: enabled: true containerPort: 54526 servicePort: 54526 hostPort: 54526 protocol: TCP ``` ## Host telemetry data By default, OTEL Collector only receives telemetry data sent by telemetry network protocols, such as OTLP or rSyslog, from Ripple Custody components. OTEL Collector can also read and export host telemetry data, such as stdout logs and host metrics. Because OTEL Collector uses the Kubernetes API to access host telemetry data, it needs the correct Role Based Access Control (RBAC) permissions. Set appropriate RBAC permissions in the Kubernetes cluster for OTEL Collector to access the desired host telemetry data. For more information, see the OTEL Collector documentation. ### Presets configuration | Parameter | Type | Default | Description | | --- | --- | --- | --- | | `presets.logsCollection.enabled` | boolean | `false` | Enable stdout log collection from Ripple Custody components | | `presets.logsCollection.includeCollectorLogs` | boolean | `false` | Include OTEL Collector's own logs in collection | | `presets.logsCollection.storeCheckpoints` | boolean | `false` | Store log collection checkpoints for resumption | | `presets.hostMetrics.enabled` | boolean | `false` | Enable host-level metrics collection | | `presets.kubernetesAttributes.enabled` | boolean | `false` | Enrich telemetry with Kubernetes metadata | | `presets.kubernetesEvents.enabled` | boolean | `false` | Collect Kubernetes events | | `presets.clusterMetrics.enabled` | boolean | `false` | Collect cluster-wide metrics | | `presets.kubeletMetrics.enabled` | boolean | `false` | Collect kubelet metrics | Basic log collection ```yaml opentelemetry-collector: enabled: true presets: logsCollection: enabled: true ``` Full observability ```yaml opentelemetry-collector: enabled: true presets: logsCollection: enabled: true includeCollectorLogs: true storeCheckpoints: true hostMetrics: enabled: true kubernetesAttributes: enabled: true kubernetesEvents: enabled: true clusterMetrics: enabled: true kubeletMetrics: enabled: true ``` For more information, see OTEL Collector presets. ## Pipeline configuration You must update the default `pipelines` configuration for your environment. OTEL Collector executes each pipeline in three steps: receivers, processors, and exporters. There are four pipelines: - `logs/syslog` - Syslog-received logs - `logs` - OTLP-received logs - `traces` - Distributed traces - `metrics` - Application and system metrics ```yaml opentelemetry-collector: config: service: pipelines: logs/syslog: receivers: [syslog] processors: [batch] exporters: [logging] logs: receivers: [otlp] processors: [batch] exporters: [logging] traces: receivers: [otlp] processors: [tail_sampling] exporters: [logging] metrics: receivers: [otlp] processors: [batch] exporters: [logging] ``` The OTEL Collector Contrib GitHub repository provides public implementations and documentation for OTEL Collector receivers, processors, and exporters. ## Processors In OpenTelemetry, a processor sits between the receiver and exporter in the data pipeline. Processors modify telemetry data such as traces, metrics, and logs. Common tasks include filtering, batching, and data enrichment. The Helm charts provide a default processor configuration named `tail_sampling` for handling traces: ```yaml opentelemetry-collector: config: processors: tail_sampling: decision_wait: 30s num_traces: 50000 expected_new_traces_per_sec: 50 policies: [...] ``` The `tail_sampling` processor defines policies that specify rules for selecting which traces to keep for analysis. Examples include capturing domain-related traces, authenticated requests, and forced logs. Policies also filter traces based on criteria such as specific HTTP targets, sources, payloads, latency, and status codes. The `samples-some` policy is a probabilistic sampling policy that keeps a percentage of traces to reduce storage and processing costs while providing a representative sample. ## Exporter configuration Exporters transmit telemetry data from OTEL Collector to external systems or storage. They package and send telemetry data to destinations such as monitoring platforms, logging systems, or analytics tools. Elasticsearch ```yaml opentelemetry-collector: config: exporters: elasticsearch/trace: endpoints: ['http://my-elastic-search-host:9200'] traces_index: trace_index elasticsearch/log: endpoints: ['http://my-elastic-search-host:9200'] logs_index: log_index sending_queue: enabled: true num_consumers: 20 queue_size: 1000 elasticsearch/syslog: endpoints: ['http://my-elastic-search-host:9200'] logs_index: syslog_index sending_queue: enabled: true num_consumers: 20 queue_size: 1000 pipelines: logs/syslog: exporters: [elasticsearch/syslog] logs: exporters: [elasticsearch/log] traces: exporters: [elasticsearch/trace] ``` Azure Monitor ```yaml opentelemetry-collector: config: exporters: azuremonitor: endpoint: 'https://my-azure-region.applicationinsights.azure.com/' instrumentation_key: 6ac20654-450e-29e4-65e2-1bdecb7db7c4 maxbatchsize: 1024 maxbatchinterval: 10s service: pipelines: logs/syslog: exporters: [azuremonitor] logs: exporters: [azuremonitor] traces: exporters: [azuremonitor] metrics: exporters: [azuremonitor] ``` ## Related topics - [Telemetry overview](/products/custody/operations-and-maintenance/monitoring/telemetry) - [Telemetry configuration](/products/custody/deployment/reference/telemetry-configuration) - [Security model](/products/custody/overview/security-model)