Deploy the Consul telemetry collector

Warning

HashiCorp will deprecate HCP Consul Central on November 6, 2024. Learn more.

This page describes the process to deploy the Consul telemetry collector for cluster observability. HCP Consul Central uses the collected data to visualize service mesh metrics. Refer to Consul observability for more information.

Overview

The Consul telemetry collector is an implementation of Envoy's StatsSink service interface that you deploy as a service in your service mesh. After deploying the Consul telemetry collector, Envoy proxies push metrics to it. The telemetry collector exports these metrics to HCP Consul Central.

Image of servers and consul telemetry collector both pushing metrics to hcp

Prerequisites

To deploy the telemetry collector manually, you must meet the following prerequisites:

The Consul cluster must be running on Kubernetes.
The Consul cluster must be managed by or linked to HCP Consul Central.
Consul on Kubernetes users must use consul-k8s v1.1.2 or later.
Helm users must update their chart to use new Consul telemetry collector configuration values.

Automated collector deployment

The Consul telemetry collector supports automated proxy telemetry collection for self-managed consul-k8s clusters that link to HCP Consul using the cloud preset. Refer to link self-managed Community and Enterprise clusters to HCP Consul for more information about this process.

Deploy the collector manually

The workflow to manually deploy the telemetry collector consists of the following steps:

Update the cluster's configuration
Create service intentions to authorize the collector
Restart proxies to pick up the new Envoy bootstrap configuration

Upgrade cluster configuration

You must enable the telemetry collector by updating your cluster's configuration. To access the current configuration, run the appropriate command for your runtime:

$ consul-k8s config read > values.yaml

Update the configuration file by adding the following values:

The following example example configuration highlights these values in context:

connectInject:
  enabled: true
controller:
  enabled: true

telemetryCollector:
  enabled: true
  cloud:
    clientId:
      secretKey: client-id
      secretName: consul-hcp-client-id
    clientSecret:
      secretKey: client-secret
      secretName: consul-hcp-client-secret

global:
  metrics:
    enableTelemetryCollector: true
  cloud:
    enabled: true
    clientId:
      secretKey: client-id
      secretName: consul-hcp-client-id
    clientSecret:
      secretKey: client-secret
      secretName: consul-hcp-client-secret
    resourceId:
      secretKey: resource-id
      secretName: consul-hcp-resource-id
  acls:
    manageSystemACLs: true
  datacenter: mesh-metrics
  name: consul
  tls:
    enableAutoEncrypt: true
    enabled: true

After updating the configuration file, apply it to your Kubernetes cluster.

$ consul-k8s upgrade -f values.yaml

Authorize collector with service intentions

The Consul telemetry collector runs as a service in the mesh. To get metrics from other service's proxies, you need to create a service intention that authorizes proxies to push metrics to the collector.

Note

When you install the demo application by including the -demo flag, consul-k8s automatically creates a wildcard service intention for the Consul telemetry collector as part of the installation process.

Create a service-intentions configuration entry that allows all traffic to consul-telemetry-collector:

apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceIntentions
metadata:
  name: consul-telemetry-collector
spec:
  destination:
    name: consul-telemetry-collector
  sources:
  - action: allow
    name: '*'

Create the configuration entry:

$ kubectl apply --namespace consul --filename consul-telemetry-collector.yaml

Restart Envoy proxies to pick up new bootstrap configuration

Consul configures Envoy to send metrics to the Consul Telemetry Collector in its bootstrap configuration.

If this is your first time deploying the Consul Telemetry Collector, you will need to restart pods to pick up the changed configuration. For example:

kubectl rollout restart deployments

Forward metrics to another collector

It is possible to forward metrics from the Consul Telemetry Collector to a compatible OpenTelemetry Collector.

For more information on setting up a compatible OpenTelemetry collector, visit this documentation.

At minimum, the compatible collector must be configured with an OTLP HTTP receiver and a metrics pipeline that uses the OTLP receiver.

To forward metrics from the Consul Telemetry Collector, add the customExporterConfig field to the telemetryCollector stanza of the configuration file. Set the exporter_config stanza to configure an exporter and forward metrics to an OTLP compatible endpoint. The following examples demonstrate these configurations:

Config using the otlphttp exporter:

telemetryCollector:
  enabled: true
  customExporterConfig: |-
    {
      "exporter_config": {
        "otlphttp": {
          "endpoint": "otel.example.com",
          "headers": {
            "authorization": "<auth>"
          },
          "timeout": "2s"
        }
      }
    }

Config using the otlp exporter:

telemetryCollector:
  enabled: true
  customExporterConfig: |-
    {
      "exporter_config": {
        "otlp": {
          "endpoint": "otel.example.com",
          "headers": {
            "authorization": "<auth>"
          },
          "timeout": "2s"
        }
      }
    }

Apply the updated configuration to your deployment to begin forwarding metrics.