[BUG] Brokers/Controllers enter crashloop when OTLP HTTP metrics exporter is enabled

[gjpin]

AutoMQ Version

1.6.4-strimzi

Operating System

Bottlerocket 1.56

Installation Method

other

Hardware Configuration

• 4 cores
• 16GB of memory

Other Relevant Software

• Kafka 3.9.0
• Strimzi 0.47
• Kubernetes 1.35

What Went Wrong?

AutoMQ crashes with the following error in the logs:

kafka java.lang.AbstractMethodError: Receiver class io.opentelemetry.exporter.sender.okhttp.internal.OkHttpHttpSenderProvider does not define or inherit an implementation of the resolved method 'abstract io.opentelemetry.exporter.internal.http.HttpSender createSender(java.lang.String, io.opentelemetry.exporter.internal.compression.Compressor, boolean, java.lang.String, long, long, java.util.function.Supplier, io.opentelemetry.exporter.internal.auth.Authenticator, io.opentelemetry.sdk.common.export.RetryPolicy, javax.net.ssl.SSLContext, javax.net.ssl.X509TrustManager)' of interface io.opentelemetry.exporter.internal.http.HttpSenderProvider.

What Should Have Happened Instead?

AutoMQ should not crash and should send the metrics in OTLP format to the defined OTLP endpoint.

Steps to Reproduce

Configure AutoMQ to export metrics via OTLP with the following configuration:

s3.telemetry.metrics.exporter.type=otlp
s3.telemetry.exporter.otlp.protocol=http
s3.telemetry.exporter.otlp.endpoint=http://${endpoint}:9090/api/v1/otlp/v1/metrics

Source: https://github.com/automq/automq/wiki/Integrating-Metrics-with-Prometheus

Additional Information

Related issue: #3111
Relevant PR that should fix this issue: #3124
Image in use: docker.io/automqinc/automq:1.6.4-strimzi

[manmohak07]

sender-okhttp is not in the global force() list. So if any transitive dependency brings in an older version, it can win the resolution.

According to this issue (gradle/gradle#7317) transitive resolution is not guaranteed.

I think we can add these two lines.

libs.opentelemetryExporterSenderOkhttp,
libs.opentelemetryExporterSenderJdk,

The global force() ensures that across all subprojects, if sender-okhttp appears anywhere transitively, it is pinned to 1.40.0 as specified in dependencies.gradle.