Embed OTel Collector Lambda extension in connector images#61
Draft
Embed OTel Collector Lambda extension in connector images#61
Conversation
Add the opentelemetry-lambda collector extension to Lambda connector images via a multi-stage Docker build. The extension runs as a sidecar process in the Lambda execution environment, receiving OTLP telemetry on localhost and forwarding it to the central collector. This ensures telemetry is flushed reliably during Lambda shutdown or crashes. Changes: - Dockerfile template: multi-stage build downloads otel-lambda collector extension v0.16.0 (arm64) and embeds it at /opt/extensions/collector - collector.yaml: OTLP receiver on localhost, exports to central collector - GoReleaser template: add collector.yaml as extra_files for Docker context - Release workflow: copy collector.yaml into caller build context
GoReleaser detected collector.yaml as an untracked file in the caller repo. Move it to the _generated dir alongside the Dockerfile and reference via extra_files path outside the caller repo.
GoReleaser extra_files must be relative to workdir (the caller repo). Copy collector.yaml into the caller repo and add it to .git/info/exclude so GoReleaser can find it without dirtying the git state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Lambda connectors have in-process OTEL instrumentation but no reliable way to flush telemetry before Lambda shuts down, crashes, or freezes. The otel-lambda collector extension runs as a separate process in the Lambda execution environment and can flush buffered telemetry during the Shutdown phase.
What this changes
Embeds the opentelemetry-lambda collector extension into Lambda connector images via a multi-stage Docker build:
/opt/extensions/collectoralongside the connector binaryotel-collector.c1.internal:4317withinsecure_skip_verifycollector.yamlasextra_filesfor Docker build contextcollector.yamlinto caller build context during Lambda config generationDependencies
gontzess/otel-col-ext-lambda— setsBATON_OTEL_COLLECTOR_ENDPOINT=localhost:4317on Lambda functionsgontzess/otel-col-ext-lambda— points release at this workflow branch for testing