Skip to content

Adding documentation on otel with vector#835

Open
Maleware wants to merge 4 commits intomainfrom
docs/otel-with-vector
Open

Adding documentation on otel with vector#835
Maleware wants to merge 4 commits intomainfrom
docs/otel-with-vector

Conversation

@Maleware
Copy link
Copy Markdown
Member

This PR adds documentation about how to connect otel logs emitted by operators to a vector sink matching the product logs.

@netlify
Copy link
Copy Markdown

netlify bot commented Apr 16, 2026

Deploy Preview for stackable-docs ready!

Name Link
🔨 Latest commit c4a56b0
🔍 Latest deploy log https://app.netlify.com/projects/stackable-docs/deploys/69e62585a449340008330538
😎 Deploy Preview https://deploy-preview-835--stackable-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@Maleware Maleware marked this pull request as ready for review April 17, 2026 07:10
@Maleware Maleware moved this to Development: Waiting for Review in Stackable Engineering Apr 17, 2026
@xeniape xeniape self-requested a review April 20, 2026 13:15
@xeniape xeniape moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Apr 20, 2026
Copy link
Copy Markdown
Member

@Techassi Techassi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done, just two small comments.


== OpenTelemetry with Vector

OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`:
OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[Vector Helm Chart] requires `containerPorts`, `service.ports`, and `customConfig.sources.otel` to be configured:


[NOTE]
====
Namespace information is currently not emitted by the operator
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Namespace information is currently not emitted by the operator
Namespace information is currently not emitted by the operator.

Copy link
Copy Markdown
Member

@xeniape xeniape left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read it kind of "blind" as a non-expert, so maybe my confusions/questions could be used improve the documentation for other users not familiar with the topic. Keeping that in mind, my suggestions might not be the best, they are no blockers :)


== OpenTelemetry with Vector

OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "behave like product logging" mean exactly? A short explanation what that refers to, would be nice.


== OpenTelemetry with Vector

OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`:
Copy link
Copy Markdown
Member

@xeniape xeniape Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`:
OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[Vector Helm Chart,window=_blank] requires to configure the Helm values `containerPorts`, `service.ports`, and `customConfig.sources.otel`:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted the suggestion to include the external link indicator for the external Helm Chart (why is the suggestion italic now?? 😆 )

address: 0.0.0.0:4318
----

The `endpoint` in the operator's Helm values must point to the ports defined above.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `endpoint` in the operator's Helm values must point to the ports defined above.
The `endpoint` Helm values for the telemetry exporters in the Stackable Operator Helm Chart must point to the otel-grpc ports defined above.

telemetry:
otelLogExporter:
enabled: true
endpoint: http://vector-aggregator.<namespace>.svc.cluster.local:4317
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, if there is a specific reason why this port was used and not otel-http for example, even though they are defined in Vector, feel free to elaborate why. If any of them can be used, maybe the above sentence can be adjusted to reflect that.

endpoint: http://vector-aggregator.<namespace>.svc.cluster.local:4317
----

=== Normalizing operator logs to the Stackable log schema
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a bit confused, above the sections starts with OpenTelemetry signals (but behaving like logs?), but here the subsection is about logs, so something different? So either that's something general for logs, so maybe needs to be a separate 2nd level section (==) or at least for me it needs a bit more of an explanation connecting the two concepts here.


=== Normalizing operator logs to the Stackable log schema

Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the
Copy link
Copy Markdown
Member

@xeniape xeniape Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the
Stackable Operators do not run Vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the


=== Normalizing operator logs to the Stackable log schema

Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VRL? I think the abbrevation written out (or written out in parentheses next to it) would be more useful for people not knowing what that is (me :D)

Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the
xref:tutorials:logging-vector-aggregator.adoc#_watch_the_logs[Stackable log schema] is needed to avoid re-indexing:

[source,yaml]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these Helm values for Opensearch now? (Because of "using Opensearch" above) Maybe this also needs a linked Helm Chart then or mentioning that it's our own Opensearch Operator Helm Chart? And a clarification where to set the values.

Comment on lines +179 to +180
<1> The `opentelemetry` source fans out into `otel.logs`, `otel.metrics`, and
`otel.traces`. Unconsumed sub-streams produce a startup warning.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<1> The `opentelemetry` source fans out into `otel.logs`, `otel.metrics`, and
`otel.traces`. Unconsumed sub-streams produce a startup warning.
<1> The `opentelemetry` source type diverges into `otel.logs`, `otel.metrics`, and
`otel.traces` signals. Unconsumed sub-streams produce a startup warning.

(Don't know if "signals" is the right word, but some kind of naming would sound better I think. Oh, or is sub-stream the correct term? Then feel free to use that instead of signal in the suggestion)


<1> The `opentelemetry` source fans out into `otel.logs`, `otel.metrics`, and
`otel.traces`. Unconsumed sub-streams produce a startup warning.
<2> Operators emit `service.name` (e.g. `stackable-airflow-operator`) as their
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is service.name correct here? Or should it be service_name like in the line it references?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Development: In Review

Development

Successfully merging this pull request may close these issues.

3 participants