Adding documentation on otel with vector#835
Conversation
✅ Deploy Preview for stackable-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
xeniape
moved this from Development: Waiting for Review
to Development: In Review
in Stackable Engineering
xeniape
left a comment
xeniape
left a comment
There was a problem hiding this comment.
Read it kind of "blind" as a non-expert, so maybe my confusions/questions could be used improve the documentation for other users not familiar with the topic. Keeping that in mind, my suggestions might not be the best, they are no blockers :)
|
|
||
| == OpenTelemetry with Vector | ||
|
|
||
| OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`: |
There was a problem hiding this comment.
What does "behave like product logging" mean exactly? A short explanation what that refers to, would be nice.
|
|
||
| == OpenTelemetry with Vector | ||
|
|
||
| OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`: |
There was a problem hiding this comment.
| OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[vector helmChart] requires to configure `containerPorts` and `service.ports` as well as `customConfig.sources.otel`: | |
| OpenTelemetry signals can be configured to behave like xref:logging.adoc[product logging]. Using the {vector-helm-chart}[Vector Helm Chart{external-link-icon}^] requires to configure the Helm values `containerPorts`, `service.ports`, and `customConfig.sources.otel`: |
There was a problem hiding this comment.
Adjusted the suggestion to include the external link indicator for the external Helm Chart
| telemetry: | ||
| otelLogExporter: | ||
| enabled: true | ||
| endpoint: http://vector-aggregator.<namespace>.svc.cluster.local:4317 |
There was a problem hiding this comment.
Actually, if there is a specific reason why this port was used and not otel-http for example, even though they are defined in Vector, feel free to elaborate why. If any of them can be used, maybe the above sentence can be adjusted to reflect that.
There was a problem hiding this comment.
The way I understood it, you can choose whatever port you want as long as you expose it in the aggregator.yaml shown above.
I understood that the gRPC standard port is 4317 and thus I kept the convention. Technically you could choose anything. I'm now not sure, I'd like to preserve some kind of convention and story throughout the guide. I think we could introduce <gRPC-port> and <http-port> if you like it better?
| endpoint: http://vector-aggregator.<namespace>.svc.cluster.local:4317 | ||
| ---- | ||
|
|
||
| === Normalizing operator logs to the Stackable log schema |
There was a problem hiding this comment.
Just a bit confused, above the sections starts with OpenTelemetry signals (but behaving like logs?), but here the subsection is about logs, so something different? So either that's something general for logs, so maybe needs to be a separate 2nd level section (==) or at least for me it needs a bit more of an explanation connecting the two concepts here.
There was a problem hiding this comment.
Good catch, hope it's better now: Adressed with 3c4d60f
|
|
||
| === Normalizing operator logs to the Stackable log schema | ||
|
|
||
| Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the |
There was a problem hiding this comment.
VRL? I think the abbrevation written out (or written out in parentheses next to it) would be more useful for people not knowing what that is (me :D)
There was a problem hiding this comment.
Vector Remap Language. I shall do, thanks
| Operators do not run vector sidecars. Using OpenSearch, a VRL transform that maps the OTLP fields onto the | ||
| xref:tutorials:logging-vector-aggregator.adoc#_watch_the_logs[Stackable log schema] is needed to avoid re-indexing: | ||
|
|
||
| [source,yaml] |
There was a problem hiding this comment.
Are these Helm values for Opensearch now? (Because of "using Opensearch" above) Maybe this also needs a linked Helm Chart then or mentioning that it's our own Opensearch Operator Helm Chart? And a clarification where to set the values.
There was a problem hiding this comment.
Do you mean the following customConfig ?
If so, no. It's a map that defines how to map otel log fields so they fit the same schema as stackable products would look like. Open search is just a very good example for a indexed backend. Since it's in our stack you are right that we might wanna hint it here. I'll fix it.
|
|
||
| <1> The `opentelemetry` source fans out into `otel.logs`, `otel.metrics`, and | ||
| `otel.traces`. Unconsumed sub-streams produce a startup warning. | ||
| <2> Operators emit `service.name` (e.g. `stackable-airflow-operator`) as their |
There was a problem hiding this comment.
Is service.name correct here? Or should it be service_name like in the line it references?
There was a problem hiding this comment.
It's service.name from the operator pov and only caught here as service_name as I don't want an attribute to the type service but rather something storing away that information
Co-authored-by: Techassi <git@techassi.dev> Co-authored-by: Xenia <xeniafischer@hotmail.de>
3 participants
This PR adds documentation about how to connect otel logs emitted by operators to a vector sink matching the product logs.