Use HTTP tracer if env var is set

[trajano]

Fixes #5743

Tested via example from https://github.com/trajano/trajano-portfolio/blob/master/Dockerfile#L17-L24

FROM caddy:builder AS builder
RUN git clone https://github.com/trajano/caddy /mnt/
RUN xcaddy build \
  --with github.com/caddyserver/caddy/v2=github.com/trajano/caddy/v2@ea6a6298f5ef94efef2900af22d200583ca09f3f

FROM caddy:alpine
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

Using Grafana.com to receive the OTEL traces.

This would require a change to the tracing documentation https://caddyserver.com/docs/caddyfile/directives/tracing#tracing

Enables integration with OpenTelemetry tracing facilities, using [opentelemetry-go](https://github.com/open-telemetry/opentelemetry-go) and [opentelemetry-go-contrib](https://github.com/open-telemetry/opentelemetry-go-contrib) .

When enabled, it will propagate an existing trace context or initialize a new one.

It uses the exporter protocol specified by `OTEL_EXPORTER_OTLP_PROTOCOL` and the uses [gRPC](https://github.com/grpc/) as the default.  In addition it uses W3C [tracecontext](https://www.w3.org/TR/trace-context/) and [baggage](https://www.w3.org/TR/baggage/) as propagators.

The trace ID is added to [access logs](https://caddyserver.com/docs/caddyfile/directives/log) as the standard traceID field.  The trace ID is also set as the `http.vars.trace_id` placeholder value.

[CLAassistant]

All committers have signed the CLA.

[trajano]

Added another commit primarily to offload the work to OTEL's autoexport function.

Hopefully this will eventually allow us to send logs to OTEL as well.

[trajano]

Had to revert that last commit. It didn't seem to work as advertised.

[trajano]

Un reverting the revert. My test was actually flawed and it was working

[trajano]

Note this will just get it to function the same way as before. There's a lot missing in Caddy's OTEL messages. To name a fiew

• there's no Span.Kind pieces for the Client portion so there's no offload shown in the traces from Caddy to the reverse proxy. Add Otel Trace client interactions #6487 will add that
• Status should be ok not just left as unset when it's the final response.
• The foward_auth appears as a separate trace
• The OTEL exports only supports tracing but not others like logs or metrics

[mholt]

Thanks for working on this. We'll revisit this shortly as we prepare the 2.9 beta releases! In the meantime how would you feel about resolving the merge conflicts

[trajano]

Thanks for working on this. We'll revisit this shortly as we prepare the 2.9 beta releases! In the meantime how would you feel about resolving the merge conflicts

maybe this weekend, I only work on it on during my hobby hours