[v2] ddtrace/tracer: Report datadog.tracer.queue.enqueued.traces as health metric

[mtoffl01]

What does this PR do?

Reports the number of traces in the queue at a specified interval.
NOTE: This implementation does not currently differentiate between true "traces" and partial trace "chunks"

Motivation

Report more data from our health metrics

Reviewer's Checklist

• Changed code has unit tests for its functionality at or near 100% coverage.
• System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
• There is a benchmark for any new code, or changes to existing code.
• If this interacts with the agent in a new way, a system test has been added.
• Add an appropriate team label so this PR gets put in the right place for the release notes.
• Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
• For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

[hannahkm]

Could we rebase this to get rid of the other commits, and also make sure to be merging into v2-dev instead of main!

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-12-20 16:37:39

Comparing candidate commit 76243a0 in PR branch mtoff/v2-queue.enqueued.traces with baseline commit 822d81b in branch main.

Some scenarios are present only in baseline or only in candidate runs. If you didn't create or remove some scenarios in your branch, this maybe a sign of crashed benchmarks 💥💥💥
Check Gitlab CI job log to find if any benchmark has crashed.

Scenarios present only in baseline:

• BenchmarkSampleWAFContext-24
• BenchmarkStartRequestSpan-24

Found 3 performance improvements and 12 performance regressions! Performance is the same for 37 metrics, 1 unstable metrics.

scenario:BenchmarkHttpServeTrace-24

• 🟥 allocated_mem [+231 bytes; +283 bytes] or [+2.433%; +2.974%]
• 🟥 allocations [+5; +5] or [+6.329%; +6.329%]
• 🟥 execution_time [+1.098µs; +1.214µs] or [+6.944%; +7.675%]

scenario:BenchmarkPartialFlushing/Disabled-24

• 🟥 execution_time [+10.233ms; +12.589ms] or [+3.798%; +4.672%]

scenario:BenchmarkPartialFlushing/Enabled-24

• 🟥 execution_time [+12.075ms; +14.647ms] or [+4.402%; +5.339%]

scenario:BenchmarkSetTagMetric-24

• 🟩 execution_time [-7.490ns; -5.450ns] or [-5.922%; -4.310%]

scenario:BenchmarkSetTagString-24

• 🟩 execution_time [-11.883ns; -8.337ns] or [-9.465%; -6.640%]

scenario:BenchmarkSetTagStringer-24

• 🟩 execution_time [-9.093ns; -3.527ns] or [-6.003%; -2.328%]

scenario:BenchmarkSingleSpanRetention/no-rules-24

• 🟥 execution_time [+14.296µs; +15.025µs] or [+6.047%; +6.355%]

scenario:BenchmarkSingleSpanRetention/with-rules/match-all-24

• 🟥 execution_time [+12.031µs; +13.096µs] or [+5.051%; +5.498%]

scenario:BenchmarkSingleSpanRetention/with-rules/match-half-24

• 🟥 execution_time [+10.855µs; +12.004µs] or [+4.534%; +5.014%]

scenario:BenchmarkStartSpan-24

• 🟥 execution_time [+66.446ns; +81.954ns] or [+2.998%; +3.698%]

scenario:BenchmarkStartSpanConcurrent-24

• 🟥 execution_time [+186.166ns; +296.234ns] or [+3.479%; +5.536%]

scenario:BenchmarkTracerAddSpans-24

• 🟥 allocated_mem [+56 bytes; +57 bytes] or [+2.456%; +2.484%]
• 🟥 execution_time [+80.209ns; +129.791ns] or [+2.031%; +3.286%]