OpenTelemetry InMemoryExporter not working deterministically in unit testing

[Yogesh19921]

Hi folks,

So I have a piece of code that uses Meter to write some metrics (observablegauege). For obvious reasons, I can't paste the exact code here, but I can provide similar code.

// This is the method I am testing.
public static async Task EmitAMetric(Meter meter, string eventName, Func<Task> func)
{
    try
    {
        await func().ConfigureAwait(false);

        meter.EmitMetric(eventName, 1);
    }
    catch (Exception ex)
    {
         // Error logic
    }
}

public static ObservableGauge<T> EmitMetric<TValue>(this Meter meter, string metricName, T value)
    where TValue : struct
{
    return meter.CreateObservableGauge<TValue>(metricName, () => value);
}

I am writing multiple tests to test this method but all of them follow the same pattern. I am writing a sample test here.

        [Test]
        public async Task EmitAMetric_OnSuccessfulAsyncMethod_EmitsMetric()
        {
            string meterName = this.GenerateRandomString(10);
            string eventName = this.GenerateRandomString(10);

            using Meter meter = new Meter(meterName);
            List<Metric> metrics = new List<Metric>();

            await EmitAMetric(meter, eventName, () => Task.Delay(0)).ConfigureAwait(false);

            using MeterProvider meterProvider = Sdk.CreateMeterProviderBuilder()
                .AddMeter(meter.Name)
                .AddInMemoryExporter(metrics)
                .Build();

            bool flushed = meterProvider.ForceFlush(10);

            // The below is where failure occurs, if it does
            Assert.IsTrue(metrics.Any(metric => metric.Name.Equals(eventName)));
            metrics.Clear();
        }

The tests are all non-deterministic and fail sometimes.
I have tried to follow the same pattern from here.

What I have already tried ?

• Adding delay of upto 1 sec before calling meterProvider.ForceFlush(10).
• Using meterProvider.ForceFlush() instead.
• Using meterProvider.ShutDown() at the end of each test.
• Creating a single instance of meter and meterprovider for the entire tests class and implementing IDisposable to dispose meterProvier and meter.
• Not cleaning metrics list.

Has anyone had success using InMemoryExporter for unit testing ?

Any help or guide would be much appreciated.

[utpilla]

This is not the correct usage of ObservableGauge. You only have to create it once. The SDK would call the callback you provide for ObservableGauge during collection. If you simply want that callback associated with the ObservableGauge to be called multiple times, then you can just call ForceFlush for the required number of times.

Here is an example that shows how you could use an ObservableGauge: https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/metrics/learning-more-instruments/Program.cs#L35

Here is a test you could refer to: https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/test/OpenTelemetry.Tests/Metrics/MetricApiTestsBase.cs#L99-L130