OpenTelemetry request tracing doesn't properly handle sampling

When tracing, a very common trace configuration is to pair two concepts for sampling:

• If there is a parent span, follow and use it's sampling decision
• If there is no parent span, make a ratio-based sampling decision

An example result of this approach is typically that 5% of all incoming HTTP requests are traced from beginning to end. In OpenTelemetry, this could be setup as follows:

builder.Services.AddOpenTelemetry()

    .WithTracing(b =>

    {

          b

              .AddHttpClientInstrumentation()

              .AddAspNetCoreInstrumentation()

              .AddCouchbaseInstrumentation()

              .SetSampler(new ParentBasedSampler(new TraceIdRatioBasedSampler(0.05)));

    });

Currently, if the parent span (i.e. the incoming HTTP request handler) is not sampled then this is not respected by the Couchbase "request_encoding", "dispatch_to_server", and similar spans (i.e. compression).

This is because the parent span link is lost when the request is queued on the connection pool when the parent span is not sampled. In this case, when the inner span is created a new sampling decision is made based on the ratio, and the parent span's decision to not be sampled is ignored.

The end result, for the example above, is that 5% of all HTTP requests are fully traced with all Couchbase internal operations traced. However, of the other 95% of operations, 5% of the Couchbase internal operations are randomly sampled as spans without a linked parent.