Effortless Tracing for .NET Applications Using Open Telemetry

As applications evolve, there’s a noticeable shift from traditional monolithic designs to contemporary microservices architectures, bringing numerous operational advantages. While this transition enhances scalability and flexibility, it also presents challenges in terms of monitoring and logging, especially given the distributed nature of modern application designs. Conventional methods struggle to provide comprehensive end-to-end visibility, creating gaps in the monitoring, troubleshooting, and analysis processes required by both developers and administrators. In addressing these challenges, distributed tracing emerges as a pivotal solution, offering a robust approach to track the flow of requests and responses across diverse services and components. This method not only enhances overall visibility but also provides valuable insights into performance and error patterns within the application ecosystem.

What is Open Telemetry Tracing?

Open Telemetry represents an open-source observability framework within the Cloud Native Computing Foundation (CNCF) incubating project. Comprising a suite of APIs, libraries, agents, and SDKs, Open Telemetry empowers development teams to seamlessly instrument, capture, and export telemetry data from contemporary applications.

This versatile framework provides developers with a standardized instrumentation library, generating telemetry data encompassing logs, metrics, and traces from diverse sources. Leveraging Open Telemetry agents, this telemetry data can be efficiently collected and exported to various systems designed for logging, tracing, and monitoring. A key tenet of Open Telemetry is its vendor-agnostic approach, allowing for the transfer of collected data to different backends without necessitating client-side modifications.

The OTel tracing framework delivers a range of advantages, particularly beneficial in the context of distributed microservice-oriented architectures. The following key features underscore these advantages:

1. Standardization: OpenTelemetry establishes a common instrumentation and data format standard, enabling developers to produce and export telemetry data in a manner that is independent of vendors. This fosters consistency and interoperability across diverse systems, simplifying the analysis and troubleshooting of issues. By aggregating telemetry data from logs, metrics, and traces, it ensures comprehensive observability of distributed systems.
2. Flexibility: OpenTelemetry is versatile, supporting multiple programming languages, frameworks, and cloud environments. This flexibility makes it a valuable tool adaptable to various scenarios and ecosystems.
3. Interoperability: OpenTelemetry seamlessly integrates with other observability tools, including tracing systems, logging platforms, and monitoring tools. This seamless integration facilitates the adoption and extension of existing observability solutions, enhancing the overall efficacy of the observability ecosystem.

How Open Telemetry Works

Open Telemetry relies on its numerous components to instrument and collects logs, metrics, and traces from distributed applications.

Over view of the Open Telemetry Tracing Process

While OpenTelemetry offers a robust and comprehensive approach to tracing, the process can be distilled into the following streamlined steps:

1. Instrumentation: Developers employ the OpenTelemetry SDK to instrument their applications, adding code to initiate and conclude spans, set attributes, and introduce events to spans.
2. Span Creation: Upon receiving a request, the application generates a new span to represent that specific request. Spans have designated start and end times, encompassing attributes, events, and child spans.
3. Context Propagation: OpenTelemetry propagates trace context seamlessly across service boundaries using headers. This facilitates the continuation of traces across multiple services and systems.
4. Span Export: The exporter gathers and dispatches telemetry data to a backend system, such as Jaeger or Zipkin, for storage and analysis, ensuring backward compatibility.

Open Telemetry Integration

The flexibility of the OpenTelemetry framework supports a diverse array of integration options at various levels:

• Integration with Different Programming Languages and Platforms: OpenTelemetry provides SDKs for multiple programming languages, including Java, Python, Go, Node.js, .NET, Kubernetes, and more. These SDKs empower developers to instrument their applications uniformly for telemetry data collection.
• Integration with Different Frameworks: OpenTelemetry seamlessly integrates with popular application frameworks like Spring, Flask, and Django. These integrations automatically instrument these frameworks, enabling developers to collect telemetry data without additional coding.
• Integration with Different Services: OpenTelemetry offers exporters for various backend systems, such as Jaeger, Zipkin, Prometheus, and more. Developers can leverage these exporters to transmit telemetry data to backend systems for storage and analysis. OpenTelemetry also supports integration with major cloud providers like AWS, Azure, and Google Cloud Platform, facilitating the collection of telemetry data from services in these environments.

Open Telemetry Instrumentation

Instrumentation, in the context of Open Telemetry, entails adding code to an application to gather telemetry data related to performance, behavior, and other metrics. The goal of instrumentation is to enhance visibility into an application’s behavior and performance, empowering developers to identify and troubleshoot issues effectively.

Auto Instrumentation of Tracing in .NET

Auto-instrumentation in the context of tracing typically refers to the automatic instrumentation of code for capturing traces without requiring developers to manually add trace statements.

Here are the general steps to achieve auto-instrumentation for tracing in a .NET application using Open Telemetry:

Add Open Telemetry Packages:

dotnet add package OpenTelemetry
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.Console
dotnet add package OpenTelemetry.Extensions.Hosting

1.Trace the incoming HTTP Requests

In our Program class, let’s configure OpenTelemetry for our application.

builder.Services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddConsoleExporter());

Here, we start by calling the AddOpenTelemetry() method, which configures the services required for instrumenting our .NET application. Then, we configure tracing with the AddAspNetCoreInstrumentation() method. This records tracing information for all incoming HTTP requests. Next, we’ll start by exporting to console, with the AddConsoleExporter() method.

Let’s run our application and make a request to /WeatherForeceast. If we take a look at the console, we see a trace for our request:

Activity.TraceId:            56349c704e56e3ef878c833bddf66476
Activity.SpanId:             1efc1ddb7688530d
Activity.TraceFlags:         Recorded
Activity.ActivitySourceName: Microsoft.AspNetCore
Activity.DisplayName:        GET WeatherForecast
Activity.Kind:               Server
Activity.StartTime:          2024-01-24T19:06:52.6879554Z
Activity.Duration:           00:00:00.0448304
Activity.Tags:
    server.address: localhost
    server.port: 7240
    http.request.method: GET
    url.scheme: https
    url.path: /WeatherForecast
    network.protocol.version: 2
    user_agent.original: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
    http.route: WeatherForecast
    http.response.status_code: 200
Resource associated with Activity:
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.7.0
    service.name: unknown_service:TracingWithOpenTelemetry.Web Api

There’s a lot of information here. This is the benefit of having auto-instrumentation for our applications. We see ids, such as Activity.TraceId and Activity.SpanId. The TraceId property is used to correlate spans so we can view all the events that occurred during a single request. In this case, we only have one, but with multiple downstream events, they would all use the same TraceId.

In the Activity.Tags property, we get some more useful information such http.url, which tells us the URL that was requested, as well as http.status_code, telling us the returned HTTP status code.

2.Trace the outgoing HTTP Requests

In .NET, we use the HttpClient class to make outgoing HTTP requests, so we can configure this to emit tracing events.

To achieve this, let’s add the OpenTelemetry.Instrumentation.Http NuGet package and configure it in our Program class:

builder.Services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddConsoleExporter());

Here, we simply call the AddHttpClientInstrumentation() method to set up traces for outgoing HttpClient requests.

Next, let’s create a new endpoint on our existing WeatherForecast controller to make an outgoing HTTP request:

[HttpGet("OutgoingHttp")]
public async Task OutgoingHttpRequest()
{
    var client = new HttpClient();

    var response = await client.GetAsync("https://athen.tech/");
    response.EnsureSuccessStatusCode();
}

In our new endpoint, we simply instantiate a new HttpClient and make a GET request to create a trace.

Now, let’s run our application and make a request to /WeatherForecast/OutgoingHttp. Checking our console, we see a new trace emitted:

Activity.TraceId:            ee3fae93fe853f0fbe91fbe6c674430e
Activity.SpanId:             6b173ab3e8803b24
Activity.TraceFlags:         Recorded
Activity.ActivitySourceName: Microsoft.AspNetCore
Activity.DisplayName:        GET WeatherForecast/OutgoingHttp
Activity.Kind:               Client
Activity.StartTime:          2024-01-25T14:03:54.1610500Z
Activity.Duration:           00:00:01.7198435
Activity.Tags:
    server.address: localhost
    server.port: 7240
    http.request.method: GET
    url.scheme: https
    url.path: /WeatherForecast/OutgoingHttp
    network.protocol.version: 2
    user_agent.original: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
    http.route: WeatherForecast/OutgoingHttp
    http.response.status_code: 200
Resource associated with Activity:
    service.name: Tracing.Net
    service.instance.id: 1eda6029-908a-49b8-8a53-b7cc2da972d4
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.7.0

This looks very similar to our previous, incoming HTTP trace. But there are a couple of slight differences. First, our ActivitySourceName tag value is different, as this is generated by the HttpClient class. Secondly, Activity.Kind has a value of Client this time.

Additional Links

• https://uptrace.dev/opentelemetry/distributed-tracing.html#span-names
• https://www.servicenow.com/products/observability/what-is-opentelemetry.html