Durable Functions’ Distributed Tracing
I’m happy to announce the Distributed Tracing feature for Durable Functions. If you’ve never heard of Durable Functions, then I recommend you check out our patterns and concepts documentation that explains what it is and why you should care. I’m a contributor to Durable Functions, so I work with them and implement the Distributed Tracing feature.
Why Distributed Tracing?
The Azure Functions team received a lot of requests for the Distributed Tracing for Durable Functions. The host of the Azure Functions will track telemetry automatically with Application Insights, so people easy to diagnose the function execution.
The durable functions enable us to execute orchestrations. It allows us to run a complex workflow, long-running execution, and stateful services.
However, the orchestration is consist of several functions executions. For each function, instance Id is different and no correlation information. It was hard to identify which telemetries are belonged to the same orchestrations.
You can find the overview and demo with this video.
What is the Distributed Tracing for Durable Functions?
The feature correlates the telemetries that related the same orchestration. You can see the telemetry on the end to end transaction with Application Insights. It supports W3C TraceContext and HttpCorrelation Protocol.
It also supports the incoming telemetry correlation and custom telemetry for the customer. For more details of the external correlation, you can refer to this video.
Since this is the alpha release for getting feedback on this feature, All the programming models are not supported. You can refer to which programming models are already supported.
How does it work?
The Activity class managed the correlation using Application Insights. However, It doesn’t allow re-assign the Id and serialize. Under the hood, the durable functions use queues to talk with the other orchestrators and activity functions. For the correlation, we need to pass through the telemetry data through the orchestration. We implement the wrapper of the Activity class that enables re-assign and serialization. Also, we track the telemetry at the side of durable functions for orchestrators.
If you want to know the architecture, you can see the details on the DurableTask Distributed Tracing page, Durable Functions working on the Durable Task. You can refer to the detailed architecture from this page.
How to use
I’ll show you the overview. If you need a step-by-step instruction, you can refer Getting Started.
Step1. Create a Application Insights on your Azure Portal and get the instrumentation key.
Step2. Create Durable Functions or clone the samples. The sample is reside on the correlation branch on the Durable Functions repository.
Step3. Make sure to add configuration for NuGet source for `https://www.myget.org/F/azure-appservice-staging/api/v3/index.json`. The NuGet package is reside in there.
Step4. Add Microsoft.Azure.WebJobs.Extensions.DurableTask.Telemetry version 2.2.0-alpha on your csproj file. This csproj file is the samples.
Step5. Configure host.json and local.settings.json for setting up for Distributed Tracing.
Step6. Run your durable functions.
Step7. Go to the Application Insights on your portal and see the End To End Tracing with choosing a telemetry that has Dt prefix. If you can’t find the telemetry, Go to Getting Started page. You can find more instructions.
You will find more resources:
- Distributed Tracing for Durable Functions (Overview)
- Getting Started
- Reference
- Configuration
- Distributed Tracing Durable Functions samples
Limitation
Currently, Orchestrator and Activity Functions are tracked on the Durable Functions side. It is missing the custom property that is tracked on the Azure Functions Host. In the near feature, Only the Orchestrator telemetry is tracked on the Durable Functions side with keeping each Orchestration telemetries are tracked on Azure Functions Host.
As the first alpha version, we support C# only. However, we are going to support other languages as well.
Also, Not all programming model is supported; we are implementing it shortly before GA.
The alpha release of distributed tracing is primarily intended to gather feedback. We plan on shipping it as a GA feature shortly, so now is the time to provide feedback that will help influence the priorities and the direction it takes. We hope you’ll give it a try and let us know what you’d like to see in future updates.
Feedback
If you have feedback and requests, please submit issues on this repo. I hope you like this new feature.