Microsoft Agent Framework (MSAF) is the next-generation successor to both Semantic Kernel and AutoGen, designed to help developers build interactive, robust, and safe AI applications. While most existing frameworks are written in Python, MSAF has the advantage of supporting both C# and Python, making it ideal for .NET developers.
MSAF has a lot of cool features such as workflows, compatibility with most of the vendors (the example I will show uses Anthropic) and native support of AI Foundry.
Why Monitor Agents?
Nowadays, applications increasingly rely on AI-enabled agents for critical processes. When things work well, it's great—but when things go wrong, it's frustrating to troubleshoot due to the non-deterministic nature of LLMs. This is why monitoring is a key capability that every production agent application needs.
Monitoring includes:
- Capturing the requests / responses exchanged with LLMs
- Tracking the costs in terms of input / output token
How to Monitor with MSAF?
MSAF has a powerful feature called middlewares. In MSAF, middleware provides a powerful way to intercept, inspect, and modify AI interactions without altering your core agent or function logic.
MSAF supports three types of middleware:
- Agent middleware for intercepting overall inputs and outputs before or after a run
- Function middleware for validating or transforming tool calls and their results
- Chat middleware for controlling the raw messages and options sent directly to the AI model
Demo
Calling the LLM
In this demo, we'll create a simple application that asks an LLM about countries—straightforward and easy to follow.
Console.Write("Enter a country name: ");
var countryName = Console.ReadLine()?.Trim();
if (string.IsNullOrWhiteSpace(countryName))
{
Console.Error.WriteLine("Country name is required.");
return;
}
var prompt =
$"Provide a concise factual overview for {countryName}. Include: name, capital, population, officialLanguages, currency, and a short summary.";
var options = new AgentRunOptions()
{
ResponseFormat = ChatResponseFormat.ForJsonSchema<Country>()
};
var agentResponse = await agent.RunAsync(prompt, options:options);Writing the middleware
Middlewares are classes created by inheriting DelegatingChatClient. DelegatingChatClient has two virtual methods: GetResponseAsync and GetStreamingResponseAsync that, as you can guess, can be overridden to intercept normal or streaming communications.
The DemoMiddleware below is a simple middleware that writes requests, responses and usage under the logs directory. We'll break it down into logical sections:
Class Setup & Constructor
public sealed class DemoMiddleware : DelegatingChatClient
{
private readonly string _requestsDirectory;
private readonly string _responsesDirectory;
private readonly string _usageDirectory;
private int _requestCounter;
public DemoMiddleware(IChatClient innerClient, string logsRootPath)
: base(innerClient)
{
var runFolder = DateTimeOffset.UtcNow.ToString("yyyyMMdd-HHmmss-fff");
RunDirectory = Path.Combine(logsRootPath, runFolder);
_requestsDirectory = Path.Combine(RunDirectory, "requests");
_responsesDirectory = Path.Combine(RunDirectory, "responses");
_usageDirectory = Path.Combine(RunDirectory, "usage");
Directory.CreateDirectory(_requestsDirectory);
Directory.CreateDirectory(_responsesDirectory);
Directory.CreateDirectory(_usageDirectory);
}
public string RunDirectory { get; }
private static readonly JsonSerializerOptions JsonOptions = new()
{
WriteIndented = true,
ReferenceHandler = ReferenceHandler.IgnoreCycles
};
}The constructor creates a timestamped subdirectory under logs for this run, with separate folders for requests, responses, and usage metrics.
Intercepting Non-Streaming Calls
public override async Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> chatMessages,
ChatOptions? options = null,
CancellationToken cancellationToken = default)
{
var messages = chatMessages.ToList();
var requestNumber = Interlocked.Increment(ref _requestCounter);
await WriteRequestAsync(messages, requestNumber, cancellationToken);
var response = await base.GetResponseAsync(messages, options, cancellationToken);
await WriteResponseAsync(response, requestNumber, cancellationToken);
await WriteUsageAsync(response.Usage?.InputTokenCount, response.Usage?.OutputTokenCount, requestNumber,
cancellationToken);
return response;
}This method intercepts standard (non-streaming) requests. It logs the outgoing prompt, captures the full response object, and records token usage before passing the response back to the caller.
Intercepting Streaming Calls
public override async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> chatMessages,
ChatOptions? options = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var messages = chatMessages.ToList();
var requestNumber = Interlocked.Increment(ref _requestCounter);
await WriteRequestAsync(messages, requestNumber, cancellationToken);
var responseText = new StringBuilder();
await foreach (var update in base.GetStreamingResponseAsync(messages, options, cancellationToken))
{
foreach (var content in update.Contents)
{
if (content is TextContent text)
{
responseText.Append(text.Text);
}
}
yield return update;
}
await WriteResponseAsync(responseText.ToString(), requestNumber, cancellationToken,
isPartialStreamingPayload: true);
await WriteUsageAsync(null, null, requestNumber, cancellationToken,
"Usage is not available in streaming updates.");
}For streaming responses, we aggregate the text chunks as they arrive while still yielding them to the caller, then write the full aggregated response once the stream completes.
Helper Methods
private async Task WriteRequestAsync(List<ChatMessage> messages, int requestNumber,
CancellationToken cancellationToken)
{
var payload = new
{
TimestampUtc = DateTimeOffset.UtcNow,
Messages = messages.Select(m => new
{
Role = m.Role.ToString(),
Contents = m.Contents.Select(FormatContent)
})
};
var path = Path.Combine(_requestsDirectory, $"{requestNumber:D4}-request.json");
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(payload, JsonOptions), cancellationToken);
}
private async Task WriteResponseAsync(ChatResponse response, int requestNumber, CancellationToken cancellationToken)
{
var path = Path.Combine(_responsesDirectory, $"{requestNumber:D4}-response.json");
try
{
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(response, JsonOptions), cancellationToken);
}
catch (NotSupportedException)
{
var fallbackPayload = new
{
TimestampUtc = DateTimeOffset.UtcNow,
Note = "Full response object could not be serialized. Writing known fields instead.",
Text = response.Text,
Usage = response.Usage
};
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(fallbackPayload, JsonOptions),
cancellationToken);
}
}
private async Task WriteUsageAsync(
long? inputTokens,
long? outputTokens,
int requestNumber,
CancellationToken cancellationToken,
string? note = null)
{
var payload = new
{
TimestampUtc = DateTimeOffset.UtcNow,
InputTokens = inputTokens,
OutputTokens = outputTokens,
Note = note
};
var path = Path.Combine(_usageDirectory, $"{requestNumber:D4}-usage.json");
await File.WriteAllTextAsync(path, JsonSerializer.Serialize(payload, JsonOptions), cancellationToken);
}
private static object FormatContent(AIContent content)
{
return content switch
{
TextContent text => new { Type = "text", Text = text.Text },
TextReasoningContent reasoning => new { Type = "reasoning", Text = reasoning.Text },
DataContent => new { Type = "data" },
_ => new { Type = content.GetType().Name, Text = content.ToString() }
};
}The helper methods handle the actual file I/O and serialization. WriteResponseAsync has special handling for objects that may not serialize cleanly—it falls back to writing just the essential fields if needed.
Wire things up
Now that we have our DemoMiddleware class, we need to integrate it into the chat client pipeline when we create it. Below is an example of wiring it into an Anthropic client:
using var anthropicClient = new AnthropicClient
{
ApiKey = apiKey,
HttpClient = new HttpClient { Timeout = TimeSpan.FromSeconds(60) }
};
using var chatClient = anthropicClient.AsIChatClient();
var logsRoot = Path.Combine(Directory.GetCurrentDirectory(), "logs");
using var monitoredChatClient = chatClient.AsBuilder()
.Use(innerClient => new DemoMiddleware(innerClient, logsRoot))
.Build();The key line is .Use(innerClient => new DemoMiddleware(innerClient, logsRoot)), which wraps the chat client in our middleware. Now whenever the agent makes requests or receives responses, they'll be logged automatically.
Results
Below are examples of the requests, responses and usage that our middleware captures:
Request
{
"TimestampUtc": "2026-06-20T10:46:04.222982+00:00",
"Messages": [
{
"Role": "user",
"Contents": [
{
"Type": "text",
"Text": "Provide a concise factual overview for Spain. Include: name, capital, population, officialLanguages, currency, and a short summary."
}
]
}
]
}Response
The response JSON below contains the complete ChatResponse object returned by the LLM. Key fields to note:
Messages[0].Contents[0].Text— Contains the actual response from the model. In this case, it's the Country object serialized as a JSON string (escaped as\u0022for quotes).Usage.InputTokenCount— Tokens consumed by the prompt (314 in this example).Usage.OutputTokenCount— Tokens generated in the response (97 in this example).$type— Indicates the content type (here,text). This helps identify response variants (text vs. reasoning, etc.).FinishReason— Why the model stopped generating (stop= reached natural completion).
{
"Messages": [
{
"AuthorName": null,
"CreatedAt": "2026-06-20T10:46:06.352067+00:00",
"Role": "assistant",
"Contents": [
{
"$type": "text",
"Text": "{\u0022capital\u0022: \u0022Madrid\u0022, \u0022currency\u0022: \u0022Euro (EUR)\u0022, \u0022officialLanguages\u0022: [\u0022Spanish\u0022], \u0022name\u0022: \u0022Spain\u0022, \u0022population\u0022: 47615000, \u0022summary\u0022: \u0022Spain is a country located in southwestern Europe on the Iberian Peninsula. It is known for its rich cultural heritage, diverse landscapes, Mediterranean climate, and significant contributions to art, architecture, and history. Spain is a member of the European Union and NATO.\u0022}",
"Annotations": null,
"AdditionalProperties": null
}
],
"MessageId": "msg_01B1LUYaA8n9XqsCTHddbabp",
"AdditionalProperties": null
}
],
"ResponseId": "msg_01B1LUYaA8n9XqsCTHddbabp",
"ConversationId": null,
"ModelId": "claude-haiku-4-5-20251001",
"CreatedAt": "2026-06-20T10:46:06.352067+00:00",
"FinishReason": "stop",
"Usage": {
"InputTokenCount": 314,
"OutputTokenCount": 97,
"TotalTokenCount": 411,
"CachedInputTokenCount": 0,
"ReasoningTokenCount": null,
"inputAudioTokenCount": null,
"inputTextTokenCount": null,
"outputAudioTokenCount": null,
"outputTextTokenCount": null,
"AdditionalCounts": null
},
"continuationToken": null,
"AdditionalProperties": null
}Usage Metrics
{
"TimestampUtc": "2026-06-20T10:46:06.388226+00:00",
"InputTokens": 314,
"OutputTokens": 97,
"Note": null
}Setup & Running the Demo
Before running the application:
1. Set your Anthropic API key:
export ANTHROPIC_API_KEY=your_key_here2. Create the Country class (or place it in the same project):
public class Country
{
public required string Name { get; set; }
public required string Capital { get; set; }
public int Population { get; set; }
public string[] OfficialLanguages { get; set; } = [];
public required string Currency { get; set; }
public required string Summary { get; set; }
}3. Run the application:
dotnet run --project MSAFMonitoringDemo/MSAFMonitoringDemo4. Logs will be written to:
./logs/yyyyMMdd-HHmmss-fff/
├── requests/
├── responses/
└── usage/Wrap-Up
We created a lightweight middleware that automatically captures requests, responses, and token usage for every LLM interaction. This foundation can be extended to write logs to cloud storage, integrate with monitoring platforms, or toggle logging via configuration.
The full source code of this demo can be found here.