AI agent architecture patterns describe how you combine language models, tools, memory and control flow into systems that complete multi-step tasks reliably. This guide is for .NET architects and senior developers who have built a chatbot or a tool-calling prototype and now need to decide how much autonomy to grant, when to split work across several agents, and how to keep the result safe, observable and affordable. You will learn the main orchestration patterns (sequential, concurrent, handoff, group chat and Magentic), planning and memory strategies, human-in-the-loop approvals, guardrails, durable execution and the A2A protocol, with C# sketches based on Microsoft Agent Framework.

What Is an AI Agent Architecture?#

An agent is a loop around a model: it receives a goal, decides on the next action, calls a tool or produces an answer, observes the result and repeats until done. Four ingredients define it: instructions (role and constraints), tools (functions, APIs, other agents), memory (what it remembers within and across conversations) and a control policy (who decides the next step, and when to stop).

Architecture is about where you put control. At one end sit deterministic workflows with LLM steps, where your code decides the sequence and the model fills in content. At the other end sit autonomous agents that plan their own steps. Most production systems land in between: a fixed outer workflow with bounded agentic loops inside. The rule of thumb is simple: use the least autonomy that solves the problem, because every decision you hand to a model becomes a source of variance, latency and cost.

How Agents Work in Microsoft Agent Framework#

Microsoft Agent Framework brings together ideas from Semantic Kernel and AutoGen, and its repository links migration guides from both. The .NET packages reached 1.0 in April 2026, and Microsoft.Agents.AI was at 1.22 as of September 2026. It builds on Microsoft.Extensions.AI, so any IChatClient can become an agent. For a broader introduction, see Microsoft Agent Framework: Build AI Agents in C#. The key types are:

  • AIAgent, the abstraction every agent implements, with RunAsync and RunStreamingAsync. ChatClientAgent is the implementation that wraps an IChatClient.
  • AgentSession, which carries conversation state across turns and can be serialized with SerializeSessionAsync for storage.
  • AIContextProvider, a hook that injects instructions, messages or tools before each run and observes results afterwards. This is the extension point for memory.
  • Agent middleware, added with AsBuilder().Use(...), which wraps agent runs and function calls, for example to add guardrails.
  • Workflows in Microsoft.Agents.AI.Workflows, which compose agents and plain executors into graphs, including ready-made multi-agent orchestrations.

Getting Started: A Single Agent with Tools#

Most problems need only one agent with well-designed tools. The agent below answers order questions; the session keeps the conversation so the second question can refer to the first.

C#
using System.ComponentModel;
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
    ?? throw new InvalidOperationException("Set AZURE_OPENAI_ENDPOINT.");
var deployment = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT") ?? "gpt-5-mini";

IChatClient chatClient = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
    .GetChatClient(deployment)
    .AsIChatClient();

AIAgent support = chatClient.AsAIAgent(
    name: "SupportAgent",
    instructions: "You answer order questions. Use tools for facts; never guess order data.",
    tools: [AIFunctionFactory.Create(OrderTools.GetOrderStatusAsync)]);

AgentSession session = await support.CreateSessionAsync();
Console.WriteLine(await support.RunAsync("Where is order 10042?", session));
Console.WriteLine(await support.RunAsync("Will it arrive before Friday?", session));

public static class OrderTools
{
    [Description("Gets the shipping status and estimated delivery date for an order.")]
    public static Task<string> GetOrderStatusAsync(
        [Description("The order number.")] string orderId,
        CancellationToken cancellationToken) =>
        Task.FromResult($"Order {orderId}: shipped, estimated delivery Thursday.");
}

Tool design matters more than agent count. Give tools narrow purposes, precise descriptions and typed parameters, and return compact, factual results. For the mechanics of tool calling, see Function Calling and Tool Use with LLMs in C#.

Single Agent vs Multi-Agent Systems#

Split into multiple agents when a single agent's context becomes overloaded: too many tools (large tool lists make selection harder and consume context on every call), conflicting instructions, or distinct security boundaries such as one agent that reads customer data and another that may only draft text. Multi-agent designs also let you use different models per role, for example a small model for classification and a larger one for synthesis.

The cost is real. Every agent re-reads context, every hop adds latency, and failures now happen between agents as well as inside them. Before building a multi-agent system, try the lightest form of delegation: expose specialist agents as tools of a coordinating agent.

C#
AIAgent billing = chatClient.AsAIAgent(
    name: "BillingSpecialist",
    description: "Answers invoice, refund and payment questions.",
    instructions: "You are a billing specialist. Cite invoice numbers when you use them.");

AIAgent shipping = chatClient.AsAIAgent(
    name: "ShippingSpecialist",
    description: "Answers delivery, tracking and returns logistics questions.",
    instructions: "You are a shipping specialist.",
    tools: [AIFunctionFactory.Create(OrderTools.GetOrderStatusAsync)]);

AIAgent coordinator = chatClient.AsAIAgent(
    name: "Coordinator",
    instructions: "Delegate each part of the question to the right specialist, then answer once.",
    tools: [billing.AsAIFunction(), shipping.AsAIFunction()]);

The coordinator stays in control, and each specialist has its own instructions and tool set. This "agents as tools" pattern covers a surprising share of multi-agent needs without a workflow engine.

Orchestration Patterns in Microsoft Agent Framework#

When delegation through tools is not enough, the framework's workflow layer provides explicit orchestrations. All of them produce a Workflow that you run the same way, so a small helper keeps the samples short:

C#
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;

public static class WorkflowRunner
{
    public static async Task<List<ChatMessage>> RunAsync(
        Workflow workflow, List<ChatMessage> input, CancellationToken cancellationToken)
    {
        await using StreamingRun run = await InProcessExecution.RunStreamingAsync(
            workflow, input, cancellationToken: cancellationToken);
        await run.TrySendMessageAsync(new TurnToken(emitEvents: true));

        await foreach (WorkflowEvent evt in run.WatchStreamAsync(cancellationToken))
        {
            switch (evt)
            {
                case AgentResponseUpdateEvent update:
                    Console.Write(update.Update.Text);
                    break;
                case WorkflowOutputEvent output:
                    return output.As<List<ChatMessage>>() ?? [];
            }
        }

        return [];
    }
}

Sequential and concurrent orchestration#

A sequential orchestration is a pipeline: each agent receives the conversation so far and adds its contribution, which suits draft-review-polish flows. A concurrent orchestration fans the same input out to several agents and merges their answers, which suits independent reviews, such as security, performance and API design, or getting diverse proposals quickly.

C#
AIAgent drafter = chatClient.AsAIAgent(name: "Drafter",
    instructions: "Draft a customer reply from the case notes.");
AIAgent reviewer = chatClient.AsAIAgent(name: "PolicyReviewer",
    instructions: "Fix anything that conflicts with the refund policy. Output the final reply.");

Workflow pipeline = AgentWorkflowBuilder.BuildSequential([drafter, reviewer]);

Workflow reviews = AgentWorkflowBuilder.BuildConcurrent([securityReviewer, performanceReviewer]);

List<ChatMessage> reply = await WorkflowRunner.RunAsync(
    pipeline, [new ChatMessage(ChatRole.User, caseNotes)], cancellationToken);

Sequential latency is the sum of its steps; concurrent latency is the slowest branch, but token cost is the sum of all branches. BuildConcurrent also accepts an aggregator function if you want to merge results yourself instead of receiving all messages.

Handoff orchestration#

In a handoff, control moves between agents as the conversation evolves, like a call center transferring a caller. Each agent gets handoff tools for the agents it may transfer to, and only the active agent responds. It suits customer-facing flows where the right specialist is not known upfront.

C#
AIAgent triage = chatClient.AsAIAgent(name: "Triage",
    instructions: "Work out what the customer needs and hand off. Never answer yourself.");
AIAgent refunds = chatClient.AsAIAgent(name: "Refunds",
    description: "Handles refund eligibility and refund requests.",
    instructions: "Handle refunds only. Hand back to triage for anything else.");
AIAgent techSupport = chatClient.AsAIAgent(name: "TechSupport",
    description: "Troubleshoots product and account problems.",
    instructions: "Troubleshoot technical issues only. Hand back to triage for anything else.");

Workflow helpDesk = AgentWorkflowBuilder.CreateHandoffBuilderWith(triage)
    .WithHandoffs(triage, [refunds, techSupport])
    .WithHandoffs([refunds, techSupport], triage)
    .Build();

Define the allowed routes explicitly. A fully connected graph invites "ping-pong" handoffs, where two agents bounce a conversation back and forth.

Group chat orchestration#

A group chat puts several agents in one shared conversation, with a manager deciding who speaks next. The built-in round-robin manager takes turns; custom managers can choose speakers with an LLM or rules. Writer-critic loops and design debates are typical uses.

C#
Workflow critique = AgentWorkflowBuilder
    .CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents)
    {
        MaximumIterationCount = 4, // hard stop: the loop must end even if nobody is satisfied
    })
    .AddParticipants([writer, editor])
    .Build();

Magentic orchestration#

The Magentic pattern, based on the Magentic-One design, adds a manager agent that plans. The manager keeps a task ledger (facts and plan) and a progress ledger (whether the request is satisfied, whether the team is looping or making progress, and who should act next). It re-plans when progress stalls. Use it for open-ended research or analysis tasks where the steps cannot be known in advance.

C#
Workflow investigation = new MagenticWorkflowBuilder(manager)
    .AddParticipants([researcher, analyst])
    .RequirePlanSignoff(true) // a human approves the plan before work starts
    .WithMaxRounds(12)
    .WithMaxStalls(3)
    .WithMaxResets(1)
    .Build();

The limits are not optional decoration: rounds, stalls and resets bound both cost and the damage a confused manager can do. Stream the plan and progress events to your logs, because they explain why the team did what it did.

Planning Strategies#

Planning determines how an agent decides its next step. Three strategies cover most systems:

  • Reactive tool loops. The model interleaves reasoning, tool calls and observations until it answers. This is what ChatClientAgent does with function calling. It is simple and adaptive, but opaque for long tasks.
  • Plan-then-execute. A planner produces an explicit plan as structured output, your code validates it (allowed tools, step count, budget), then executes steps with checkpoints. Plans become reviewable artifacts, which makes this the best fit for regulated domains. Structured outputs make such plans machine-checkable.
  • Manager-led re-planning. Magentic-style managers maintain ledgers and re-plan when progress stalls. This is the most flexible strategy and also the hardest to predict.

Reasoning models make reactive loops stronger, but they do not remove the need for explicit limits on iterations, tools and time.

Memory: Short-Term, Long-Term and Vector Memory#

Agent memory comes in three layers, and conflating them is a common design error:

  • Short-term (session) memory is the conversation history in an AgentSession. It grows with every turn, so production systems persist it (serialize the session, or plug in a chat history provider) and reduce it (message-count limits or summarization) before it overflows the context window.
  • Long-term memory holds durable facts and preferences across sessions, such as "prefers aisle seats". Store it outside the conversation, with explicit user controls to view and delete it.
  • Vector (semantic) memory retrieves relevant past interactions or documents by similarity, using the same embeddings and vector search machinery as RAG.

In Agent Framework, long-term and vector memory plug in through AIContextProvider. The sketch below recalls related notes before each run and stores the exchange afterwards; IMemoryIndex stands for your own vector-store wrapper:

C#
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

public sealed class RecallMemory(IMemoryIndex index) : AIContextProvider
{
    protected override async ValueTask<AIContext> ProvideAIContextAsync(
        InvokingContext context, CancellationToken cancellationToken = default)
    {
        var query = context.AIContext.Messages?.LastOrDefault(m => m.Role == ChatRole.User)?.Text;
        if (string.IsNullOrWhiteSpace(query))
        {
            return new AIContext();
        }

        IReadOnlyList<string> notes = await index.SearchAsync(query, top: 3, cancellationToken);
        return new AIContext
        {
            Instructions = notes.Count == 0
                ? null
                : "Relevant notes from earlier conversations:\n- " + string.Join("\n- ", notes),
        };
    }

    protected override async ValueTask StoreAIContextAsync(
        InvokedContext context, CancellationToken cancellationToken = default)
    {
        if (context.InvokeException is null && context.ResponseMessages is { } responses)
        {
            // In production, extract or summarize facts instead of storing raw text.
            var exchange = string.Join("\n", context.RequestMessages.Concat(responses)
                .Select(m => $"{m.Role}: {m.Text}"));
            await index.AddAsync(exchange, cancellationToken);
        }
    }
}

// Registration:
// AIAgent concierge = chatClient.AsAIAgent(new ChatClientAgentOptions
// {
//     Name = "Concierge",
//     ChatOptions = new ChatOptions { Instructions = "You are a travel concierge." },
//     AIContextProviders = [new RecallMemory(memoryIndex)],
// });

Treat memory as untrusted input. Anything written into memory from a conversation can carry an injected instruction into future sessions, so scope memory per user and sanitize what you store.

Human-in-the-Loop Approvals#

Some actions should never run on the model's say-so alone: refunds above a threshold, emails to customers, production changes. Wrap such tools in ApprovalRequiredAIFunction. Instead of executing the call, the agent returns a ToolApprovalRequestContent, and your application collects a decision and sends it back in the same session.

C#
AIAgent refundsAgent = chatClient.AsAIAgent(
    name: "Refunds",
    instructions: "Issue refunds only when the refund policy allows it.",
    tools: [new ApprovalRequiredAIFunction(
        AIFunctionFactory.Create(RefundTools.IssueRefundAsync))]);

AgentSession session = await refundsAgent.CreateSessionAsync(cancellationToken);
AgentResponse response = await refundsAgent.RunAsync(
    "Refund order 10042; it arrived broken.", session, cancellationToken: cancellationToken);

List<ToolApprovalRequestContent> pending = PendingApprovals(response);
while (pending.Count > 0)
{
    List<AIContent> decisions = [];
    foreach (var request in pending)
    {
        bool approved = await approvals.AskSupervisorAsync(request.ToolCall, cancellationToken);
        string? reason = approved ? null : "Rejected by supervisor.";
        decisions.Add(request.CreateResponse(approved, reason));
    }

    response = await refundsAgent.RunAsync(
        new ChatMessage(ChatRole.User, decisions), session, cancellationToken: cancellationToken);
    pending = PendingApprovals(response);
}

Console.WriteLine(response.Text);

static List<ToolApprovalRequestContent> PendingApprovals(AgentResponse response) =>
    response.Messages.SelectMany(m => m.Contents).OfType<ToolApprovalRequestContent>().ToList();

Show approvers the exact tool name and arguments, not the model's paraphrase of them, and log every decision with the approver's identity.

Guardrails with Agent Middleware#

Guardrails are checks that run outside the model: input screening, output filtering, tool allow-lists and argument validation. Agent middleware is the natural place for them, because it wraps every run and every function call regardless of what the model decides.

C#
AIAgent guarded = refundsAgent
    .AsBuilder()
    .Use(EnforceToolPolicy)
    .Use(ScreenInput, null)
    .Build();

async ValueTask<object?> EnforceToolPolicy(
    AIAgent agent,
    FunctionInvocationContext context,
    Func<FunctionInvocationContext, CancellationToken, ValueTask<object?>> next,
    CancellationToken cancellationToken)
{
    if (!AllowedTools.Contains(context.Function.Name))
    {
        context.Terminate = true; // stop the tool loop for this request
        return $"Tool '{context.Function.Name}' is not permitted for this agent.";
    }

    return await next(context, cancellationToken);
}

async Task<AgentResponse> ScreenInput(
    IEnumerable<ChatMessage> messages,
    AgentSession? session,
    AgentRunOptions? options,
    AIAgent innerAgent,
    CancellationToken cancellationToken)
{
    List<ChatMessage> input = [.. messages];
    if (!await inputScreen.IsSafeAsync(input, cancellationToken)) // for example, Prompt Shields
    {
        return new AgentResponse(new ChatMessage(ChatRole.Assistant, "I can't help with that."));
    }

    return await innerAgent.RunAsync(input, session, options, cancellationToken);
}

Middleware enforces policy; it does not make a model trustworthy. Combine it with least-privilege tools and the defenses in Responsible AI and LLM Security for .NET Applications.

Durable and Long-Running Agents#

Agents that wait for humans, call slow systems or run for hours cannot live in a single HTTP request. The Durable Agent Framework extension (Microsoft.Agents.AI.DurableTask and Microsoft.Agents.AI.Hosting.AzureFunctions, prerelease as of September 2026) runs agents on Durable Task: sessions are persisted automatically, orchestrations checkpoint after each step, and waits for external events consume no compute. Register agents with ConfigureDurableAgents(options => options.AddAIAgent(agent)), then call them from orchestrations:

C#
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.DurableTask;
using Microsoft.Azure.Functions.Worker;
using Microsoft.DurableTask;

public static class RefundReview
{
    [Function(nameof(RunRefundReviewAsync))]
    public static async Task<string> RunRefundReviewAsync(
        [OrchestrationTrigger] TaskOrchestrationContext context)
    {
        var request = context.GetInput<RefundRequest>()!;
        DurableAIAgent assessor = context.GetAgent("RefundAssessor");
        AgentSession session = await assessor.CreateSessionAsync();

        AgentResponse<RefundAssessment> assessment = await assessor.RunAsync<RefundAssessment>(
            message: $"Assess this refund request: {request.Description}",
            session: session);

        if (assessment.Result.Amount <= 100m)
        {
            return "auto-approved";
        }

        try
        {
            var decision = await context.WaitForExternalEvent<ManagerDecision>(
                eventName: "ManagerDecision", timeout: TimeSpan.FromDays(3));
            return decision.Approved ? "approved" : "rejected";
        }
        catch (OperationCanceledException)
        {
            return "expired"; // nobody decided within three days
        }
    }
}

Orchestrator code must be deterministic because it replays, so keep I/O inside agents and activities. Outside Azure Functions, workflow checkpointing in Microsoft.Agents.AI.Workflows and background responses with continuation tokens cover many long-running needs.

Connecting Agents with the A2A Protocol#

The Agent2Agent (A2A) protocol standardizes how independent agents discover and call each other across frameworks, vendors and organizations. It is now a Linux Foundation project under the Apache 2.0 license; version 1.0 was released in March 2026, followed by 1.0.1 in May. An agent publishes an Agent Card describing its skills and endpoints, and clients exchange messages, tasks and artifacts over HTTP with JSON-RPC, including Server-Sent Events for streaming and push notifications for long tasks. There is an official .NET SDK (the A2A NuGet package), and Agent Framework can both host and consume A2A agents.

C#
using A2A;
using Microsoft.Agents.AI;

A2ACardResolver resolver = new(new Uri("https://agents.contoso.example/travel/"));
AgentCard card = await resolver.GetAgentCardAsync();
AIAgent travelAgent = card.AsAIAgent();

AIAgent assistant = chatClient.AsAIAgent(
    name: "Assistant",
    instructions: "Help employees with trips. Delegate bookings to the travel agent.",
    tools: [travelAgent.AsAIFunction()]);

A2A complements MCP rather than competing with it. MCP connects an agent to tools and data; A2A connects an agent to another opaque agent that has its own reasoning, state and policies. See Model Context Protocol (MCP) in C# for the tool side. Treat remote agents as untrusted: authenticate them, validate their outputs and never forward secrets.

Failure Modes and How to Contain Them#

  • Runaway loops. Agents repeat tool calls or bounce handoffs. Set iteration caps, round limits and timeouts everywhere.
  • Tool misuse. Wrong tool, hallucinated arguments or repeated side effects. Validate arguments, make tools idempotent and require approval for irreversible actions.
  • Context overflow and drift. Long sessions push instructions out of the window or dilute them. Reduce history and restate critical constraints.
  • Error amplification. In pipelines, one agent's mistake becomes the next agent's premise. Add validation steps between agents.
  • Indirect prompt injection. Tool results, documents and remote agents can carry instructions. Treat them as data and screen them.
  • Silent cost blowups. Multi-agent chatter multiplies tokens. Track usage per run and enforce budgets.

Choosing an Orchestration Pattern#

PatternLatencyToken costPredictabilityBest for
Single agent with toolsLowLowHighMost assistants and task bots
Agents as toolsMediumMediumHighSpecialists behind one coordinator
SequentialSum of stepsMediumHighDraft, review and polish pipelines
ConcurrentSlowest branchHighHighIndependent reviews and diverse proposals
HandoffVaries per routeMediumMediumCustomer journeys with specialist routing
Group chatHighHighMediumWriter-critic loops and debates
MagenticHighestHighestLowOpen-ended research and analysis

Model choice interacts with pattern choice. Workers doing narrow tasks often run well on small models, while planners and managers benefit from stronger ones. Measure with an evaluation suite, as described in Evaluating AI Applications in .NET, rather than assuming that more agents means better answers.

Best Practices#

  • Start with one agent and good tools, and add agents only when you can name the problem they solve.
  • Bound everything: iterations, rounds, handoff routes, time and tokens.
  • Make side effects explicit with approval-required tools and idempotent operations.
  • Persist sessions and checkpoints so a crash or deployment does not lose work.
  • Keep memory scoped and inspectable, with per-user isolation and deletion support.
  • Trace every run with OpenTelemetry so you can see which agent did what and at what cost.
  • Version agent definitions (instructions, tools, models) and evaluate changes before release.

Common Pitfalls#

  • Multi-agent by default. More agents mean more latency, cost and failure surfaces, often with no quality gain.
  • Unbounded group chats and managers. Without limits, conversations continue until the budget runs out.
  • Trusting the model with authorization. The model must never decide who may do what; your code must.
  • Storing raw conversations as long-term memory. This leaks data across contexts and preserves injected instructions.
  • Non-deterministic orchestrator code. In durable orchestrations, calling DateTime.Now or making HTTP calls directly breaks replay.
  • Ignoring streaming and observability. Users and operators both need to see progress in long runs.

Frequently Asked Questions#

When should I use multiple agents instead of one?#

Use multiple agents when one agent's instructions or tool list become too large to handle reliably, when roles need different security boundaries or models, or when independent subtasks can run in parallel. If none of these apply, a single agent with good tools is cheaper, faster and easier to debug.

What happened to Semantic Kernel agents and AutoGen?#

Microsoft Agent Framework brings together ideas from both projects, combining Semantic Kernel's enterprise features with AutoGen's multi-agent orchestration patterns. For new .NET agent projects, Agent Framework is the natural starting point. Existing Semantic Kernel code can move incrementally, since both build on Microsoft.Extensions.AI abstractions.

Is the A2A protocol ready for production?#

The specification reached version 1.0 in March 2026 and is governed by the Linux Foundation, with an official .NET SDK. The protocol itself is stable enough to adopt; the harder production questions are authentication, authorization and trust between organizations, which you must design explicitly.

How do I stop agents from looping forever?#

Set hard limits at every level: maximum iterations on group chat managers, rounds, stalls and resets on Magentic workflows, limited handoff routes, timeouts on runs and a token budget per request. Log when a limit is hit, since it usually signals an unclear task or a poorly described tool.

Do I need durable agents for human approvals?#

Not always. For approvals that arrive within seconds or minutes, ApprovalRequiredAIFunction with a persisted session is enough. When approvals can take hours or days, or must survive restarts and deployments, durable orchestrations that wait for external events are the safer design.

Summary#

  • Use the least autonomy that solves the problem: single agents with good tools first, then agents as tools, then explicit orchestrations.
  • Microsoft Agent Framework provides sequential, concurrent, handoff, group chat and Magentic orchestrations, all with explicit limits.
  • Separate short-term session memory from long-term and vector memory, and treat stored memory as untrusted.
  • Put approvals, guardrails and tool policies in code through approval-required functions and middleware.
  • Use durable orchestrations for long waits, and A2A for calling agents across framework and organizational boundaries.

Further Reading#