AI-901: how to understand agent architectures in Microsoft Foundry
I will teach the competency of understanding agent architectures in Microsoft Foundry — a practical topic important for AI-901 because it shows how to coordinate LLMs, tools and workflows to solve complex tasks. Knowing this helps both for the exam (conceptual understanding) and in practice, when designing AI-assisted automation solutions.
What you need to know
Agent architectures describe how multiple components (one or more LLMs, observation modules or external tools, memory, orchestration) work together to perform autonomous or assisted tasks. An agent is not just the model; it is the pattern that manages model calls, decides when to use tools (e.g., search, APIs, computation), stores context and makes decisions about next steps.
Simple example: a customer support agent receives a question about invoices. The typical flow:
- Receive the question (user input).
- Use an intent classifier (or the LLM) to decide if it needs to access account data.
- If yes, invoke a tool that queries the database or an external service.
- Aggregate the answer with available context in memory and generate the final response to the customer.
How it works (step-by-step)
I will describe, step by step, a typical agent architecture implemented in Microsoft Foundry for a practical case: an agent that answers questions about internal documents (FAQ and contracts) and can perform semantic searches.
-
Main components:
- LLM: generative engine that produces text and justifies actions.
- Tooling: functions the agent can invoke (e.g., search, calculator, paid APIs).
- Memory: context storage (conversation history, embeddings).
- Orchestrator: logic that determines when to call each component.
-
Simplified logical flow:
- Receive user input.
- The orchestrator sends a prompt to the LLM with instructions (system prompt) and reduced context.
- The LLM decides: respond directly, ask for more information, or invoke a tool (e.g., semantic search).
- If invoking a tool, the orchestrator executes the call and returns the result to the LLM.
- The LLM produces the final response; the orchestrator updates the memory and returns to the user.
-
Example of a prompt (schema) used by the orchestrator:
System: "You are a support assistant with access to tools: SEARCH(documents), GET_ACCOUNT(account_id). Use tools when you need factual data." User: "How can I cancel my contract?" #Instructions for the LLM: describe if you need to use SEARCH, and what query."The LLM responds with an action: for example, "CALL_TOOL: SEARCH(query='cancel contract policy')" or provides the answer directly if it already knows it.
-
Practical implementation in Foundry (overview):
- Define connectors to sources (OneLake, document repositories) and index with embeddings, if needed.
- Create the tools as microservices or functions in Foundry that expose secure endpoints.
- Configure an orchestration pipeline where the system/user prompts and decision logic reside — you can use Foundry visual components to compose the flow.
- Test with prompts and logs to tune when the agent should call tools and how to summarize results.
In practice: minimal example (pseudocode)
This pseudocode illustrates the logic of an orchestrator that decides between direct response and tool use.
function handleUserInput(userInput, sessionContext) {
prompt = buildPrompt(systemInstructions, sessionContext, userInput)
llmResponse = callLLM(prompt)
if (llmResponse.intent == 'CALL_TOOL') {
toolResult = callTool(llmResponse.toolName, llmResponse.toolArgs)
updatedPrompt = buildPrompt(systemInstructions, sessionContext, userInput, toolResult)
final = callLLM(updatedPrompt)
updateMemory(sessionContext, userInput, final)
return final
} else {
updateMemory(sessionContext, userInput, llmResponse.answer)
return llmResponse.answer
}
}
Common mistakes
1) Relying on the LLM for all decisions without setting boundaries: without clear rules, the LLM may attempt to answer factually without accessing up-to-date data.
2) Excessive context in the prompt: sending the entire history or large documents creates slow responses and can exceed token limits; instead, summarize and use retrieval when necessary.
3) Unsafe or unvalidated tools: allowing the agent to perform actions (e.g., modify data) without proper authentication and logging can cause security and audit issues.
How to practice
To study this competency effectively: practice building a simple agent in Microsoft Foundry, indexing documents with embeddings and creating a search tool. Test different system prompts and record decisions and tool calls for diagnostics.
Official and free resources you should use: the Microsoft OFFICIAL Practice Assessment (it is free) and the Microsoft study guide for AI-901 (also free). These materials help validate your understanding of the concepts without ever resorting to dumps or real exam questions.
Summary
- Agent architectures combine LLMs, tools, memory and orchestration to solve complex tasks.
- A good orchestrator decides when to call tools, reduces the context sent to the LLM and maintains security and logs.
- Test and iterate: tune prompts, thresholds and tool-use policies to avoid incorrect or unsafe responses.
- Use Microsoft's official free resources (Practice Assessment and study guide) to practice safely and aligned with the exam.