AI-901: understand and apply LLMs with Microsoft Foundry

This lesson focuses on the skill of understanding and applying Large Language Models (LLMs) using Microsoft Foundry, a core ability for the AI-901. Knowing what LLMs are, when they are the right choice and how to integrate them safely is useful for the exam and in real-world AI solution development scenarios.

What you need to know

Large Language Models (LLMs) are models trained on vast amounts of text to predict and generate natural language. They are used for tasks such as text generation, summarization, translation, text classification and conversation. In the context of Microsoft Foundry (which integrates AI capabilities and pre-trained models), the competency requires that you understand:

• What distinguishes LLMs from other AI models (for example vision models): focus on text, statistical reasoning capabilities and continuous token generation).
• Appropriate use cases: response generation, conversational assistants, content creation, information extraction and text transformation.
• Limitations: hallucinations (invented responses), prompt sensitivity, resource consumption and privacy/security needs.

Practical example: asking an LLM to summarize internal policies. The LLM can produce a coherent summary, but it may also omit critical points or fabricate details. Therefore, validation with rules, fact-checking and confidentiality control is necessary.

How it works

Simple, practical technical summary on how LLM integrations in Foundry work:

1. Input (prompt): user text, possibly with additional context (instrumentation with embeddings/knowledge bases).
2. Tokenization: the text is converted into tokens that the model processes.
3. Inference by the LLM: prediction of subsequent tokens to generate the response.
4. Post-processing: filtering, safety (sensitive content detection) and formatting before delivering to the user.

In practice, Foundry acts as an orchestration layer: it allows choosing a model (for example a model from Microsoft's offering or a custom model), managing calls, applying security policies and monitoring costs and usage.

// Exemplo conceptual (pseudocódigo) de requisição a um LLM via API em Foundry
request = {
  model: "nome_do_modelo",
  prompt: "Resume a política X em 3 pontos claros:",
  max_tokens: 200,
  temperature: 0.2
}
response = foundry.llm.generate(request)
summary = response.text
// Aplicar validações adicionais
if (containsSensitiveInfo(summary)) {
  redact(summary)
}
return summary

In practice

Step-by-step for a simple integration, oriented to an exam/practical scenario:

1. Define the use case: choose the task (e.g., summarization of internal documents).
2. Select model: in Foundry, evaluate trade-offs (cost vs. capability vs. latency). For precision tasks, use models with low temperature.
3. Build robust prompts: include clear instructions, examples (few-shot) and constraints (length, output format).
4. Add security layers: input/output filtering, sensitive data detection and restricted logging.
5. Implement human validation: approval flows for critical outputs.

Example of a robust prompt (template):

"You are a helpful assistant. Summarise the following document in 3 bullet points, each with at most 20 words. If the document contains confidential details, respond: 'CONFIDENTIAL — human review required.' Document: "

Common mistakes

Here are frequent pitfalls when working with LLMs in Foundry:

• Blindly trusting outputs: not validating facts nor implementing human review for critical outputs.
• Vague or overly open prompts: produce inconsistent or irrelevant responses; specificity and examples are needed.
• Ignoring data governance: sending sensitive data to models without applying encryption, redaction or residing in environments with approved compliance.

How to practice

To consolidate this competency, use Microsoft's official resources: take the OFFICIAL Practice Assessment (free) for AI-901 and follow the official Study Guide (free) that covers LLM concepts and integrations. Practice in labs with Foundry (or demo environments) implementing small flows: summarization, classification and an assistant with human validation. Do not use or consult exam dumps — use only the official resources for practice.

In summary

• LLMs generate and transform text; they are useful for summarization, conversation and content generation.
• Foundry orchestrates models, security policies and monitoring; choosing the right model and tuning prompts is critical.
• Implement human validation, output filtering and protections for sensitive data to minimize risks of hallucination and exposure.
• Practice with Microsoft's official resources: Practice Assessment and Study Guide (free).