AI-901: implement text classifiers with Microsoft Foundry

In this guide I explain how to implement a text classifier using Microsoft Foundry — from preparing labeled data to training and evaluating the model. This skill is common in the "Identify AI concepts and capabilities" area and is very useful in practice to automate message triage, ticket categorization, and moderation.

What you need to know

A text classifier assigns labels to pieces of text (for example: "support", "sales", "spam"). Key concepts: labels, supervised training, train/validation/test split, metrics (accuracy, precision, recall, F1), overfitting and preprocessing (tokenization, cleaning). In Foundry you use a pipeline that prepares the data, chooses an algorithm (or a pre-trained model) and runs training and evaluation. Example: you have 3,000 emails labeled as {"Suporte","Vendas","Outros"}. You will split the data, train, and confirm that the model distinguishes the classes well.

How it works

Essential high-level steps:

1. Creation and labeling: obtain representative examples for each label.
2. Cleaning and preprocessing: normalize case, remove HTML, tokenize and, possibly, apply lemmatization/stemming.
3. Data splitting: typically 70% train, 15% validation, 15% test (adjust according to availability).
4. Model choice: lightweight models (Logistic Regression, Naive Bayes) for a baseline; then experiment with models from the LLM ecosystem or embeddings combined with a classifier.
5. Training and validation: tune hyperparameters, use early stopping to reduce overfitting.
6. Evaluation: compute accuracy, precision, recall and F1 per class; analyze the confusion matrix.
7. Deployment: package the pipeline in Foundry as an inference endpoint.

Conceptual example of preprocessing in pseudo-code (can be run in notebooks inside Foundry):

# Exemplo simplificado de pré-processamento
def preprocess(text):
    text = text.lower()
    text = remove_html(text)
    text = remove_punctuation(text)
    tokens = tokenize(text)
    tokens = remove_stopwords(tokens)
    return ' '.join(tokens)

corpus = [preprocess(t) for t in textos]

In practice (step by step in Foundry)

Summary of the concrete actions you will perform in Microsoft Foundry:

1. Import data: load the labeled dataset (CSV, parquet) into the Foundry data workspace and validate integrity.
2. Explore and clean: use notebooks or the data preparation tools to inspect label imbalance, remove duplicates and apply preprocessing.
3. Generate features: choose BoW/TF-IDF or embeddings. For a quick prototype, generate TF-IDF; for better results, obtain embeddings with a model supported by Foundry and use them as classifier input.
4. Train: create a training experiment that uses an algorithm (e.g., Logistic Regression, XGBoost) or a supervised model over embeddings. Configure checkpoints and cross-validation, if applicable.
5. Evaluate: produce per-class metrics and a confusion matrix. Inspect misclassified examples to identify patterns (for example, ambiguous labels).
6. Publish endpoint: package the preprocessing pipeline + model as an inference endpoint in Foundry; test with real examples before production.

Evaluation example in pseudo-code (notebook):

from sklearn.metrics import classification_report, confusion_matrix
preds = model.predict(X_test)
print(classification_report(y_test, preds))
print(confusion_matrix(y_test, preds))

Common mistakes

1) Imbalanced classes: training without addressing imbalance (undersampling/oversampling/class weighting) leads to misleading metrics (high accuracy but poor detection of minority classes).

2) Data leakage: including metadata or future information in training (for example, fields derived from the label) or not properly separating train/validation/test sets skews evaluation.

3) Relying on a single global metric: accuracy can hide poor per-class performance. Use precision/recall/F1 per label and analyze the confusion matrix.

How to practice

To prepare for the AI-901 practice this flow with small public datasets (e.g., review sets, emails, tickets). Microsoft provides an official free Practice Assessment; use it to gauge exam knowledge. Also consult the Microsoft official study guide, which is free and lists the exam topics. Do not use or share unofficial materials like "exam dumps" — focus on learning the concepts and applying them in Foundry.

In summary

• A text classifier maps inputs to labels; it requires labeled data, preprocessing and careful evaluation.
• In Foundry you build pipelines that prepare data, generate features (TF-IDF or embeddings), train and publish inference endpoints.
• Avoid imbalance, data leakage and single-metric reliance — analyze precision/recall/F1 and the confusion matrix.
• Practice with public datasets and use the official Practice Assessment and the Microsoft free study guide to validate knowledge.