AI-901: how to master computer vision concepts with Foundry

I will teach the computer vision competency relevant for the AI-901 exam: identify and understand image classification and object detection models, and how they integrate into solutions with Microsoft Foundry. This competency is important both to answer the theoretical areas of the exam and to design practical solutions that automate image analysis in the enterprise.

What you need to know

Computer vision is the area of AI that enables machines to interpret images and video. The two fundamental tasks are:

• Image classification: assign a label (class) to an entire image — for example, indicate whether a photograph contains a cat or a dog.
• Object detection: identify and locate multiple objects in an image, typically returning bounding boxes and classes — for example, detect cars in a road image and indicate their positions.

Practical example: in a warehouse you want to classify an image as "damaged box" or "intact box" (image classification). In a security camera you want to detect people and equipment for security monitoring (object detection).

How it works (conceptual flow)

The typical workflow to use computer vision in a solution with Foundry involves several steps:

1. Image collection and labeling.
2. Model choice: pre-trained models (transfer learning) vs training from scratch.
3. Training / fine-tuning the model with a labeled dataset.
4. Deploying the model and integrating it into the application (batch or real-time inference).
5. Monitoring and continuous re-training.

In Microsoft Foundry (part of the Fabric/Foundry ecosystem), you will typically use a service or component that allows you to:

• Upload image datasets (stored in OneLake or a supported container).
• Use training and evaluation pipelines with vision models (e.g., ViT, Faster R-CNN, YOLO or specialized models provided by Foundry).
• Manage model versions and deploy to inference endpoints.

In practice: step-by-step for a mini-lab

Here is a simplified practical flow to train an object detection model with Foundry. I do not describe the exact product interface (it may change), but I explain the steps and conceptual commands.

1) Prepare the dataset

# Typical structure
/images/
  img001.jpg
  img002.jpg
/annotations/
  img001.json  # bounding boxes and classes in COCO or Pascal VOC format
  img002.json

Ensure annotations follow a standard format (COCO is very common) to facilitate training with frameworks that Foundry supports.

2) Import to OneLake / Foundry

Upload the images and annotations to Foundry's data area. Tag the dataset with metadata: type (images), annotations format (COCO), classes present.

3) Choose model and training configuration

# Conceptual example of parameters
model: faster_rcnn_resnet50
input_size: 800
batch_size: 8
epochs: 30
learning_rate: 0.001
augmentation: [flip, color_jitter]

Use transfer learning if you have few examples: preserve parts of the pre-trained model and fine-tune the final layers.

4) Train and validate

Run the training job in Foundry. Monitor metrics such as mAP (mean Average Precision) for object detection and accuracy / F1 for classification. Validate with a separate set of unseen images.

5) Deploy and inference

After evaluation, create an inference endpoint. Decide between batch inference (e.g., analyze stored images weekly) or real-time inference (camera with streaming).

# Conceptual example of inference call (pseudo)
POST /predict
{
  "image": "base64-encoded-image"
}
# response:
{
  "predictions": [
    {"class": "person", "score": 0.92, "bbox": [x,y,w,h]},
    {"class": "helmet", "score": 0.88, "bbox": [x,y,w,h]}
  ]
}

6) Monitoring and continuous improvement

Log accuracy and latency metrics at the endpoint. Periodically collect misclassified cases and re-train the model with more labeled data.

Common mistakes

• Underestimating the quality of annotations: imprecise bounding boxes or inconsistent classes strongly degrade performance.
• Ignoring class imbalances: many images of one class and few of another lead to biased models — use augmentation or resampling techniques.
• Not evaluating latency vs accuracy: a very heavy model may be accurate but impractical for real-time inference.

How to practice

To prepare for the exam and consolidate this competency:

• Do the Microsoft OFFICIAL Practice Assessment for AI-901 (it's free) — it helps you assess your knowledge of the measured topics.
• Read the Microsoft official study guide for AI-901 (also free) where the skills measured and recommended resources are described.
• Practice in a lab environment: import a small dataset (e.g., a subset of COCO or Pascal VOC), train an object detection model and deploy an inference endpoint. If you have access to Foundry/Fabric, follow the import, training and deploy flow there.

In summary

• Computer vision includes image classification and object detection — each serves distinct purposes.
• In a practical flow: collection/labeling → training (transfer learning when possible) → deploy → monitoring.
• Annotation quality, class balance and latency requirements are crucial for success.
• Use the official Practice Assessment and Microsoft study guide (free) and practice with a mini-lab in Foundry to consolidate knowledge.