How to detect and classify industrial images with Azure AI Vision: step by step

This tutorial shows how to use Azure AI Vision to detect and classify defects in industrial images (for example, scratches or cracks). The approach is useful to automate visual inspection, reduce human error and integrate vision models into production pipelines.

Prerequisites

• Azure account with an active subscription and permissions to create resources.
• Azure AI Vision resource (or Azure OpenAI Vision) created and key/endpoint available.
• Python 3.8+ installed locally and pip.
• Labeled image dataset for training (folder with subfolders per class: ok, defect).
• Basic knowledge of Python and the command line.

Step 1: Prepare the data and folder structure

Organize the images into two main folders: train/ok, train/defeito, and optionally validation/ok, validation/defeito. Small images (224x224) make training faster. Check common formats (.jpg, .png).

# Estrutura de exemplo
dataset/
  train/
    ok/
      img001.jpg
      img002.jpg
    defeito/
      img101.jpg
  validation/
    ok/
    defeito/

Step 2: Create and configure the Azure AI Vision resource

In the Azure Portal create an 'Azure AI Vision' resource (or similar). Save the endpoint and the key. You will need these for inference calls and, if you use training on the service, for uploading the data.

Step 3: Train a local classification model (optional) with PyTorch/fastai

If you prefer to train locally before using on Azure, a simple flow with fastai speeds up the process. This produces a model that can be converted and published.

pip install fastai==2.7.12 torchvision pillow --quiet

from fastai.vision.all import *
path = Path('dataset')
dls = ImageDataLoaders.from_folder(path/'train', valid_pct=0.2, item_tfms=Resize(224))
learn = cnn_learner(dls, resnet18, metrics=accuracy)
learn.fine_tune(3)
learn.export('model.pkl')

Step 4: Convert and package the model for inference (ONNX)

Converting to ONNX enables deployment on multiple services. Below is a minimal example of exporting to ONNX from a PyTorch/fastai model.

import torch
from torchvision import models

# exemplo com um modelo simples do PyTorch
model = models.resnet18(pretrained=False)
model.eval()
dummy_input = torch.randn(1,3,224,224)
torch.onnx.export(model, dummy_input, 'model.onnx', opset_version=11)

Step 5: Trigger the model for inference (Azure)

There are several options: use Azure Machine Learning for endpoints or use Azure Container Instances with a service that serves ONNX. For simplicity, we will show how to do direct inference with the Azure AI Vision service for detection/classification when available (analysis/assessment API) or call a hosted model endpoint.

# Exemplo de chamada HTTP simples para um endpoint de inferência básico
import requests

endpoint = 'https://.cognitiveservices.azure.com/vision/models//analyze'
key = ''
image_path = 'dataset/validation/defeito/img101.jpg'
with open(image_path, 'rb') as f:
    img_data = f.read()

headers = {'Ocp-Apim-Subscription-Key': key, 'Content-Type': 'application/octet-stream'}
resp = requests.post(endpoint, headers=headers, data=img_data)
print(resp.status_code, resp.text)

Step 6: Run batch inference and evaluate metrics

Run inference on all validation images and compute accuracy, precision and recall to identify weaknesses (imbalanced classes, false positives).

import os
from sklearn.metrics import classification_report

y_true = []
y_pred = []
for cls in ['ok','defeito']:
    folder = f'dataset/validation/{cls}'
    for fname in os.listdir(folder):
        path = os.path.join(folder, fname)
        # chamar a função que faz a request e retorna a classe predita
        pred = inferir_imagem(path)  # implemente conforme exemplo anterior
        y_true.append(cls)
        y_pred.append(pred)

print(classification_report(y_true, y_pred))

Verify the result

Confirm that predictions match the validation labels. Check the confusion matrix and metrics: an accuracy above 85% is a good initial target, but it depends on the case. Test new images from the production process and assess false negatives (very critical in inspection).

Conclusion

With Azure AI Vision and a simple training/inference cycle it is possible to automate industrial visual inspection. Next steps: improve the data (augmentations, more images), test Azure Machine Learning for managed deployment and monitor drift. Tip: start by checking for imbalanced classes and increase images synthetically before complicating the model — which inspection problem do you want to solve first?