How to create an anomaly detector for medical images in Azure AI & Machine Learning

This tutorial shows how to create an anomaly detector for medical images using Azure AI & Machine Learning, useful for flagging images that need clinical review. It explains the rationale for the one-class detection method (only normal examples) and provides a practical step-by-step to train and validate a simple model.

Prerequisites

• Azure account with an active subscription and permissions to create resources.
• Azure Machine Learning workspace created.
• Python 3.8+ installed locally and pip.
• Normal medical images (e.g., X-rays/ultrasounds) in PNG/JPEG format for training.
• Basic familiarity with CLI commands and Jupyter Notebook.

Step 1: Prepare the environment in Azure ML

Create a Compute Instance or Compute Cluster in Azure Machine Learning to train the model. The goal is to have an environment with optional GPU to accelerate training. Also create a datastore for the images.

# Exemplo mínimo com Azure CLI
az ml workspace show -n  -g 
# Criar compute cluster (exemplo)
az ml compute create -n gpu-cluster --type amlcompute --size STANDARD_NC6 --min-instances 0 --max-instances 4

Step 2: Organize data (only normal images)

For anomaly detection with a one-class approach, train with normal images. Place the images in a local folder organized accordingly and upload to the workspace datastore.

mkdir -p data/train/normal
# copiar imagens para data/train/normal
az ml datastore upload -n workspaceblobstore -p data/train --target-path data/train --overwrite

Step 3: Choose the approach and model

A simple and effective solution is to use a CNN-based autoencoder: the model learns to reconstruct normal images; anomalous images present higher reconstruction error. Explanation: the autoencoder learns normal patterns and fails to reconstruct what is different.

# Exemplo de arquitetura mínima em Keras (ficheiro train_autoencoder.py)
from tensorflow import keras
from tensorflow.keras import layers

def build_autoencoder(input_shape=(128,128,1)):
    inputs = keras.Input(shape=input_shape)
    x = layers.Conv2D(32, 3, activation='relu', padding='same')(inputs)
    x = layers.MaxPool2D(2, padding='same')(x)
    x = layers.Conv2D(64, 3, activation='relu', padding='same')(x)
    encoded = layers.MaxPool2D(2, padding='same')(x)

    x = layers.Conv2DTranspose(64, 3, strides=2, activation='relu', padding='same')(encoded)
    x = layers.Conv2DTranspose(32, 3, strides=2, activation='relu', padding='same')(x)
    decoded = layers.Conv2D(1, 3, activation='sigmoid', padding='same')(x)

    return keras.Model(inputs, decoded)

Step 4: Train on Azure ML

Package the training script and submit an experiment to Azure ML. Configure the environment (conda/conda_dependencies.yml) and use the compute you created.

# Exemplo de submissão com SDK (snippet)
from azureml.core import Workspace, Experiment, ScriptRunConfig, Environment
ws = Workspace.from_config()
env = Environment.from_conda_specification(name='ae-env', file_path='conda_dependencies.yml')
src = ScriptRunConfig(source_directory='.', script='train_autoencoder.py', compute_target='gpu-cluster', environment=env)
exp = Experiment(ws, 'ae_medical_images')
run = exp.submit(src)
run.wait_for_completion(show_output=True)

Step 5: Compute anomaly threshold

After training, compute the mean reconstruction error on normal validation images; set a threshold (for example mean + 3*std) to classify anomalies.

# Código de exemplo para calcular erro de reconstrução
import numpy as np
from tensorflow.keras.models import load_model

model = load_model('outputs/autoencoder.h5')
# carregar validação normal como array: X_val (N, H, W, 1)
recon = model.predict(X_val)
mse = np.mean((X_val - recon)**2, axis=(1,2,3))
threshold = mse.mean() + 3*mse.std()
print('Threshold:', threshold)

Step 6: Run inference and detect anomalies

For a new image, normalize, predict, compute the reconstruction error and compare with the threshold. If the error is higher, mark it as a potential anomaly.

def is_anomaly(img, model, threshold):
    # img: array 1,H,W,1 normalized 0-1
    recon = model.predict(img)
    err = np.mean((img - recon)**2)
    return err > threshold, err

# exemplo
an, err = is_anomaly(X_new[0:1], model, threshold)
print('Anomaly:', an, 'Error:', err)

Verify the result

Validate with a labeled set of images (normal and anomalous) and compute simple metrics: confusion matrix, precision, recall. Visualize images with the highest error and their reconstructions to confirm whether the anomalies are plausible.

# Exemplo rápido de avaliação
from sklearn.metrics import confusion_matrix, precision_score, recall_score
preds = [is_anomaly(x[None,...], model, threshold)[0] for x in X_test]
cm = confusion_matrix(y_test, preds)
print('Confusion matrix:\n', cm)
print('Precision:', precision_score(y_test, preds))
print('Recall:', recall_score(y_test, preds))

Conclusion

With a simple autoencoder in Azure AI & Machine Learning it is possible to create an anomaly detector for medical images for initial triage. Next steps include improving preprocessing (data augmentation, intensity normalization), experimenting with more advanced models (Variational Autoencoder, GANs) and integrating the inference endpoint for clinical use. Tip: test several thresholds and validate with clinical experts before production — which threshold seems safest for your application?