(+351) 21 24 10006  ·  info@bconcepts.pt
Carnaxide, Lisbon

How to create a multimodal translation service in Azure AI & Machine Learning

João Barros 20 de November de 2026 5 min read

This guide shows how to create a simple multimodal translation service in Azure AI & Machine Learning: extract text from images, transcribe audio and translate to another language. Useful to automate subtitling, customer support assistance or processing multimedia content.

Prerequisites

  • Azure account with permissions to create resources (Cognitive Services / Azure ML).
  • Azure CLI installed and authenticated (az login).
  • Python 3.8+ and pip.
  • Key and endpoint for Azure AI Vision and Azure AI Speech or a single resource that supports both.
  • Text editor and sample files: image with text and a short audio file (wav).

Step 1: Explain the architecture and choose services

Why: it is important to understand the pieces. We will use Azure AI Vision to extract text from images (OCR), Azure AI Speech to transcribe audio and Azure AI Language Translator (or the resource's integrated Language endpoint) to translate text. The flow is: obtain text (image/audio) → normalize → translate → return result.

Step 2: Prepare the Python environment

Install the official libraries and configure environment variables for the keys and endpoints. This simplifies HTTP/SDK calls.

pip install azure-ai-formrecognizer azure-cognitiveservices-speech requests python-dotenv # create .env with AZURE_VISION_KEY, AZURE_VISION_ENDPOINT, AZURE_SPEECH_KEY, AZURE_SPEECH_REGION, AZURE_TRANSLATOR_KEY, AZURE_TRANSLATOR_ENDPOINT 

Step 3: Extract text from an image with Azure AI Vision (OCR)

Use the OCR endpoint or Form Recognizer/Read API. Minimal example in Python using the Read API via HTTP to keep it simple and compatible.

import os, time, requests
from dotenv import load_dotenv
load_dotenv()
VISION_ENDPOINT = os.getenv('AZURE_VISION_ENDPOINT')
VISION_KEY = os.getenv('AZURE_VISION_KEY')
headers = {'Ocp-Apim-Subscription-Key': VISION_KEY, 'Content-Type': 'application/octet-stream'}
img_path = 'sample.jpg'
with open(img_path, 'rb') as f:
    data = f.read()
resp = requests.post(VISION_ENDPOINT + '/vision/v3.2/read/analyze', headers=headers, data=data)
operation_location = resp.headers.get('Operation-Location')
status = ''
while status != 'succeeded':
    time.sleep(1)
    r = requests.get(operation_location, headers={'Ocp-Apim-Subscription-Key': VISION_KEY})
    result = r.json()
    status = result['status']
texts = []
if status == 'succeeded':
    for readResult in result['analyzeResult']['readResults']:
        for line in readResult['lines']:
            texts.append(line['text'])
image_text = '\n'.join(texts)
print('Texto extraído:', image_text)

Step 4: Transcribe audio with Azure AI Speech

Use the SDK azure-cognitiveservices-speech to transcribe a local WAV file to text. We keep the example synchronous for small files.

import azure.cognitiveservices.speech as speechsdk
SPEECH_KEY = os.getenv('AZURE_SPEECH_KEY')
SPEECH_REGION = os.getenv('AZURE_SPEECH_REGION')
audio_file = 'sample.wav'
speech_config = speechsdk.SpeechConfig(subscription=SPEECH_KEY, region=SPEECH_REGION)
audio_input = speechsdk.AudioConfig(filename=audio_file)
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_input)
result = speech_recognizer.recognize_once()
if result.reason == speechsdk.ResultReason.RecognizedSpeech:
    audio_text = result.text
else:
    audio_text = ''
print('Transcrição:', audio_text)

Step 5: Translate text with Azure Translator (or Azure AI Language)

Send the extracted/transcribed text to the translation endpoint. HTTP example for the Translator Text service.

TRANSLATOR_KEY = os.getenv('AZURE_TRANSLATOR_KEY')
TRANSLATOR_ENDPOINT = os.getenv('AZURE_TRANSLATOR_ENDPOINT')
params = {'api-version': '3.0', 'to': 'pt'}
headers = {'Ocp-Apim-Subscription-Key': TRANSLATOR_KEY, 'Content-Type': 'application/json'}
# traduzir imagem
body = [{'text': image_text}]
resp = requests.post(TRANSLATOR_ENDPOINT + '/translate', params=params, headers=headers, json=body)
translated_image = resp.json()[0]['translations'][0]['text']
# traduzir áudio
body2 = [{'text': audio_text}]
resp2 = requests.post(TRANSLATOR_ENDPOINT + '/translate', params=params, headers=headers, json=body2)
translated_audio = resp2.json()[0]['translations'][0]['text']
print('Imagem traduzida:', translated_image)
print('Áudio traduzido:', translated_audio)

Step 6: Join the flow and handle common errors

Combine the steps into a script that decides the source (image/audio) and executes OCR → translation or transcription → translation. Common errors: wrong keys, invalid endpoints, files with unsupported formats. Check the HTTP status and error messages in the error JSON.

# Schematic example: call functions extract_text_from_image(), transcribe_audio(), translate_text()
# Handle exceptions, check resp.status_code and error fields before accessing the JSON

Verify the result

Confirm that you receive readable text from the image and audio and that the translations make sense. Test with multiple images (different fonts, languages) and audio files. Check response times and error messages in the terminal. If you get 401/403 codes, confirm the keys; if you get 429, you are exceeding the quota and should reduce call rate or use a higher tier.

Conclusion

You now have a basic multimodal translation service in Azure AI & Machine Learning that combines Azure AI Vision, Azure AI Speech and Translator. Next steps: improve text normalization, add automatic language detection, parallelize calls for greater scale and integrate into an Azure Functions endpoint or Azure ML endpoint for production. Tip: log payloads and timings to optimize costs and performance — which multimodal flow do you want to automate first?