AI-901: implement entity detection with Microsoft Foundry

I will teach how to implement and apply named entity recognition (NER) with Microsoft Foundry — a useful skill for the AI-901 because it tests your practical understanding of language processing and integration of AI services into real solutions.

What you need to know

NER is the task of locating and classifying words or phrases in a text into predefined categories, such as People, Organizations, Locations, Dates, Monetary amounts, among others. In practical terms, NER converts unstructured text into structured entities that can feed reports, workflows, or knowledge bases.

Simple example: the sentence "A reunião com Maria Silva em Lisboa será dia 10 de outubro" can produce the entities: Maria Silva (Person), Lisbon (Location), October 10 (Date).

How it works

The typical workflow to implement NER with Microsoft Foundry involves:

1. Selecting or training a NER model (pretrained or custom model).
2. Preparing and normalizing the text data (cleaning, tokenization, labeling training data, if needed).
3. Packaging the model in the Foundry environment and exposing an inference endpoint.
4. Consuming the endpoint in applications to extract entities and map results to your data schema.

// Conceptual example (pseudo-code) of calling a NER endpoint:
request = {
  "text": "Reunião com Maria Silva em Lisboa dia 10 de outubro."
}
response = callFoundryEndpoint("ner-endpoint", request)
// response.entities -> [{text: "Maria Silva", type: "Person", start: 14, end: 25}, ...]

In Foundry, this can be done using a language model that supports NER tasks or a specialized component. If you choose a custom model, you need to annotate a set of examples and train with Foundry's pipeline.

In practice

Step-by-step practical for a typical scenario:

1. Choose the model:

   Use a pretrained model to start. In Foundry, look for models capable of NER tasks or use an LLM with an appropriate prompt for entity extraction.

2. Prepare data:

   Gather text examples representative of your domain. If you will customize, annotate them with the desired entity schema (e.g. Person, Org, Location, Product).

3. Train/tune (optional):

   If domain entities are not covered by generic models, use Foundry's training environment for fine-tuning. Split the data into training/validation and monitor metrics like precision, recall and F1.

4. Package and deploy:

   Create a deployment in Foundry to expose an inference endpoint. Define quotas, target latencies and logs for monitoring.

5. Integrate with applications:

   Implement calls to the endpoint from services that need the entities (ETL pipelines, Power BI dashboards, chatbots, etc.). Mapping the output to your schema is critical: convert character positions into structured fields.

6. Monitor and iterate:

   Collect error examples and feed them back into the dataset. Adjust the model and retrain according to the most frequent errors.

Common mistakes

• Ignoring domain context: pretrained models may not recognize entities specific to your sector (e.g. internal product names). The solution is to annotate and train on a representative set.
• Not validating boundaries and overlapping entities: sentences can contain nested or overlapping entities; explicitly handle how your system marks priorities and resolutions.
• Assuming performance is only accuracy: don’t forget recall and F1, as well as latency and cost when operating in production on Foundry.

How to practice

To practice safely and aligned with the AI-901:

• Do the OFFICIAL Microsoft Practice Assessment (it’s free) to understand what types of skills are evaluated.
• Consult the official Microsoft study guide for AI-901 (also free) and follow the modules on language processing and AI services.
• Create a small lab in Foundry: use a public dataset (e.g. a news dataset) and implement a simple NER pipeline, exposing an endpoint and consuming it from a script.

In summary

• NER converts unstructured text into structured entities useful in business scenarios.
• With Microsoft Foundry you can use pretrained models or tune models to your domain, expose endpoints and integrate into applications.
• Prepare representative data, evaluate with precision/recall/F1 and monitor latency and costs in production.
• Practice with the OFFICIAL Practice Assessment and the free Microsoft study guide and implement a mini-lab in Foundry to consolidate the skills.