DP-900: understanding and using horizontal partitioning (sharding) for non-relational data

I will teach you the skill of choosing and using a partition key (horizontal partitioning / sharding) in non-relational databases on Azure — essential for the DP‑900 and for building scalable applications. Knowing this helps you optimize performance, cost and scalability in services such as Azure Cosmos DB.

What you need to know

Horizontal partitioning (sharding) consists of splitting a collection of documents or items into multiple physical partitions based on a key value (partition key). In Azure NoSQL services, such as Azure Cosmos DB, each item is stored in a logical partition determined by the partition key; the service will manage the physical partitions behind the scenes, enabling scalability and parallelism.

Example: an e‑commerce application can use "tenantId" or "customerId" as the partition key. An order document might look like:

{
  "orderId": "A123",
  "customerId": "C789",
  "items": [...],
  "total": 79.90
}

If the partition key is customerId, all orders for the same customer reside in the same logical partition — useful for frequent customer‑based queries and for local transactions. The choice of partition key affects throughput (RU/s), latency and cost.

How it works and key points

Main properties and decisions to know:

• Cardinality: the ideal partition key has high cardinality (many distinct values) to evenly distribute data across partitions.
• Load uniformity: besides cardinality, partition key values should generate similar load — avoid keys that concentrate traffic in a small set (hot partitions).
• Queries and transactions: choose a partition key that supports query patterns. Queries that filter by the partition key are faster and cheaper; multi‑item atomic operations are typically limited to a single partition key.
• Immutability: ideally, the partition key is immutable (do not change the document value), because moving data between partitions has cost and limitations.

Practical decision: weigh the number of items per value, the read/write frequency per value and the need for local transactions. If you have users with a large number of records, avoid using a value with too many items that causes hot partitions.

In practice — configuration in Azure Cosmos DB (conceptual example)

Simplified steps to create a collection with a partition key (concept applicable to other NoSQL services):

1. Identify the access pattern: which filters are most common? by customer, by region, by product?
2. Choose the candidate partition key (e.g., /customerId or /regionId).
3. Create the account/collection specifying the partition key. In the Azure portal or via ARM/CLI, define the partition key path.
4. Test with representative loads: measure latency, RU/s and partition distribution.
5. Adjust if you detect hot partitions — a re‑modeling may be necessary (e.g., combine values: customerId_regionId) or use hashing of the id.

Conceptual example of creation via JSON (simplified, not a real command):

{
  "id": "orders",
  "partitionKey": {
    "paths": ["/customerId"],
    "kind": "Hash"
  }
}

Note: in the Azure portal/CLI you specify the partition key when creating the collection; afterwards it is difficult to change it without recreating the collection and migrating data.

Common mistakes

• Choosing a partition key with low cardinality (e.g., "status" with values like "open/closed"): causes unbalanced partitions and hot partitions.
• Choosing a key that concentrates traffic (e.g., using a date when most traffic occurs on recent dates), creating spikes and performance degradation.
• Changing the partition key frequently or using mutable keys: forces complex migrations and increases operational cost.

How to practice

Practice in the Azure portal or with the free account: create a Cosmos DB account, experiment with collections using different partition keys and use load tools to observe RU/s, latency and partition distribution. For exam preparation, take the OFFICIAL free Microsoft Practice Assessment for DP‑900 and follow the free study guide on the Microsoft site. Those official resources help you measure knowledge without using prohibited material.

Summary

• Partition key (sharding) divides data horizontally; it is vital for scalability in NoSQL like Azure Cosmos DB.
• Choose a key with high cardinality and balanced load distribution; consider query patterns and the need for local transactions.
• Avoid mutable keys or keys that concentrate traffic (hot partitions) and test with real loads before production.
• Practice with the Azure portal/CLI and use the official free Practice Assessment and study guide from Microsoft to prepare for DP‑900.