Sharding Concept in MongoDB

Data Partitioning

MongoDB distributes data or shards at the collection level or the data of a collection may be distributed into several shards. Distribution of a collection’s data is done by a shard key.

Shard Key

A shard key shards a collection and determines the distribution of the collection’s documents among the cluster’s shards. A shard key may be an indexed field or an indexed compound field present in every document of the collection. The shard key is also divided into chunks and these chunks are evenly distributed across the shards. MongoDB uses the following two techniques to divide the shard key into chunks.

• Range Based Partitions
• Hash Based Partitions

Range Based Partitions

MongoDB uses range-based partitions for range-based sharding of the collection’s data. In range-based sharding MongoDB divides the shard key into a number of non-overlapping ranges called Chunks. Each chunk contains a range of values from minimum values to maximum values.

Hash Based Sharding

MongoDB uses hash-based partitions for hash-based sharding. In a hash-based partition MongoDB computes a hash of the field’s value and then uses this hash to create the chunks in the shard key. A hash-based partition provides a more random distribution of collections in the clusters.

Balanced Data Distribution

In the process of sharding there is a large possibility that the stored data becomes imbalanced in the cluster. Data imbalance may occur due to any of the following reasons.

• Addition/Removing of new data.
• Addition/Removing of cluster.
• A specific shard contains more chunks than other chunks.
• The size of a specific chunk is greater than another chunk's size.

MongoDB always tries to balance the data distribution, for this MongoDB uses the following two approaches:

• Splitting
• Balancing

Splitting

Splitting is a background process that restricts the chunks from growing too large. When the size of a chunk crosses the value of a specified chunk size, MongoDB splits the chunks into half. In the split process MongoDB does not modify the shards or migrate any data. The splits provide sufficient meta-data change.

Balancing

Balancing is a background process. Due to an uneven distribution of sharded collections, the query router runs a balancer process. The balancer process migrates chunks from the shard containing a large number of chunks to the shard that contains the least numbers of chunks. The balancer process can be initiated from any query router. After successful chunk migration the metadata regarding the location of the chunks on the config server is updated.

In the preceding image we can see that the number of chunks in shard A is just double that of shard B. So this is an imbalance condition. Now the migration of chunks from shard A to shard B will occur.

Now we can see that after migration of chunks from shard A to shard B both shards are in a balanced state.

Advantages of Sharding

• Sharding distributes the data over multiple shards so it reduces the number of operations for each shard.
• Removes the dependency from a single server.
• Protects against system failover.
• Increases capacity and throughput.

References taken from:

• MongoDB - Day 16 (Sharding)
• Introduction to Sharding

Here's a free e-book on MongoDB: MongoDB Architecture Guide

MongoDB, Inc. Goes Public

MongoDB has started trading publicly on NASDAQ.

#news