Sharding is how MongoDB spreads one collection across several servers. Each document is assigned to a server, called a shard, according to the value of a field you choose, the shard key. Applications connect to a router process that sends each operation to the shard or shards that hold the relevant data, so the cluster looks like a single database.
Sharding a collection is one command, run through the router:
sh.shardCollection("shop.orders", { customerId: 1, orderDate: 1 })
The command is the easy part. Choosing the shard key and deciding whether you need sharding at all are where the work is.
Scaling up versus scaling out
When a database outgrows its server you have two options.
Vertical scaling means a bigger machine: more CPU, RAM and disk. It needs no changes to the application, but hardware has a ceiling and the price climbs steeply near it.
Horizontal scaling means more machines, each holding part of the data. Capacity grows by adding servers. The cost is operational complexity: more processes to run, monitor, back up and upgrade.
Sharding is MongoDB's horizontal scaling. It is separate from replication. A replica set keeps several copies of the same data for availability; sharding splits the data so each shard holds a different part. A production cluster uses both.
Components of a sharded cluster
| Component | Process | Role |
|---|---|---|
| Shard | mongod | Stores a subset of the data. Each shard is a replica set. |
| Config servers | mongod | Store the cluster metadata: which ranges of shard key values live on which shard. Deployed as a replica set. |
| Router | mongos | Receives client operations, looks up the metadata and forwards each operation to the right shards. Stores no data. |
Config servers have always been a replica set in supported releases; the old arrangement of exactly three independent mirrored config servers was removed years ago. Starting in MongoDB 8.0, one shard can also act as the config server (a config shard), which removes a replica set from small clusters.
Run more than one mongos so the router is not a single point of failure, and list them all in the application's connection string. Clients must always connect through mongos, never directly to a shard. Since MongoDB 8.0 a direct connection to a shard rejects most commands for this reason.
Shard keys
The shard key is one field or several fields of the documents. MongoDB divides the range of possible key values into contiguous, non-overlapping ranges called chunks and assigns each chunk to a shard. A shard key needs an index that starts with the key fields. For an empty collection sh.shardCollection() creates the index; for a collection that already has data you create it first:
use shop
db.orders.createIndex({ customerId: 1, orderDate: 1 })
sh.shardCollection("shop.orders", { customerId: 1, orderDate: 1 })
Since MongoDB 6.0 you do not need to run sh.enableSharding() on the database beforehand.
What makes a good shard key
- High cardinality. Many distinct values. A key such as
countrywith a few dozen values cannot be divided finely enough, and all documents sharing one value must stay in the same chunk. - Even frequency. No single value accounts for a large share of the documents. One huge customer under a
customerIdkey produces a chunk that cannot be split. - Not monotonically increasing. With a ranged key on a timestamp or an
ObjectId, every new document falls into the last chunk, so one shard takes all the inserts. - Present in your common queries. This matters as much as distribution, as the next section shows.
A compound key often satisfies all four. { customerId: 1, orderDate: 1 } spreads writes across customers, keeps one customer's orders together and lets a large customer's data be split by date.
Ranged and hashed sharding
Ranged: { customerId: 1 } | Hashed: { customerId: "hashed" } | |
|---|---|---|
| Placement | Nearby key values share a chunk | A hash of the value decides the chunk |
| Range queries on the key | Routed to the few shards covering the range | Usually sent to all shards |
| Monotonic keys | Create a hot shard | Spread evenly |
| Unique constraint on the key | Supported | Not supported |
Hashing fixes a monotonic key, not a low-cardinality one. Identical values hash identically.
How queries are routed
If a query includes the shard key, or a leading prefix of a compound key, mongos sends it only to the shards that can contain matching documents. Otherwise it sends the query to every shard and merges the results.
// Targeted: customerId is the prefix of the shard key
db.orders.find({ customerId: 4211 })
// Broadcast to every shard: no shard key field in the filter
db.orders.find({ status: "pending" })
Broadcast queries get slower as you add shards, which is the opposite of what sharding is for. Check how a query is routed with explain(): the winning plan's top stage is SINGLE_SHARD for a query that reached one shard and SHARD_MERGE when results were combined from several.
db.orders.find({ customerId: 4211 }).explain().queryPlanner.winningPlan.stage
Balancing
A background process, the balancer, keeps each sharded collection's data spread evenly. It runs on the primary of the config server replica set and moves ranges of documents from fuller shards to emptier ones.
In current versions (MongoDB 6.0 and later) the balancer compares the amount of data per shard, not the number of chunks. A collection counts as balanced when the difference between the fullest and emptiest shard is less than three times the range size. The default range size is 128 MB, so migration starts at a difference of 384 MB. This also means a collection smaller than that stays on one shard, which confuses people testing sharding with a few thousand documents.
Migrations add load. They run one at a time per shard, and you can restrict them to a quiet time window. Check the state with:
sh.status() // shards, databases, shard keys
sh.balancerCollectionStatus("shop.orders")
db.orders.getShardDistribution() // data size and document count per shard
Adding a shard leaves it empty until the balancer moves data to it, which for a large collection can take a long time.
Collections that are not sharded
Sharding is per collection. A database in a sharded cluster can contain both sharded and unsharded collections. Each database has a primary shard where its unsharded collections live by default. Since MongoDB 8.0 you can move an unsharded collection to another shard with sh.moveCollection().
Changing your mind
A shard key used to be permanent. That is no longer true, though every option is a heavy operation on a large collection.
| Need | Command | Available from |
|---|---|---|
| Change a document's shard key value | An ordinary update (not if the key is _id) | MongoDB 4.2 |
| Add fields to the end of the key | refineCollectionShardKey | MongoDB 4.4 |
| Switch to a different key | sh.reshardCollection("shop.orders", { customerId: "hashed" }) | MongoDB 5.0 |
| Stop sharding a collection | sh.unshardCollection("shop.orders", "shardA"), naming the shard that receives the data | MongoDB 8.0 |
Resharding rewrites the collection in the background and needs spare disk space and I/O capacity on the shards while it runs.
Restrictions to plan for
- Unique indexes. MongoDB can enforce uniqueness across shards only for the shard key itself, or for a compound index that has the shard key as its prefix. A unique index on
emailis not possible in a collection sharded oncustomerId. The_idindex is the exception, but then keeping_idunique across shards is the application's job unless_idis the shard key. - Documents without the key. Documents may lack shard key fields; they are placed as if the value were
null, so many of them pile up in one chunk. - Jumbo chunks. When too many documents share one key value, the range cannot be split and the balancer may be unable to move it. It is a symptom of a low-cardinality or skewed key.
- Operations. Backups, restores and upgrades involve every shard plus the config servers and must be coordinated.
Do you need sharding?
Usually not at first. A replica set on good hardware handles a large workload, and it is far simpler to run. Fix slow queries and missing indexes, then scale the machine. Sharding earns its cost when the data no longer fits on one server's storage, write throughput exceeds what one primary can sustain, or data must be kept in particular regions (zones can pin ranges of the shard key to specific shards).
If you use MongoDB Atlas, the managed service deploys and operates the cluster components for you, but choosing the shard key and writing queries that include it remain your job.
Further reading
- Sharding (MongoDB Manual)
- Shard Keys
- Choose a Shard Key
- Sharded Cluster Balancer
Get the weekly commit
New database deep dives every week.
