Sharding for Beginners: What It Is, Why It Matters, and How It Works

Image Credit:

Unsplash

In this article, we’ll be going to learn what sharding means in the context of databases, why we need it, and what are its pros and cons.

Sharding means breaking down a big dataset into small disjoint subsets. Now, how you break it down is more of an implementation detail. You can decide it using what is called the sharding key.

But why break down a dataset? What benefit will it give?

Let’s say you are a software developer who wants to send out a newsletter about what is going on with the tech world to people. In order to send those letters, you will want to store interested people’s names and email addresses.

A dataset being split into multiple smaller subsets.

Let’s say you opt-in for a SQL database to store their data, and you decide to run the SQL server on your own computer. You had, let’s say, 100 GB of space available, and you are confident that 100 GB is more than enough.

People started subscribing to your newsletter, and whenever someone subscribed, their information gets stored in your DB. You are one of the happiest people alive because so many people are…