Click Below to Get the Code

Browse, clone, and build from real-world templates powered by Harper.
Blog
GitHub Logo

The Economics of Distributed Data: Replication, Sharding, and Hot Replicas

Sharding can make reads slower, not faster. See when it pays off, how hot replicas change the math, and test your own setup with five interactive models.
Blog

The Economics of Distributed Data: Replication, Sharding, and Hot Replicas

Kris Zyp
SVP of Engineering
at Harper
November 17, 2026
Kris Zyp
SVP of Engineering
at Harper
November 17, 2026
Kris Zyp
SVP of Engineering
at Harper
November 17, 2026
November 17, 2026
Sharding can make reads slower, not faster. See when it pays off, how hot replicas change the math, and test your own setup with five interactive models.
Kris Zyp
SVP of Engineering

If your data lives in one place today, there are three ways to spread it out. They cost very different amounts, and they are not equally fast.

Most systems start with their data in one place: one database, one region. That works until your users are a long way from it, and every request pays for the trip.

The fix is to put copies of the data closer to people, and there are three ways to do it. You can keep a full copy everywhere. You can shard, so each node holds one slice. Or you can shard and let each node keep hot replicas, local copies of the records people read most. This post walks through all three in that order, with a simple model you can adjust, so you can see what each one costs, what it buys, and which one fits the system you have.

A note on the baseline. This analysis started with Harper, where a node typically answers a read from memory or local disk in about a millisecond. That makes it a clean starting point: with server time that small, nearly everything left is network, so the model isolates what distance costs and what a multi-region, sharded cluster can do about it. The reasoning carries over to any data platform. If yours takes longer to answer a read, add that time on top.

Latency is distance

Once a request reaches a node that has the record in memory or on local disk, serving it is the quick part. Almost everything else a user waits for is the network path between them and that node.

A useful rule of thumb is about 0.014 ms of round trip per kilometer, plus a small fixed cost per hop. New York to London works out to roughly 70 ms. No query optimizer gets that back. The only way to shorten the path is to put a node closer to the user.

So for read-heavy applications, the question "how do I get lower latency" mostly reduces to "how many regions can I afford to be in."

Latency is distance

Start with all your data in one region, then add more. Every node here holds every record, so the only thing changing is how far a request travels.

NodeUsers, sized by population

Illustrative model. A read costs 10 ms of last-mile access, the round trip to the nearest node (0.014 ms per km plus a fixed hop), and 1 ms on the server. Each new node goes where it cuts the most user latency.

With everything in one region, the average read in this model takes 106 ms. Four regions bring it down to 38 ms, and eleven to 17 ms. The first few regions do most of the work, but they do not finish it. The problem is paying for it.

Cost is node size times node count

The simplest way to add a region is to put a full copy of the data there. Every node can answer every read, and there is nothing to tune. If every node holds a full copy of the dataset, every node has to be sized for all of it: disk for the data, memory for the working set, and enough write capacity to receive every write in the cluster.

That ties together two things you would rather keep separate. Say you have 200 GB of data and can pay for 800 GB of node capacity in total. That is four full copies, in four regions, and that is the end of it. A fifth region means paying for another 200 GB. Wider distribution costs proportionally more, and a node can never be smaller than the whole dataset.

Sharding alone makes reads worse

Sharding breaks that link. Each node owns a slice of the data, so it can be smaller, and the same budget buys more nodes in more regions.

The catch is obvious once you say it out loud. A nearby node does not help if it does not have the record you asked for. Split the data three ways and only a third of reads are served by the node that received them. The rest are relayed to an owner somewhere else, which adds a second network round trip on top of the first.

In the model below, a node that owns a third of the data needs about 73 GB, so the same 800 GB of capacity buys ten nodes instead of four. The average read goes from 38 ms to 62 ms. Same money, more than twice as many regions, slower application.

Hot replicas recover the locality

Real read traffic is not uniform. A small share of records gets most of the reads: the popular products, the trending content, the active sessions. This is usually modeled with a Zipf distribution, and for typical web and catalog traffic the skew factor is close to 1.0.

That skew is what makes the whole approach work. If every node keeps its owned slice plus local copies of the most-read records from the other shards, a small replica budget covers a large share of reads. At a skew of 1.0, replicas of the hottest 2% of records cover about 78% of reads.

Two properties keep these replicas cheap:

  • Nobody tracks them. Ownership is deterministic (a hash of the primary key picks the shard), so every node knows where the authoritative copy lives. A replica is a cached copy on a non-owner. The cluster does not record where replicas exist, so creating one never turns a read into a cluster-wide write.
  • They follow access. A replica is created when a non-owner fetches a record, is invalidated when the record changes, and expires when it stops being read. Cold records leave on their own.

Run the same ten-node cluster with hot replicas and 85% of reads are served locally. The average read drops to 28 ms, about a quarter lower than the four-node cluster, for the same spend.

The tool below runs this for your own situation. Tell it how much data you have, how much hardware you can run, how concentrated your reads are, and where your users are, and it recommends a layout:

Should you shard? Check your own setup

Describe your data, your hardware and your traffic. The model compares full copies with sharded layouts on the same total hardware and recommends one.

Recommended setup

Illustrative model, not a benchmark. Every layout uses the same total capacity, and cost is assumed to follow capacity. The smallest node holds 16 GB. The model tries 2 to 8 shards, requires at least two owners per shard, and prefers fewer shards when results are close. Sharding is only recommended when it cuts the average read by at least 10% and 5 ms. Non-owners keep a small index entry for every record (5% of its size) and receive a small invalidation for every write. Hot replicas are local copies of the most-read 2% of records. A read costs 10 ms of last-mile access, the round trip to the nearest node, and 1 ms on the server. A miss adds a round trip to the nearest owner. Bars are on a fixed scale of 0 to 100 ms.

Everything depends on read skew

Here is the part I did not expect to be so sharp. The benefit is extremely sensitive to how skewed your reads are.

With the default settings, the four-node baseline averages 38 ms. At a skew of 1.0, where the most-read 2% of records get 78% of reads, the sharded cluster averages 28 ms. At 0.8, where they get 44%, it averages 43 ms. Somewhere in between, the sharded cluster crosses over and becomes worse than the cluster it replaced. A small change in the access pattern moves a three-shard design from a clear win to a net loss.

Find where you sit

Two of the three approaches from the tool above. The flat line is full copies everywhere. The sloped line is sharded + hot replicas on the same hardware, which gets faster as more of your reads go to a few popular records. Move the dot to your own numbers.

Sharded + hot replicas (best layout)Full copies everywhereYou

Illustrative model, not a benchmark. Users are around the world. The sharded line uses the best of 2 to 8 shards at each point, with at least two owners per shard and the same total capacity as the full copies. The vertical axis is the average read on a fixed scale; lower is better.

The practical advice: measure your read distribution before you choose a shard count. Take a day of access logs, rank records by read count, and check what share of reads the top 1% and 5% receive. That one number tells you more about whether sharding will help than any amount of capacity planning.

Where it helps less

This design favors applications where the latency of the first response is the product: e-commerce, catalogs, content, API caching. It helps less in a few cases.

  • Real-time workloads. Long-lived connections and streams of small messages gain little from hot replicas. They are also easier to scale on demand, because there is less stationary data to move.
  • Uniform access. If every record is equally likely to be read, no replica budget short of a full copy keeps reads local.
  • Too few owners. Smaller nodes do not have to mean less availability, but each shard needs at least two owners in different places. A budget that buys fewer than two owners per shard is buying an outage.

A quick way to decide

Four questions, in order. The first "no" is usually your answer.

  1. Are your users spread across continents? If they sit in one region, a few full copies already put a node next to almost everyone. There is little distance left to remove, and sharding for latency will not pay.
  2. Do a few records get most of your reads? Check what share of reads goes to your most-read 2% of records. In this model, with hardware for four full copies, sharding starts to pay at around 70%. Well below that, stay with full copies.
  3. Can you afford two owners for every shard? If the hardware only stretches to one, you are trading availability for latency. Use fewer shards or do not shard.
  4. Is the speed of the first response what your users notice? Catalogs, content and cached API responses benefit. Streams and long-lived connections mostly do not.

Adding capacity without moving data

The classic objection to auto-scaling a database is that compute scales easily and data does not. Adding a node means copying data to it, and copying data at the moment you are short on capacity makes things worse.

A sharded cluster with replicas has another option: a node that owns nothing. It joins with index entries only, takes requests immediately, and fills its replica set as records are asked for. For the first few minutes it relays most reads. As it warms up, it absorbs more and more of the read traffic without a bulk copy and without a load spike on the owners.

That does not make a database scale in seconds. It does shrink the lead time from hours to minutes, which is enough for a planned sale, a holiday, or a forecast that says the next hour will be busy.

How long until a new node helps?

Adding a node means copying data to it first. How much it has to copy depends on what kind of node it is.

Illustrative model. Time is 3 minutes to start the node plus the copy at the transfer rate. Index entries are 5% of a record. A replica-only node starts with no hot records and gets faster as reads arrive, so its first minutes relay most requests. The time axis is logarithmic and fixed, from 1 minute to 24 hours.

It is also worth being precise about what people mean by "auto-scaling." In most conversations I have had, the real requirement is assurance that the system survives peak traffic at a price closer to average traffic. Scaling on demand is one way to get that. Shared headroom is another. When several workloads share hosts, their peaks rarely line up. Across Harper's own fleet, the combined peak is about 63% of what you get by adding up every customer's individual peak. That gap is headroom nobody has to pay for separately. Many small nodes pack onto shared hosts far better than a few large ones, and spare capacity that is already running is available instantly.

One more lever: who sends each write

Every node that stores a write has to receive it once, so a cluster of N nodes makes N − 1 sends per write however it is wired. What the wiring, or network topology, decides is which links carry those sends.

In a full mesh, the node that takes a write sends it directly to every other node, including one transfer across the ocean for every node on the other side. Full mesh is the common default. An alternative design is a grid: each node connects to the other nodes in its region and to the nodes that own the same shard in other regions. A write then crosses between regions once per remote region and fans out locally from there.

Who sends each write?

Every node has to receive every write once. What changes is which links carry it. Pick a wiring, then click any node to write there and follow the copy.

Node taking the writeCopy sent between regionsCopy sent inside a regionGateway

Illustrative. In the grid, each row is one shard: a node connects to the other nodes in its region and to the nodes holding the same shard in other regions. With region gateways, two nodes per region carry all traffic between regions. Whether fewer cross-region copies saves money depends on how your provider prices transfer between regions compared with inside one.

Whether that saves money depends on your provider's pricing for transfer between regions versus inside one. Where the two cost the same, the gain is fewer connections and less CPU. Where cross-region transfer is priced several times higher, it shows up on the bill.

This analysis came out of building sharding and hot replicas into Harper. If you want to see how that looks in practice, the Harper 5.4 announcement covers what shipped.

What to do next

If your data sits in one place today, here is a sensible order to work in.

  1. Find out where your users are. Break your traffic down by region. If nearly everyone is close to your current system, distance is not your problem and none of this is urgent.
  2. Start with full copies. Put a complete copy in the two or three regions with the most users. It is the simplest layout to run, every node can answer every read, and it gives you redundancy you probably want anyway.
  3. Measure how concentrated your reads are. Take a day of access logs, rank records by read count, and work out what share of all reads goes to the most-read 2%. You will need this number before you consider sharding.
  4. Watch your capacity ratio. Add up the capacity of every node and divide by the size of your data. As your data grows, or as you want more regions than you can afford full copies for, that ratio falls. That is the point to open the tool above and see what it recommends.
  5. If you shard, take your time choosing the layout. Use the fewest shards that give you the gain, keep two owners for every shard, and turn on hot replicas from the start. Shard layouts are hard to change once data is in them, so this is the decision worth a second look.

Want a second opinion on your numbers?

The model in this post is simple on purpose. Real systems have write-heavy tables, uneven traffic between regions, and data that has to stay in one place. If you are planning the move from one central system to many regions and would like to work through it with someone who builds this for a living, tell us what you are building and an expert at Harper will go through it with you.

Talk to an expert

If your data lives in one place today, there are three ways to spread it out. They cost very different amounts, and they are not equally fast.

Most systems start with their data in one place: one database, one region. That works until your users are a long way from it, and every request pays for the trip.

The fix is to put copies of the data closer to people, and there are three ways to do it. You can keep a full copy everywhere. You can shard, so each node holds one slice. Or you can shard and let each node keep hot replicas, local copies of the records people read most. This post walks through all three in that order, with a simple model you can adjust, so you can see what each one costs, what it buys, and which one fits the system you have.

A note on the baseline. This analysis started with Harper, where a node typically answers a read from memory or local disk in about a millisecond. That makes it a clean starting point: with server time that small, nearly everything left is network, so the model isolates what distance costs and what a multi-region, sharded cluster can do about it. The reasoning carries over to any data platform. If yours takes longer to answer a read, add that time on top.

Latency is distance

Once a request reaches a node that has the record in memory or on local disk, serving it is the quick part. Almost everything else a user waits for is the network path between them and that node.

A useful rule of thumb is about 0.014 ms of round trip per kilometer, plus a small fixed cost per hop. New York to London works out to roughly 70 ms. No query optimizer gets that back. The only way to shorten the path is to put a node closer to the user.

So for read-heavy applications, the question "how do I get lower latency" mostly reduces to "how many regions can I afford to be in."

Latency is distance

Start with all your data in one region, then add more. Every node here holds every record, so the only thing changing is how far a request travels.

NodeUsers, sized by population

Illustrative model. A read costs 10 ms of last-mile access, the round trip to the nearest node (0.014 ms per km plus a fixed hop), and 1 ms on the server. Each new node goes where it cuts the most user latency.

With everything in one region, the average read in this model takes 106 ms. Four regions bring it down to 38 ms, and eleven to 17 ms. The first few regions do most of the work, but they do not finish it. The problem is paying for it.

Cost is node size times node count

The simplest way to add a region is to put a full copy of the data there. Every node can answer every read, and there is nothing to tune. If every node holds a full copy of the dataset, every node has to be sized for all of it: disk for the data, memory for the working set, and enough write capacity to receive every write in the cluster.

That ties together two things you would rather keep separate. Say you have 200 GB of data and can pay for 800 GB of node capacity in total. That is four full copies, in four regions, and that is the end of it. A fifth region means paying for another 200 GB. Wider distribution costs proportionally more, and a node can never be smaller than the whole dataset.

Sharding alone makes reads worse

Sharding breaks that link. Each node owns a slice of the data, so it can be smaller, and the same budget buys more nodes in more regions.

The catch is obvious once you say it out loud. A nearby node does not help if it does not have the record you asked for. Split the data three ways and only a third of reads are served by the node that received them. The rest are relayed to an owner somewhere else, which adds a second network round trip on top of the first.

In the model below, a node that owns a third of the data needs about 73 GB, so the same 800 GB of capacity buys ten nodes instead of four. The average read goes from 38 ms to 62 ms. Same money, more than twice as many regions, slower application.

Hot replicas recover the locality

Real read traffic is not uniform. A small share of records gets most of the reads: the popular products, the trending content, the active sessions. This is usually modeled with a Zipf distribution, and for typical web and catalog traffic the skew factor is close to 1.0.

That skew is what makes the whole approach work. If every node keeps its owned slice plus local copies of the most-read records from the other shards, a small replica budget covers a large share of reads. At a skew of 1.0, replicas of the hottest 2% of records cover about 78% of reads.

Two properties keep these replicas cheap:

  • Nobody tracks them. Ownership is deterministic (a hash of the primary key picks the shard), so every node knows where the authoritative copy lives. A replica is a cached copy on a non-owner. The cluster does not record where replicas exist, so creating one never turns a read into a cluster-wide write.
  • They follow access. A replica is created when a non-owner fetches a record, is invalidated when the record changes, and expires when it stops being read. Cold records leave on their own.

Run the same ten-node cluster with hot replicas and 85% of reads are served locally. The average read drops to 28 ms, about a quarter lower than the four-node cluster, for the same spend.

The tool below runs this for your own situation. Tell it how much data you have, how much hardware you can run, how concentrated your reads are, and where your users are, and it recommends a layout:

Should you shard? Check your own setup

Describe your data, your hardware and your traffic. The model compares full copies with sharded layouts on the same total hardware and recommends one.

Recommended setup

Illustrative model, not a benchmark. Every layout uses the same total capacity, and cost is assumed to follow capacity. The smallest node holds 16 GB. The model tries 2 to 8 shards, requires at least two owners per shard, and prefers fewer shards when results are close. Sharding is only recommended when it cuts the average read by at least 10% and 5 ms. Non-owners keep a small index entry for every record (5% of its size) and receive a small invalidation for every write. Hot replicas are local copies of the most-read 2% of records. A read costs 10 ms of last-mile access, the round trip to the nearest node, and 1 ms on the server. A miss adds a round trip to the nearest owner. Bars are on a fixed scale of 0 to 100 ms.

Everything depends on read skew

Here is the part I did not expect to be so sharp. The benefit is extremely sensitive to how skewed your reads are.

With the default settings, the four-node baseline averages 38 ms. At a skew of 1.0, where the most-read 2% of records get 78% of reads, the sharded cluster averages 28 ms. At 0.8, where they get 44%, it averages 43 ms. Somewhere in between, the sharded cluster crosses over and becomes worse than the cluster it replaced. A small change in the access pattern moves a three-shard design from a clear win to a net loss.

Find where you sit

Two of the three approaches from the tool above. The flat line is full copies everywhere. The sloped line is sharded + hot replicas on the same hardware, which gets faster as more of your reads go to a few popular records. Move the dot to your own numbers.

Sharded + hot replicas (best layout)Full copies everywhereYou

Illustrative model, not a benchmark. Users are around the world. The sharded line uses the best of 2 to 8 shards at each point, with at least two owners per shard and the same total capacity as the full copies. The vertical axis is the average read on a fixed scale; lower is better.

The practical advice: measure your read distribution before you choose a shard count. Take a day of access logs, rank records by read count, and check what share of reads the top 1% and 5% receive. That one number tells you more about whether sharding will help than any amount of capacity planning.

Where it helps less

This design favors applications where the latency of the first response is the product: e-commerce, catalogs, content, API caching. It helps less in a few cases.

  • Real-time workloads. Long-lived connections and streams of small messages gain little from hot replicas. They are also easier to scale on demand, because there is less stationary data to move.
  • Uniform access. If every record is equally likely to be read, no replica budget short of a full copy keeps reads local.
  • Too few owners. Smaller nodes do not have to mean less availability, but each shard needs at least two owners in different places. A budget that buys fewer than two owners per shard is buying an outage.

A quick way to decide

Four questions, in order. The first "no" is usually your answer.

  1. Are your users spread across continents? If they sit in one region, a few full copies already put a node next to almost everyone. There is little distance left to remove, and sharding for latency will not pay.
  2. Do a few records get most of your reads? Check what share of reads goes to your most-read 2% of records. In this model, with hardware for four full copies, sharding starts to pay at around 70%. Well below that, stay with full copies.
  3. Can you afford two owners for every shard? If the hardware only stretches to one, you are trading availability for latency. Use fewer shards or do not shard.
  4. Is the speed of the first response what your users notice? Catalogs, content and cached API responses benefit. Streams and long-lived connections mostly do not.

Adding capacity without moving data

The classic objection to auto-scaling a database is that compute scales easily and data does not. Adding a node means copying data to it, and copying data at the moment you are short on capacity makes things worse.

A sharded cluster with replicas has another option: a node that owns nothing. It joins with index entries only, takes requests immediately, and fills its replica set as records are asked for. For the first few minutes it relays most reads. As it warms up, it absorbs more and more of the read traffic without a bulk copy and without a load spike on the owners.

That does not make a database scale in seconds. It does shrink the lead time from hours to minutes, which is enough for a planned sale, a holiday, or a forecast that says the next hour will be busy.

How long until a new node helps?

Adding a node means copying data to it first. How much it has to copy depends on what kind of node it is.

Illustrative model. Time is 3 minutes to start the node plus the copy at the transfer rate. Index entries are 5% of a record. A replica-only node starts with no hot records and gets faster as reads arrive, so its first minutes relay most requests. The time axis is logarithmic and fixed, from 1 minute to 24 hours.

It is also worth being precise about what people mean by "auto-scaling." In most conversations I have had, the real requirement is assurance that the system survives peak traffic at a price closer to average traffic. Scaling on demand is one way to get that. Shared headroom is another. When several workloads share hosts, their peaks rarely line up. Across Harper's own fleet, the combined peak is about 63% of what you get by adding up every customer's individual peak. That gap is headroom nobody has to pay for separately. Many small nodes pack onto shared hosts far better than a few large ones, and spare capacity that is already running is available instantly.

One more lever: who sends each write

Every node that stores a write has to receive it once, so a cluster of N nodes makes N − 1 sends per write however it is wired. What the wiring, or network topology, decides is which links carry those sends.

In a full mesh, the node that takes a write sends it directly to every other node, including one transfer across the ocean for every node on the other side. Full mesh is the common default. An alternative design is a grid: each node connects to the other nodes in its region and to the nodes that own the same shard in other regions. A write then crosses between regions once per remote region and fans out locally from there.

Who sends each write?

Every node has to receive every write once. What changes is which links carry it. Pick a wiring, then click any node to write there and follow the copy.

Node taking the writeCopy sent between regionsCopy sent inside a regionGateway

Illustrative. In the grid, each row is one shard: a node connects to the other nodes in its region and to the nodes holding the same shard in other regions. With region gateways, two nodes per region carry all traffic between regions. Whether fewer cross-region copies saves money depends on how your provider prices transfer between regions compared with inside one.

Whether that saves money depends on your provider's pricing for transfer between regions versus inside one. Where the two cost the same, the gain is fewer connections and less CPU. Where cross-region transfer is priced several times higher, it shows up on the bill.

This analysis came out of building sharding and hot replicas into Harper. If you want to see how that looks in practice, the Harper 5.4 announcement covers what shipped.

What to do next

If your data sits in one place today, here is a sensible order to work in.

  1. Find out where your users are. Break your traffic down by region. If nearly everyone is close to your current system, distance is not your problem and none of this is urgent.
  2. Start with full copies. Put a complete copy in the two or three regions with the most users. It is the simplest layout to run, every node can answer every read, and it gives you redundancy you probably want anyway.
  3. Measure how concentrated your reads are. Take a day of access logs, rank records by read count, and work out what share of all reads goes to the most-read 2%. You will need this number before you consider sharding.
  4. Watch your capacity ratio. Add up the capacity of every node and divide by the size of your data. As your data grows, or as you want more regions than you can afford full copies for, that ratio falls. That is the point to open the tool above and see what it recommends.
  5. If you shard, take your time choosing the layout. Use the fewest shards that give you the gain, keep two owners for every shard, and turn on hot replicas from the start. Shard layouts are hard to change once data is in them, so this is the decision worth a second look.

Want a second opinion on your numbers?

The model in this post is simple on purpose. Real systems have write-heavy tables, uneven traffic between regions, and data that has to stay in one place. If you are planning the move from one central system to many regions and would like to work through it with someone who builds this for a living, tell us what you are building and an expert at Harper will go through it with you.

Talk to an expert

Sharding can make reads slower, not faster. See when it pays off, how hot replicas change the math, and test your own setup with five interactive models.

Download

White arrow pointing right
Sharding can make reads slower, not faster. See when it pays off, how hot replicas change the math, and test your own setup with five interactive models.

Download

White arrow pointing right
Sharding can make reads slower, not faster. See when it pays off, how hot replicas change the math, and test your own setup with five interactive models.

Download

White arrow pointing right

Explore Recent Resources