An API can look healthy until request volume rises, the database becomes the bottleneck, and latency starts to spike. Queries that were inexpensive under normal traffic begin piling up, connection pools fill, and every request is still making a trip back to the same database. If the system is also handling frequent writes, the database has even more work to absorb.
Most experienced architecture teams already know the usual responses. Scale the database vertically. Add read replicas. Put Redis or another caching layer between the application and the database. These are all reasonable options, and for many workloads they are exactly the right ones.
But caching introduces another architectural question.
Does solving database load require adding another distributed system to the request path?
The usual cache architecture solves a real problem
A shared cache can dramatically reduce database reads. The application checks the cache first, falls back to the database on a miss, and stores the result so the next request does not repeat the query. It is a proven pattern, particularly when several application instances need access to the same cached state.
The trade-off is that the application now depends on another service. There is another network hop, another connection to manage, another system to scale and monitor, and another consistency relationship between the cache and the database.
Those costs may be justified. The point is that they should be justified by the workload rather than assumed to be necessary.
What if the application could remove repeated database work without adding a separate cache tier at all?
Testing another architecture with a search workload
We built a "House Listings Reference Application" on Harper to explore that question. link
The workload is intentionally familiar: structured listing data, multi-field filtering, sorting, pagination, repeated searches, and associated images. It looks a lot like a simplified product catalog, marketplace, or other read-heavy API.
The architectural difference is that Harper runs the data, application logic, and result cache within the same runtime. The Listing schema defines the structured data and its API surface, while a custom SearchListings resource handles the application-specific search behavior.
On a cache miss, SearchListings scans the listing records, applies the requested filters, sorts the matches, selects the page, and caches the result. The response reports how much work occurred, including rows scanned, cache status, filtering time, and total server-side execution time.
The full scan is deliberate. This is not meant to demonstrate large-scale indexed search. It gives us a controlled amount of work so we can clearly see what happens when subsequent requests no longer have to perform it.
A cache hit removes the expensive path
The cache is a worker-local in-memory map with a 30-second TTL. Because it lives inside the runtime handling the request, serving a cached result does not require contacting another service.
Each query is normalized into a cache key that includes the filters, sorting, limit, and pagination offset. That allows repeated requests to reuse a result without mixing different queries or pages.
In this reference workload, an uncached search took roughly 5–6 milliseconds of server time. The same query served from the in-process cache took roughly 0.03–0.05 milliseconds.
The 100x difference is interesting, but the mechanism matters more. Harper did not make the database operation 100x faster. On the cached request, that work was no longer in the request path.
That becomes more meaningful as traffic increases. Under concurrent cold-cache traffic in the test, p95 server-side latency rose to roughly 100–130 milliseconds. With the relevant queries cached, the warm path remained inexpensive server-side.
The architectural question is bigger than caching
The same application stores roughly 42 MB of listing images as Harper blobs, separate from the records being searched. Search requests operate on less than 2 MB of structured data and only retrieve image bytes when the browser requests a specific photo.
That points to the broader thesis. The database, application logic, cache, API resources, and blob handling can operate as parts of the same application platform without forcing every workload through the same execution path.
A distributed cache still makes sense when cached state must be shared across independent services or instances, when it needs to scale separately, or when the application requires coordinated invalidation. This demo's cache is local to a worker, expires after 30 seconds, and does not implement mutation-driven invalidation.
But if database load is driving latency and caching is already on the table, there is another question worth asking before adding another tier:
Can the architecture remove the database work and the network boundary at the same time?
That is the alternative this application is designed to make concrete.
.webp)
.jpg)
%20(1).png){kind=icon}
.png){kind=icon}
.png){kind=icon}