Protect a database with versioned cache fills

Application background

A shop's product page repeatedly asks for the same product description and price. To reduce database work, the application keeps a reusable copy in a cache. If the copy is missing or too old, the application reads the database again.

Now a popular product expires from the cache just as thousands of shoppers arrive. They may all ask the database for the same data. Meanwhile, a slow read of the old price might finish after a newer price has already been cached.

The ordering rule in the supplied code

The local model's fill() function contains this guard:

current=cache.get(key)
if current and current['version']>version: return 'stale fill rejected'

If version 5 is already cached, an older fetch returning version 4 cannot replace it. This snippet is from a single-process model. A shared cache needs an atomic compare-and-update operation, because two processes can otherwise pass a separate check before either writes.

The database is the origin of product truth. A cache fill is the operation that loads a copy from that origin. Coordinate fills without turning a cache failure into unlimited database work.

Sizing that affects this decision

The assumed two million reads/s cannot all fall through to a database budgeted for 10,000 reads/s. Even 99% cache hits would leave about 20,000 database reads/s. Under this simplified model, staying within 10,000 requires at least 99.5% hits, fewer admitted reads or more origin capacity.

These are exercise assumptions. The estimation reference explains the units and approximations. They do not establish the local demo's measured capacity.

Your assignment

Deliver: Build controlled cache reads and fills. Prevent a burst of misses from overwhelming the database and prevent an older fill from replacing a newer value.

Required behavior: GET cache entries returns a versioned value or a miss. The database remains authoritative. Define maximum acceptable staleness per field. Price-sensitive checkout still reads its own authority.

The required first milestone is a working local implementation of the behavior above. The numbered implementation steps define the scope. The cloud architecture is a later extension, not something the starter has already provisioned.

Get the code and run the supplied example

The code is in the public junior-to-staff repository. Install Git and Python 3.12+. No AWS account or Python packages are required for this first run. If you already have a checkout, use it and skip cloning.

git clone https://github.com/Soulful-Iris/junior-to-staff.git
cd junior-to-staff
python3 examples/architecture-starts/distributed_cache.py

Supplied file: examples/architecture-starts/distributed_cache.py. You can also read or download the source here (download file, source below).

"""Local mechanism demonstration for distributed-cache. No AWS resources are created."""
cache={'item7':{'version':5,'value':'new price'}}
def fill(key,version,value):
    current=cache.get(key)
    if current and current['version']>version: return 'stale fill rejected'
    cache[key]={'version':version,'value':value}; return 'filled'
print(fill('item7',4,'old price')); print(cache)
requests=1000; per_process_coalesced=1; processes=10
print('One hot miss:',requests,'requests can still cause',processes*per_process_coalesced,'origin reads across processes')

This program is a mechanism demonstration: it runs the small scenario in one process and prints the result. It is not an HTTP service, a complete application, or an AWS deployment. A successful run demonstrates this mechanism only. It does not establish the workload or failure guarantees of the application you will build.

Example output from the supplied run:

Generated IDs and timestamps may differ. Compare the state transitions and outcomes.

stale fill rejected
{'item7': {'version': 5, 'value': 'new price'}}
One hot miss: 1000 requests can still cause 10 origin reads across processes

Set up your implementation workspace

Create work/distributed-cache/ in your checkout (or use a separate repository). Copy the supplied mechanism into that directory as mechanism.py, then extract its state transitions into functions you can call from your implementation. The record and module names below describe what you must implement. They are not a promise that files with those names already exist. Keep a README.md beside your implementation with its exact run commands and observed results.

Local components and state to implement

This table names the records, interfaces or decision inputs for your deliverable. Unless a name is explicitly linked to supplied source above, it is something you create. Implement the local state transitions first, then connect the HTTP, storage or worker boundaries required by the steps.

Implement the assignment

1. Implement versioned cache-aside

Read cache, fetch source on miss, and fill with source version and expiry. Protect fills against a newer cached version or invalidation generation. Explain the race where an old source read finishes after a fresh update. Deleting a key alone may not prevent stale refill.

2. Coalesce and bound misses

Add per-key singleflight inside each process, a bounded origin semaphore and short wait deadline. Process-local coalescing does not coordinate 200 nodes. For expensive hot keys, use a distributed refresh owner only with an expiry/fencing rule and a safe fallback.

3. Handle node membership changes

Use a stable partition map or consistent hashing with replicas and controlled rebalance. Move data gradually while limiting origin refill traffic. Replication improves availability but consumes memory and does not create authority for business writes.

4. Operate explicit degradation

Choose bounded stale serving for eligible fields, rejection for unsafe fields and randomized expiry to spread refreshes. Track hit rate by request volume, origin concurrency, refresh age and hot-key concentration.

Demonstrate the completed local result

Handoff: In your implementation README, include the start command, one successful operation, the failure case above and the resulting stored state or decision. State which dependencies are simulated. Someone with a fresh checkout should be able to reproduce this without your chat history.

Workload assumptions and capacity decisions

These are constructed exercise assumptions. The stated workload is a design target. The local demonstration does not establish that throughput. Use the estimation constants to check units before choosing capacity.

Map the local implementation to AWS

Deployment status: local only. Running the supplied command creates no AWS resources and configures no cloud connections. The diagram is a proposed deployment of the completed application. Each box needs either a deployed runtime, a provisioned service or an explicitly external dependency.

Read the diagram by following the arrows from the entry point: application code accepts the request or event, the state owner commits it, and any worker produces the later result. The table ties those roles to code and adapter work. Multiple boxes do not imply multiple Python files already exist.

ElastiCache supplies a cache service. The application supplies source-version handling, miss coalescing and origin protection. A custom cache project can implement those interfaces locally before attempting a new distributed storage engine.

Provision resources, then connect the application

Use one disposable AWS environment for the cloud exercise. Put the named resources in infra/template.yaml or your existing IaC tool, pass resource IDs through configuration, and scope each runtime role to its own tables, buckets and queues. The diagram is a design to implement. It is not a claim that these resources have been deployed. Record the commands you used to deploy and remove the exercise resources.

For concrete provisioning commands, configuration wiring and cleanup, use the AWS foundation guide. It includes a deployable table/queue/object-storage foundation and explains which application and service adapters you still implement.

A provisioned queue or table does not make the local program use it. Configure resource IDs in the deployed runtime, replace the local adapter, and replay the same successful and failing operation against that runtime. Record the deployed commit and observable result, then remove the disposable resources using your infrastructure tool.

Extend the design after the baseline works

Worked follow-up: Cache missing products without hiding a new product

A read misses product P at generation 4. Another request creates P at generation 5. If the old read finishes later and fills not-found, new readers can incorrectly see absence.

Revised architecture. Follow the changed responsibility and failure path below. This is a design to implement. The supplied local example does not provision these components.

What to implement. Represent absence as a versioned cache value, not a bare null. Maintain a source generation for the key and use an atomic compare-and-fill operation that rejects a result older than the known invalidation generation. Product creation advances that generation and publishes invalidation. Define a short negative-cache freshness bound for cases where invalidation is delayed. With Redis, the compare and fill must be one server-side atomic operation, not a Python read followed by a write.

Walk through the result. Pause the generation-4 missing read. Create P at generation 5 and apply its invalidation. Resume the old fill and show it rejected. Repeat with invalidation delayed and report the maximum absence window your contract actually permits. Add the miss-storm admission limit so dropping negative entries cannot flood the source.

Add negative caching for missing products. Define how a newly created product invalidates an older not-found result and how the version/generation rule applies to absence.