Serve recommendations with safe fallback ranking

Application background

A shopper opens a page of recommended products. The application first gathers possible products, then uses a model to choose an order suited to that shopper. It also checks which products are allowed and in stock.

The model may need recent customer features from another service. If those data arrive too late, the page still needs a useful response. A fallback list of popular products can help, but it must follow the same stock and permission rules.

Example walkthrough

A deadline is the latest time the operation may finish. A fallback is a simpler alternative result used when the preferred calculation cannot finish safely within that time.

Sizing that affects this decision

The 250 ms response target has to include the whole user-visible operation. One candidate budget is 30 ms for candidate retrieval, 50 ms for features, 80 ms for inference, 20 ms for eligibility checks, 30 ms for transport and 40 ms of reserve. These allocations add to 250 ms, but adding separately measured p95 values does not establish an end-to-end p95. A 170 ms feature wait already exceeds its allocation.

These are exercise assumptions. The estimation reference explains the units and approximations. They do not establish the local demo's measured capacity.

Your assignment

Deliver: Build product ranking within a response deadline. Provide a simpler fallback when the model or features are too slow, and apply stock and permission checks to both paths.

Required behavior: Return eligible ranked items within a 250 ms p95 budget. Personalized scoring is optional under deadline pressure. Authorization and availability filtering are mandatory for every path, including cached fallback.

The required first milestone is a working local implementation of the behavior above. The numbered implementation steps define the scope. The cloud architecture is a later extension, not something the starter has already provisioned.

Get the code and run the supplied example

The code is in the public junior-to-staff repository. Install Git and Python 3.12+. No AWS account or Python packages are required for this first run. If you already have a checkout, use it and skip cloning.

git clone https://github.com/Soulful-Iris/junior-to-staff.git
cd junior-to-staff
python3 examples/architecture-starts/personalized_ranking.py

Supplied file: examples/architecture-starts/personalized_ranking.py. You can also read or download the source here (download file, source below).

"""Local mechanism demonstration for personalized-ranking. No AWS resources are created."""
candidates=[{'id':'a','score':9,'eligible':False},{'id':'b','score':5,'eligible':True},{'id':'c','score':3,'eligible':True}]
feature_elapsed_ms=170; remaining_ms=250-feature_elapsed_ms
mode='fallback' if remaining_ms<80+20+30 else 'personalized'
ranked=sorted(candidates,key=lambda x:-x['score'])
print('Mode:',mode,'remaining budget:',remaining_ms)
print('Returned:',[x['id'] for x in ranked if x['eligible']])

This program is a mechanism demonstration: it runs the small scenario in one process and prints the result. It is not an HTTP service, a complete application, or an AWS deployment. A successful run demonstrates this mechanism only. It does not establish the workload or failure guarantees of the application you will build.

Example output from the supplied run:

Generated IDs and timestamps may differ. Compare the state transitions and outcomes.

Mode: fallback remaining budget: 80
Returned: ['b', 'c']

Set up your implementation workspace

Create work/personalized-ranking/ in your checkout (or use a separate repository). Copy the supplied mechanism into that directory as mechanism.py, then extract its state transitions into functions you can call from your implementation. The record and module names below describe what you must implement. They are not a promise that files with those names already exist. Keep a README.md beside your implementation with its exact run commands and observed results.

Local components and state to implement

This table names the records, interfaces or decision inputs for your deliverable. Unless a name is explicitly linked to supplied source above, it is something you create. Implement the local state transitions first, then connect the HTTP, storage or worker boundaries required by the steps.

Implement the assignment

1. Build an eligible baseline

Retrieve a bounded candidate list and return a deterministic popular/recent ordering after current eligibility checks. Measure that baseline’s latency and usefulness. Keep the same eligibility module for personalized and fallback paths.

2. Version feature and model contracts

Define feature names, types, defaults, observation times and training/serving transformations. Record model and feature schema versions together. Missing data is not automatically zero. Choose a documented default or omit personalization.

3. Spend one shared deadline

Pass remaining time to feature retrieval and inference. If the feature path consumes 170 ms, skip expensive scoring when the response reserve is insufficient. Cancel work that no longer contributes to a response and record the fallback reason without making the user wait for a doomed model call.

4. Evaluate the served system

Compare engagement or task success with guardrails for eligibility, latency and cohort effects. Record actual served model/feature versions and fallback frequency. Offline ranking quality alone cannot reveal how often production users receive the fallback.

Demonstrate the completed local result

Handoff: In your implementation README, include the start command, one successful operation, the failure case above and the resulting stored state or decision. State which dependencies are simulated. Someone with a fresh checkout should be able to reproduce this without your chat history.

Workload assumptions and capacity decisions

These are constructed exercise assumptions. The stated workload is a design target. The local demonstration does not establish that throughput. Use the estimation constants to check units before choosing capacity.

Map the local implementation to AWS

Deployment status: local only. Running the supplied command creates no AWS resources and configures no cloud connections. The diagram is a proposed deployment of the completed application. Each box needs either a deployed runtime, a provisioned service or an explicitly external dependency.

Read the diagram by following the arrows from the entry point: application code accepts the request or event, the state owner commits it, and any worker produces the later result. The table ties those roles to code and adapter work. Multiple boxes do not imply multiple Python files already exist.

The inference endpoint ranks candidates. The application owns deadlines and eligibility. Keeping a useful baseline makes feature/inference outages an explicit product behavior instead of a hidden timeout.

Provision resources, then connect the application

Use one disposable AWS environment for the cloud exercise. Put the named resources in infra/template.yaml or your existing IaC tool, pass resource IDs through configuration, and scope each runtime role to its own tables, buckets and queues. The diagram is a design to implement. It is not a claim that these resources have been deployed. Record the commands you used to deploy and remove the exercise resources.

For concrete provisioning commands, configuration wiring and cleanup, use the AWS foundation guide. It includes a deployable table/queue/object-storage foundation and explains which application and service adapters you still implement.

A provisioned queue or table does not make the local program use it. Configure resource IDs in the deployed runtime, replace the local adapter, and replay the same successful and failing operation against that runtime. Record the deployed commit and observable result, then remove the disposable resources using your infrastructure tool.

Extend the design after the baseline works

Worked follow-up: Keep ten thousand ranking candidates out of the serving path

Loading a separate model and feature namespace for every candidate multiplies memory, observations and operational ownership before the candidates have shown value.

Revised architecture. Follow the changed responsibility and failure path below. This is a design to implement. The supplied local example does not provision these components.

What to implement. Use an offline candidate registry containing artifact digest, feature contract, training data identity and evaluation report. Evaluate against fixed and held-out slices, then promote only a bounded shortlist into shadow serving. Shadow results must not change user-visible rankings. A later canary records exposure and guardrails by candidate version. Keep feature access subject to deletion and privacy policy throughout the pipeline.

Walk through the result. With an illustrative 200 MiB artifact per candidate, 10,000 artifacts already require about 1.9 TiB before replicas or features. Show 10,000 registered, ten shadowed and one canaried as an example policy, not a universal ratio. A candidate improving average CTR but harming a language cohort must remain visible in the decision record.

Run 10,000 model variants. Estimate artifact, feature and observation cardinality, then choose whether all variants deserve live infrastructure or whether most can be evaluated offline first.