Stage 5: Migrate the reading list to stable tag IDs

Application background

Your reading list now includes a UI, title workers, exports and optional AI tag suggestions. It still stores tags as words. You want permanent tag IDs so a display name can change without changing which tag a bookmark refers to.

People continue editing and deleting bookmarks while you convert existing data. An old conversion result must not undo a newer edit or bring back a deleted bookmark. Every component that reads tags must understand the transition.

A backfill result that is already outdated

This is an illustrative timeline of one bookmark, not a log captured from the starter:

12:00 backfill reads bookmark 41 at version 7
12:01 user deletes bookmark 41, creating deletion version 8
12:02 backfill tries to save converted version 7

The final write must not recreate the bookmark. Compare versions and preserve the newer deletion when applying the converted data.

Backfill is the conversion of existing records. Reconciliation compares old and new representations to find missing or incorrect changes before retiring the old one.

Your assignment

Deliver: Migrate the running application to stable tag IDs. Protect newer edits and deletions during conversion, keep supported consumers working and demonstrate when old-format support can be removed.

Required behavior: Versioned apply preserves the newest source state including deletion. One write authority exists at a time. Supported old contracts are adapted until retirement, and the irreversible rollback boundary is recorded before incompatible writes begin.

The required first milestone is a working local implementation of the behavior above. The numbered implementation steps define the scope. The cloud architecture is a later extension, not something the starter has already provisioned.

Get the code and run the supplied example

The code is in the public junior-to-staff repository. Install Git and Python 3.12+. No AWS account or Python packages are required for this first run. If you already have a checkout, use it and skip cloning.

git clone https://github.com/Soulful-Iris/junior-to-staff.git
cd junior-to-staff
python3 examples/architecture-starts/reading_list_it_changes.py

Supplied file: examples/architecture-starts/reading_list_it_changes.py. You can also read or download the source here (download file, source below).

"""Local mechanism demonstration for reading-list-it-changes. No AWS resources are created."""
target={'version':0,'tags':[],'deleted':False}
for version,tags,deleted in [(5,['tag-db'],False),(6,[],True),(4,['databases'],False)]:
    if version>target['version']:
        target.update(version=version,tags=tags,deleted=deleted); outcome='applied'
    else: outcome='ignored stale'
    print(version,outcome)
print('Final:',target)

This program is a mechanism demonstration: it runs the small scenario in one process and prints the result. It is not an HTTP service, a complete application, or an AWS deployment. A successful run demonstrates this mechanism only. It does not establish the workload or failure guarantees of the application you will build.

Example output from the supplied run:

Generated IDs and timestamps may differ. Compare the state transitions and outcomes.

5 applied
6 applied
4 ignored stale
Final: {'version': 6, 'tags': [], 'deleted': True}

Run the application you will extend

The reading-list API setup guide gives you a real local HTTP server, SQLite database, save/list/edit requests and controlled title success/timeout behavior. Start it in one terminal and send the documented curl requests from another. Read that setup before following the implementation steps below. The demo above isolates this lesson's mechanism. The server is where you integrate it.

Continue the application you built in the preceding stage. The supplied server is only a Stage 1 starting point. It does not contain the previous stages' completed UI, operations, jobs or AI feature.

For a first run, start this in terminal 1 from the repository root:

python3 examples/reading-list-starter/app.py --db /tmp/reading-list.sqlite3

In terminal 2, save one bookmark with a controlled title timeout:

curl -i http://127.0.0.1:8080/bookmarks \
  -H 'X-Demo-User: alice' -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com/docs","title_mode":"timeout"}'

Expect 201 Created, a bookmark id and title_status: "timeout". The URL is persisted despite the title failure. This is the supplied baseline. The assignment adds the behavior described above. The lookup is a fixture, so no external website is contacted. For members Bob or Ben in a scenario, use the starter's second demo identity bob. Alice or Ana corresponds to alice.

Work in your own branch or copy examples/reading-list-starter/ to work/05-it-changes/. app.py exists in that directory. Add the modules named below there as you separate HTTP, storage and background work. The server has demo membership, not production authentication.

Local components and state to implement

This table names the records, interfaces or decision inputs for your deliverable. Unless a name is explicitly linked to supplied source above, it is something you create. Implement the local state transitions first, then connect the HTTP, storage or worker boundaries required by the steps.

Implement the assignment

1. Inventory the entire working system

List browser request/response fields, store methods, queued payloads, exports, prompt inputs and model-output validators. Keep one canonical tag mapping and adapt old contracts deliberately. The previous stage’s optional model does not get an exception from schema compatibility.

2. Backfill with versions and tombstones

Capture a source boundary, apply live changes without a gap and checkpoint completed ranges. Use the same conditional apply function for snapshot rows and subsequent updates/deletes. Replaying an old range must not recreate deleted links.

3. Reconcile and switch authority

Compare canonical source/target state at an aligned watermark. Fence old writes, catch up the final change prefix and move the writer epoch/routing state. Keep authorized reads and personal read markers consistent through the transition.

4. Retire and hand over

Observe actual old-consumer usage, adapt or drain old queue messages and record the removal conditions. Demonstrate the supported rollback before the incompatible-write boundary and the forward-repair procedure after it. Finish with run commands, data/infra ownership and the measured limits from all five stages.

Demonstrate the completed local result

Handoff: In your implementation README, include the start command, one successful operation, the failure case above and the resulting stored state or decision. State which dependencies are simulated. Someone with a fresh checkout should be able to reproduce this without your chat history.

Workload assumptions and capacity decisions

These are constructed exercise assumptions. The stated workload is a design target. The local demonstration does not establish that throughput. Use the estimation constants to check units before choosing capacity.

Map the local implementation to AWS

Deployment status: local only. Running the supplied command creates no AWS resources and configures no cloud connections. The diagram is a proposed deployment of the completed application. Each box needs either a deployed runtime, a provisioned service or an explicitly external dependency.

Read the diagram by following the arrows from the entry point: application code accepts the request or event, the state owner commits it, and any worker produces the later result. The table ties those roles to code and adapter work. Multiple boxes do not imply multiple Python files already exist.

This stage changes the same system built in the earlier stages. The migration succeeds when its consumers and operating procedures agree on the new state, not when a copy counter reaches 100%.

Provision resources, then connect the application

Use one disposable AWS environment for the cloud exercise. Put the named resources in infra/template.yaml or your existing IaC tool, pass resource IDs through configuration, and scope each runtime role to its own tables, buckets and queues. The diagram is a design to implement. It is not a claim that these resources have been deployed. Record the commands you used to deploy and remove the exercise resources.

For concrete provisioning commands, configuration wiring and cleanup, use the AWS foundation guide. It includes a deployable table/queue/object-storage foundation and explains which application and service adapters you still implement.

A provisioned queue or table does not make the local program use it. Configure resource IDs in the deployed runtime, replace the local adapter, and replay the same successful and failing operation against that runtime. Record the deployed commit and observable result, then remove the disposable resources using your infrastructure tool.

Extend the design after the baseline works

Choose the next change from observed user demand or operating limits. Carry forward the same discipline: concrete scenario, measurable contract, explicit state authority, runnable path and honest recovery evidence.