Erase account data across stores and in-flight work

Application background

A customer closes their document account and asks for their data to be removed. Their documents are stored in the main database, exported files, search results and backups. Deleting one database row does not remove all those copies.

Some work may still be in progress. An export that started earlier could finish after the account closes, or an old import could recreate deleted records. First stop access and new work, then track cleanup of each copy.

Example walkthrough

An erasure workflow is a saved checklist of deletion steps and their results. Its completion report must distinguish removed data from data that remains under a stated retention rule.

Your assignment

Deliver: Build an account-deletion coordinator that stops new access, tracks cleanup in each store and reports both completed deletion and specifically retained data.

Required behavior: DELETE /account creates an erasure case and a durable deletion generation. New reads and writes are denied under that generation. GET /erasure/{case_id} exposes pending, completed and specifically retained categories without leaking erased content.

The required first milestone is a working local implementation of the behavior above. The numbered implementation steps define the scope. The cloud architecture is a later extension, not something the starter has already provisioned.

Get the code and run the supplied example

The code is in the public junior-to-staff repository. Install Git and Python 3.12+. No AWS account or Python packages are required for this first run. If you already have a checkout, use it and skip cloning.

git clone https://github.com/Soulful-Iris/junior-to-staff.git
cd junior-to-staff
python3 examples/architecture-starts/erasure_workflow.py

Supplied file: examples/architecture-starts/erasure_workflow.py. You can also read or download the source here (download file, source below).

"""Local mechanism demonstration for erasure-workflow. No AWS resources are created."""
ledger={'ana':6}; records={'ana':{'version':5,'email':'ana@example.invalid'}}
def import_record(subject,version,payload):
    if version<=ledger.get(subject,-1): return 'rejected by deletion generation'
    records[subject]=payload; return 'imported'
records.pop('ana',None)
print(import_record('ana',4,{'email':'old backup copy'}))
print('Readable account:',records.get('ana','absent'))
print('Deletion evidence:',ledger)

This program is a mechanism demonstration: it runs the small scenario in one process and prints the result. It is not an HTTP service, a complete application, or an AWS deployment. A successful run demonstrates this mechanism only. It does not establish the workload or failure guarantees of the application you will build.

Example output from the supplied run:

Generated IDs and timestamps may differ. Compare the state transitions and outcomes.

rejected by deletion generation
Readable account: absent
Deletion evidence: {'ana': 6}

Set up your implementation workspace

Create work/erasure-workflow/ in your checkout (or use a separate repository). Copy the supplied mechanism into that directory as mechanism.py, then extract its state transitions into functions you can call from your implementation. The record and module names below describe what you must implement. They are not a promise that files with those names already exist. Keep a README.md beside your implementation with its exact run commands and observed results.

Local components and state to implement

This table names the records, interfaces or decision inputs for your deliverable. Unless a name is explicitly linked to supplied source above, it is something you create. Implement the local state transitions first, then connect the HTTP, storage or worker boundaries required by the steps.

Implement the assignment

1. Deny access before cleanup

Commit the deletion generation and disable account access first. Every consumer that can recreate data must check the ledger before writing. Use non-sensitive identifiers in evidence and restrict ledger readers. A tombstone is not permission to keep the entire old profile.

2. Inventory and dispatch deletion

List primary tables, object versions, cache keys, search documents, analytics exports and processors. Store one idempotent step for each owner. A step records concrete evidence, such as affected version ranges or processor confirmation, before completion.

3. Handle backups and holds

Document when immutable backups expire and the procedure that reapplies the deletion ledger before a restore is served. Track a hold at the affected category rather than declaring the whole account exempt. The 30-day figure is a scenario assumption, not legal advice or a compliance guarantee.

4. Finish and explain the case

Expose pending steps, age and named escalation owner. Reconcile storage inventories and running jobs against completed cases. Mark complete only when the configured policy is satisfied and any retained categories are disclosed through the authorized case view.

Demonstrate the completed local result

Handoff: In your implementation README, include the start command, one successful operation, the failure case above and the resulting stored state or decision. State which dependencies are simulated. Someone with a fresh checkout should be able to reproduce this without your chat history.

Workload assumptions and capacity decisions

These are constructed exercise assumptions. The stated workload is a design target. The local demonstration does not establish that throughput. Use the estimation constants to check units before choosing capacity.

Map the local implementation to AWS

Deployment status: local only. Running the supplied command creates no AWS resources and configures no cloud connections. The diagram is a proposed deployment of the completed application. Each box needs either a deployed runtime, a provisioned service or an explicitly external dependency.

Read the diagram by following the arrows from the entry point: application code accepts the request or event, the state owner commits it, and any worker produces the later result. The table ties those roles to code and adapter work. Multiple boxes do not imply multiple Python files already exist.

Step Functions coordinates progress. It does not discover every copy automatically. A separate deletion ledger prevents old workers or backup restoration from reintroducing a deleted subject. Each store adapter must return evidence for its own boundary.

Provision resources, then connect the application

Use one disposable AWS environment for the cloud exercise. Put the named resources in infra/template.yaml or your existing IaC tool, pass resource IDs through configuration, and scope each runtime role to its own tables, buckets and queues. The diagram is a design to implement. It is not a claim that these resources have been deployed. Record the commands you used to deploy and remove the exercise resources.

For concrete provisioning commands, configuration wiring and cleanup, use the AWS foundation guide. It includes a deployable table/queue/object-storage foundation and explains which application and service adapters you still implement.

A provisioned queue or table does not make the local program use it. Configure resource IDs in the deployed runtime, replace the local adapter, and replay the same successful and failing operation against that runtime. Record the deployed commit and observable result, then remove the disposable resources using your infrastructure tool.

Extend the design after the baseline works

Worked follow-up: Keep erased data suppressed while a downstream store is offline

The search index is unavailable for two days. A customer should not reappear in search when it returns merely because its delete task was waiting. Technical completion and an external vendor's assurance also need separate records.

Revised architecture. Follow the changed responsibility and failure path below. This is a design to implement. The supplied local example does not provision these components.

What to implement. Create a deletion manifest listing each store, responsible owner, task identity and evidence state. Write the subject suppression record first and consult it before serving derived results. Retry physical deletion by store, then verify absence after recovery. Store vendor completion receipts as claims with scope and timestamp. Mark insufficient evidence unresolved until the designated policy owner accepts it. SQS can carry tasks and a database can track the manifest, but an empty queue is not proof of erasure.

Walk through the result. Pause search deletion, complete primary deletion and return a stale search hit. The read path must suppress it. Resume the index, finish its task and attach evidence. Show the customer status as access blocked, deletion still in progress until required stores and approvals are accounted for.

A vendor says deletion is complete but cannot provide object-level evidence. Decide which assurance is sufficient for that processor, who accepts it and what the customer sees. Keep technical progress distinct from policy approval.