Run programming submissions inside isolated workers

Application background

A programming contest accepts a participant's source code and language. The judge runs that program against controlled inputs, compares its output with the expected answers and reports a verdict such as accepted, wrong answer or time limit exceeded.

The submitted program belongs to an untrusted user. It might loop forever, create processes or try to read credentials. Running it as an ordinary function inside the API process would give it too much access.

Example walkthrough

Isolation means restricting the program's files, network, privileges and resources outside the API process. The local scheduling demonstration does not itself provide that isolation.

Your assignment

Deliver: Build submission records and the protocol between the API and judging workers. Demonstrate job handling locally, then define the isolated runtime required before executing untrusted code.

Required behavior: POST /submissions durably accepts source and language under a request identity. Execution occurs in an isolated sandbox with CPU, memory, process, output and wall-clock limits. Results identify compiler/runtime versions and hidden-case bundle version.

The required first milestone is a working local implementation of the behavior above. The numbered implementation steps define the scope. The cloud architecture is a later extension, not something the starter has already provisioned.

Get the code and run the supplied example

The code is in the public junior-to-staff repository. Install Git and Python 3.12+. No AWS account or Python packages are required for this first run. If you already have a checkout, use it and skip cloning.

git clone https://github.com/Soulful-Iris/junior-to-staff.git
cd junior-to-staff
python3 examples/architecture-starts/online_judge.py

Supplied file: examples/architecture-starts/online_judge.py. You can also read or download the source here (download file, source below).

"""Local mechanism demonstration for online-judge. No AWS resources are created."""
import subprocess,sys
program="while True: pass"
try:
    subprocess.run([sys.executable,'-c',program],timeout=0.2,capture_output=True)
except subprocess.TimeoutExpired:
    print('wall-clock limit exceeded; child terminated')
print('This subprocess example is NOT a security sandbox.')

This program is a mechanism demonstration: it runs the small scenario in one process and prints the result. It is not an HTTP service, a complete application, or an AWS deployment. A successful run demonstrates this mechanism only. It does not establish the workload or failure guarantees of the application you will build.

Example output from the supplied run:

Generated IDs and timestamps may differ. Compare the state transitions and outcomes.

wall-clock limit exceeded; child terminated
This subprocess example is NOT a security sandbox.

Set up your implementation workspace

Create work/online-judge/ in your checkout (or use a separate repository). Copy the supplied mechanism into that directory as mechanism.py, then extract its state transitions into functions you can call from your implementation. The record and module names below describe what you must implement. They are not a promise that files with those names already exist. Keep a README.md beside your implementation with its exact run commands and observed results.

Local components and state to implement

This table names the records, interfaces or decision inputs for your deliverable. Unless a name is explicitly linked to supplied source above, it is something you create. Implement the local state transitions first, then connect the HTTP, storage or worker boundaries required by the steps.

Implement the assignment

1. Accept before compiling

Validate language and source-size limits, store source privately and commit a submission record plus dispatch intent. Return a status URL. Duplicate client requests return the original submission rather than spending another execution slot.

2. Design the isolation boundary

Use a dedicated sandbox fleet with microVM or equivalent hardened isolation, no instance credentials and no external network access for submitted code. A normal subprocess or ordinary application container is not the promised hostile-code boundary. Keep the broker outside the untrusted execution environment.

3. Run reproducibly with limits

Select a pinned compiler/runtime image and case bundle. Enforce CPU, memory, process count, writable filesystem, output bytes and an independent wall deadline. Destroy the sandbox after use. Never let a prior submission’s files contaminate the next one.

4. Publish bounded evidence

Store verdict, resource usage and safe compiler output. Do not reveal hidden cases in public logs. Conditionally publish the winning run and keep retries linked to the same submission. Separate infrastructure failure from wrong answer or time limit exceeded.

Demonstrate the completed local result

Handoff: In your implementation README, include the start command, one successful operation, the failure case above and the resulting stored state or decision. State which dependencies are simulated. Someone with a fresh checkout should be able to reproduce this without your chat history.

Workload assumptions and capacity decisions

These are constructed exercise assumptions. The stated workload is a design target. The local demonstration does not establish that throughput. Use the estimation constants to check units before choosing capacity.

Map the local implementation to AWS

Deployment status: local only. Running the supplied command creates no AWS resources and configures no cloud connections. The diagram is a proposed deployment of the completed application. Each box needs either a deployed runtime, a provisioned service or an explicitly external dependency.

Read the diagram by following the arrows from the entry point: application code accepts the request or event, the state owner commits it, and any worker produces the later result. The table ties those roles to code and adapter work. Multiple boxes do not imply multiple Python files already exist.

EC2 is the host for an isolation design you must implement. Naming EC2 or Fargate does not itself prove safe execution of hostile code. The local program demonstrates only wall-clock termination.

Provision resources, then connect the application

Use one disposable AWS environment for the cloud exercise. Put the named resources in infra/template.yaml or your existing IaC tool, pass resource IDs through configuration, and scope each runtime role to its own tables, buckets and queues. The diagram is a design to implement. It is not a claim that these resources have been deployed. Record the commands you used to deploy and remove the exercise resources.

For concrete provisioning commands, configuration wiring and cleanup, use the AWS foundation guide. It includes a deployable table/queue/object-storage foundation and explains which application and service adapters you still implement.

A provisioned queue or table does not make the local program use it. Configure resource IDs in the deployed runtime, replace the local adapter, and replay the same successful and failing operation against that runtime. Record the deployed commit and observable result, then remove the disposable resources using your infrastructure tool.

Extend the design after the baseline works

Worked follow-up: Admit a new language runtime without trusting its code

A runtime is executable code inside the worker boundary. A container image label does not demonstrate that the submitted program cannot reach another submission or the host.

Revised architecture. Follow the changed responsibility and failure path below. This is a design to implement. The supplied local example does not provision these components.

What to implement. Create an image admission record with immutable digest, builder provenance, supported language version, owner and retirement date. Run each submission with fresh storage, resource limits, no application credentials and denied outbound access unless the exercise explicitly needs it. Separate the scheduling service from the execution boundary. Choose an isolation mechanism appropriate to hostile code rather than treating ECS placement alone as a sandbox. Keep old result metadata reproducible without leaving retired vulnerable workers available to new jobs.

Walk through the result. Submit a program that attempts to read another job's file, contact an internal address and exceed its time budget. Record the denied capabilities and termination reason from a disposable local exercise. Then revoke one runtime digest and show new submissions refused while historical results still name the original digest.

Allow third-party language runtimes. Define the image admission process, patch ownership, version retirement and how old submissions remain reproducible without keeping vulnerable hosts publicly reachable.