Assessor · importer diagnosis and review
Give candidate material, then use the repair key. Candidate-visible regressions are not the held-back test set. Run the additional fixtures (download file, source below) only after the candidate has repaired its baseline:
Read the supplied code · heldback_importer.py
"""Assessor-only variants. Select candidate package after its baseline is repaired."""
import argparse
import importlib
from pathlib import Path
import sys
import tempfile
import unittest
parser = argparse.ArgumentParser()
parser.add_argument("--package", choices=["reference", "starter"], default="reference")
args, remaining = parser.parse_known_args()
root = Path(__file__).resolve().parents[2] / "curriculum/02-applications/04-testing/labs/importer"
sys.path.insert(0, str(root))
run = importlib.import_module(args.package + ".importer").run
Store = importlib.import_module(args.package + ".store").Store
transport = importlib.import_module(args.package + ".transport")
def item(ident="held", amount="0.29"):
return {"id":ident, "amount":amount, "currency":"USD"}
def page(items, cursor=None):
return {"status":200, "body":{"items":items, "nextCursor":cursor}}
class HeldBack(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.path = str(Path(self.temp.name) / "held.db")
self.store = Store(self.path)
self.clock = transport.FakeClock()
def tearDown(self):
self.store.close()
self.temp.cleanup()
def test_multi_hop_cycle(self):
server = transport.FakeServer({"START":[page([],"p2")],"p2":[page([],"p3")],"p3":[page([],"p2")]},self.clock)
with self.assertRaisesRegex(ValueError,"cycle"):
run(server,self.clock,self.store)
self.assertEqual(len(server.calls),3)
def test_retry_consumes_transport_budget(self):
server = transport.FakeServer({"START":[{"status":429,"retryAfter":0.8,"elapsed":0.3}]},self.clock)
with self.assertRaises(TimeoutError):
run(server,self.clock,self.store,budget=1)
self.assertEqual(len(server.calls),1)
self.assertEqual(self.clock.waits,[])
def test_crash_and_equal_decimal_spelling(self):
server = transport.FakeServer({"START":[page([item(amount="0.290")])]},self.clock)
def crash():
raise RuntimeError("lost process")
with self.assertRaises(RuntimeError):
run(server,self.clock,self.store,after_persist=crash)
self.store.close()
self.store = Store(self.path)
server = transport.FakeServer({"START":[page([item(amount="0.29")])]},self.clock)
self.assertEqual(run(server,self.clock,self.store),[("held",29,"USD")])
def test_late_malformed_page_preserves_progress(self):
server = transport.FakeServer({"START":[page([item("first")],"next")],
"next":[page([item("valid"),item("bad","0.00001")])]},self.clock)
with self.assertRaises(ValueError):
run(server,self.clock,self.store)
self.assertEqual(self.store.rows(),[("first",29,"USD")])
self.assertEqual(self.store.checkpoint(),("next",0))
if __name__ == "__main__":
unittest.main(argv=[sys.argv[0]] + remaining)
python practice/assessor/heldback_importer.py --package starter
| Release | Held-back evidence | Expected result |
|---|---|---|
| Core executes | p2 → p3 → p2, not just a self-loop |
Reject cycle after 3 requests |
| 30 min | 429 at t=0.3, Retry-After 0.8, deadline t=1 | Refuse retry; no second request |
| 45 min | Crash after durable page, then reopen DB; equal decimal spellings 0.290 and 0.29 |
Exact normalized replay; no duplicated cents |
| 55 min | Late malformed row following a valid new row | Entire page rejected; previous checkpoint retained |
| 60 min | Three PRs | Block conflict suppression and early retry; approve bounded non-payload metadata with check |
| Evidence | Weak: 0–1 | Adequate: 2 | Strong: 3 |
|---|---|---|---|
| Debug method | Rewrites before reproducing; blames API for syntax failure | Distinguishes build, logic and integration boundaries | Uses minimal counterexamples to reject plausible competing explanations |
| Correctness | Floats/truncates cents, skips pages, hides conflicts | Exact money, bounded pagination/retry, safe restart | Tests multi-hop cycles, late page conflict rollback and normalized replays |
| Testing | Green reference tests presented as own work | Candidate code passes stated cases plus own regression | Hidden fixture survives without code rewrite; failures are explained precisely |
| Lead review | Blocks all PRs or bikesheds style | Prioritizes data correctness and assigns partner contract owner | Narrow rollback/migration plan with monitoring, ownership and completion criteria |
Failing approaches and their exact regressions are in the repair key. Do not reward an enormous rewrite that accidentally passes without explaining checkpoint order. For a strong result the candidate must state that fake transport timeouts only check the transport interface; they do not prove cancellation of a real HTTP client.
Debrief: domain identity, importer method, runtime/budgets. Next occasion use inventory units and a cursor that encodes shard+offset; keep baseline requirements unseen. See fully scored examples.