Check the combined behavior of independently valid changes

The green checks are true about two different trees. The integration failure is caused by their combination, which neither check observed.

Follow-up 1 · Main changes during CI

A third commit lands while the speculative check is running. May the old result be reused?

Follow-up 2 · The suite is flaky

The correct integration test fails 10% of the time due to leaked fixture state. Does a retry establish correctness?

Record the evidence and limitations

Build in three stops: reproduce the small case and baseline failure. Implement the protected boundary. Then replay both changed requirements with captured outputs. Record commands, fixtures, and observed results in your implementation README. A diagram is a prediction until those checks run.

Show four original outcomes and rejection of the exact combined tree. Additional lead scope: Balance throughput, check duration, bypass policy, and flake ownership. Completion demonstrates practice evidence. It does not establish interview readiness or multi-team delivery experience.

Detailed implementation and AI-assisted prompts

You end up having built the failure merge queues exist for — two PRs, each green alone, red together — and the machinery that stops it reaching main.

Build

PR A changes a function's contract. PR B, branched before A landed, adds a new call site written against the old contract, in a different file. No textual conflict. Both green on their own base. Together they break main. Then a merge queue — real or hand-rolled — catches the second one before main ever sees it.

The thought process

Start by naming why CI lied: "this PR is green" actually means "this PR was green against the main that existed when its checks ran". It is a statement about a moment. Main moved, and nobody tested the combination — no tool was wrong. The claim was smaller than everyone assumed.

Second, the construction discipline: the pair must merge cleanly. If git reports a textual conflict you built the wrong failure — that one is caught for free. You want a semantic dependency with no textual overlap, which is why the new call site lives in a different file.

Third, the prevention menu: require-branches-up-to-date serialises humans, who then babysit rebases. Test-after-merge-and-revert is optimistic and lets main go red sometimes, which tiny teams tolerate. A merge queue serialises machines, testing each PR against main plus everything ahead of it and ejecting what fails. Team size times CI duration picks the answer. The queue won because it converts human waiting into machine time.

How to organise the prompts

1. The construction brief.

Describe two changes to this repository that each pass the full suite
on their own branch, merge into main with no textual conflict, and
make the suite fail once both are merged. Name the mechanism. Do not
write code yet.

The mechanism must be a semantic dependency, and the no-conflict claim specific about which files each change touches.

2. The trap, proven.

Build both branches from the same base. Show me four results: the
suite green on A, green on B, both merging cleanly into a scratch
branch, and the suite red there.

The four results are the artefact. A green scratch branch demonstrates nothing — construct again.

3. The prevention.

Enable the merge queue — or write a landing script that refuses to
merge any PR until the suite passes on a temporary merge of that PR
onto current main plus everything ahead of it in line. Replay both
PRs through it and show me where the second one is stopped.

The failure must happen inside the queue, with main's history green throughout the replay.

4. The limits.

List what this queue still cannot catch, with one concrete example
from this repository for each.

At minimum: combinations that only fail at runtime under real data, and a flaky suite, which the queue amplifies into everyone's problem. An answer claiming the queue catches everything is selling, not thinking.

On AWS

A merge queue spends CI capacity on speculative combined trees. Estimate required runs and duration before selecting hosted Actions or CodeBuild for a needed VPC/machine shape. Cache reusable build inputs without reusing a green result for a different source tree. Store immutable images by digest in ECR, and reports/bundles by source SHA in S3. A generic S3 tarball is not an image-registry endpoint. Include storage, transfer and cleanup in the current account estimate instead of assuming a fixed free allowance.

What productionising it means

The queue gets metrics — depth, time-in-queue, ejection rate — because a queue nobody watches is a delay nobody can explain. Flakiness is now existential: one flaky test ejects innocent PRs and stalls every merge behind them, so the quarantine rule from testing stops being optional. And the bypass log from project 3 applies doubly at 6pm on a Friday.

The learning

Green is a statement about a moment, not a property of a change. Integration is a race. The queue removes the race without a person holding a lock — and you know exactly which failure it removes, because you built that failure with your own hands.

How you would know it is wrong

• Git reports a textual conflict between the pair: wrong construction — that case was already caught for free.
• The scratch branch is green: no semantic dependency, nothing demonstrated.
• The queue passes both PRs: check which ref CI ran on. If it re-ran the stale PR head instead of the speculative merge, the queue is ceremony.
• After the ejection, rebase PR B and land it properly. If it still cannot pass, the original failure was something else and you proved less than you think.

Back to the ordered project index