Post-pilot soundness assessment — September 16, 2026

Recommendation: proceed to the final study after freezing its actual schedule, budgets and readiness evidence. Another paid GPT-5.5 pilot is not indicated by this transcript. Do not dispatch the existing exploratory snapshot as a final study: it predates the optional-cost-cap task, is still gated, and specifies only one medium/max pair.

Evidence from the completed pilot

The pilot remains valid. Rechecking admission and the frozen native snapshot confirms death by an Orc soldier at 100 ft, no Silmarils, and score 0. All 730 native frames replayed exactly; all 206 model requests matched the requested identity and effort. The model explicitly recognized poison, Slowness, adjacent orcs and low-health warnings before dying. No native/interface/grading defect was identified in the observed play.

The navigation problems belong to the agent's controller and interpretation: screen coordinates were treated as persistent positions, exploration oscillated, and manual moves sometimes targeted the wrong square. The guide explicitly documents scrolling and screen coordinates. These affect performance but do not justify correcting the agent's code during an evaluation.

Replay demonstrates recording/recovery consistency, not that the engine is free of deterministic bugs. This single game covered only 100 ft. The existing native escape, jewel-possession, phase-isolation, concealment and failure fixtures remain the evidence for the critical cases the pilot never reached.

Admission gap repaired

admitted_run previously checked the number of reviewed mismatched observations but did not require resolutions for not_seen_as_supported_view. Consequently, a future qualitative review could overlook missing public-view evidence and still pass admission. The current pilot had a complete manual review already; its result was not invalidated by this omission.

Admission now requires a typed coverage record naming every unseen trial/frame exactly once, with an explanation and hash-bound source evidence. Missing, partial, duplicate, wrong-trial, wrong-frame or stale reviews fail closed. The pilot was first confirmed to fail without this binding, then passed with its 40 birth-menu and 199 abbreviated gameplay frames explicitly resolved. The old qualitative review is preserved as qualitative_review.before-coverage-check.json.

Validation: 49 focused tests passed, plus the real pilot's admission recheck. Only operator-side review code changed; no game, observation, prompt, scoring or agent code was changed, and no additional model calls were made.

Before final dispatch

• Freeze a new plan/readiness record against the tested pilot identity. The old research snapshot differs in the two task entrypoints and service image; its read-only preflight correctly refuses dispatch.
• Carry the requested no-dollar-cap policy into the final plan. The superseded frozen exploratory snapshot specifies $150; maintained templates now default to no cap. Cost recording remains on.
• Predeclare the time policy and apply the same schedule/budgets to matched medium/max conditions. The exploratory template allows three games of up to two hours each but only four hours total. That is a shared session budget, not a guarantee that every game can use its full allowance.
• Preserve every registered run. Under the current admission policy a binding time/input limit leaves the condition incomplete; it must not silently become a native death or be dropped while aggregating the remaining runs. A different time-budgeted metric would need to be declared before dispatch.
• Choose the final number of independent sessions prospectively. The prepared one-bundle 2-learning/1-scored medium/max pair is exploratory and cannot support a population saturation or reasoning-effort claim. Report native victory and depth/quest diagnostics separately, with uncertainty across sessions.

The pilot's transcript-search truncations and abbreviated custom views are resolved findings, not outstanding game faults. Long-context cost accounting is documented separately in the pilot report; use its $49.86 session estimate rather than the legacy $37.33 request-wise estimate. No new experiment was launched as part of this assessment.