Spaces:
Sleeping
Sleeping
docs: whole-suite no-cheat pass summary + engine footguns; regenerate catalog
Browse filesAll 21 active packs redesigned to the no-cheat bar (#1-#21 cadence
complete). Records the recurring defect classes eliminated, the
predicates used to enforce each capability, and the two engine
footguns surfaced for follow-up (power_surplus_gte inert, deploy
unimplemented). Catalog regenerated to the new YAMLs.
- SCENARIO_QUALITY.md +49 -0
- docs/scenarios.html +0 -0
SCENARIO_QUALITY.md
CHANGED
|
@@ -496,6 +496,55 @@ A pack no longer needs a static `.oramap` + static enemy actors:
|
|
| 496 |
can't see fogged enemy ids or command the enemy player; the bench
|
| 497 |
surface (`openra_bench/botgen.py`) validates the name at compile.
|
| 498 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 499 |
Note: `capture_actor` is deliberately absent until the engine gains a
|
| 500 |
Capture order (pending S8 / task #11) — the bench never advertises a
|
| 501 |
tool the engine can't execute (1:1 parity).
|
|
|
|
| 496 |
can't see fogged enemy ids or command the enemy player; the bench
|
| 497 |
surface (`openra_bench/botgen.py`) validates the name at compile.
|
| 498 |
|
| 499 |
+
## Whole-suite no-cheat pass (#1–#21, all 21 active packs)
|
| 500 |
+
|
| 501 |
+
Every active pack has been individually closer-looked + redesigned to
|
| 502 |
+
the **"no defect, no cheat"** bar: a scripted brute / stall / greedy /
|
| 503 |
+
shortest-path / sweep / wrong-route policy must LOSE on every level +
|
| 504 |
+
every hard seed; the intended capability policy must WIN; non-win is a
|
| 505 |
+
real reachable timeout LOSS, not a draw. Engine smoke clean across
|
| 506 |
+
seeds 1–4 each level (no panics, in-bounds, `map_supported`); hard
|
| 507 |
+
tiers in `tests/test_hard_tier.py::UPGRADED` keep ≥2 distinct seed-
|
| 508 |
+
driven spawns; reasoned `NOT_APPLICABLE` exceptions documented in the
|
| 509 |
+
test. Suite **414 passed, 1 skipped, 0 failed** on the consolidated
|
| 510 |
+
`main` after all 21 redesigns landed. Author-side artifacts: the
|
| 511 |
+
SCENARIO_REVIEW_CHECKLIST.md is the methodology; per-scenario commit
|
| 512 |
+
messages on `main` document each capability, the cheat path that was
|
| 513 |
+
closed, and the scripted before/after evidence.
|
| 514 |
+
|
| 515 |
+
Recurring defect classes the pass eliminated:
|
| 516 |
+
- **Inert deadlines** (`within_ticks` ≫ tick reachable at `max_turns`,
|
| 517 |
+
i.e. > `93+90·(max_turns-1)`) ⇒ draw degeneracy. Every level now
|
| 518 |
+
satisfies `within_ticks` AND `after_ticks` ≤ that bound.
|
| 519 |
+
- **Missing `fail_condition`** ⇒ non-win = draw. Every level now has
|
| 520 |
+
an `any_of` fail covering at least the timeout (plus force-loss /
|
| 521 |
+
attrition where applicable).
|
| 522 |
+
- **Capability not enforced by the win predicate** (the laziest play
|
| 523 |
+
satisfies it without doing the advertised skill). Fixed with the
|
| 524 |
+
right predicate per pack: `units_in_region_gte` (split / main body),
|
| 525 |
+
`waypoint_sequence` (ordered / latched), `units_lost_lte` (bait
|
| 526 |
+
budget), `buildings_discovered_gte` (fog-dependent discovery),
|
| 527 |
+
`enemy_key_buildings_destroyed` (faithful "destroy fact+proc"),
|
| 528 |
+
exact-cost `starting_cash` (binding allocation), etc.
|
| 529 |
+
- **Off-map actors** (engine panic). Every level's actors verified in
|
| 530 |
+
playable bounds across seeds 1–4.
|
| 531 |
+
- **Engine auto-`done` on enemy-elimination** terminating before the
|
| 532 |
+
intended win/fail evaluated → drew. Mitigation: an unarmed high-HP
|
| 533 |
+
enemy `fact` marker at the objective ensures the episode runs to a
|
| 534 |
+
real win or timeout LOSS.
|
| 535 |
+
|
| 536 |
+
Engine footguns surfaced during the pass (flagged for follow-up):
|
| 537 |
+
- **`power_surplus_gte` is currently inert**: the obs reports
|
| 538 |
+
`power_provided/power_drained = 0` even with placed `powr`, so the
|
| 539 |
+
predicate is effectively `0 ≥ N` and never fires meaningfully.
|
| 540 |
+
Affected packs (economy-investment, tech-production-planning, etc.)
|
| 541 |
+
intentionally do NOT rely on it as the sole discriminator.
|
| 542 |
+
- **`deploy` is not executed** by the installed wheel: an MCV reaches
|
| 543 |
+
the target cell but never becomes a Construction Yard. `building-
|
| 544 |
+
and-planning` hard was redesigned around build-radius creep instead.
|
| 545 |
+
- Both are tracked as engine-side follow-ups (separate from the
|
| 546 |
+
pre-existing pending task #11 sabotage/specials).
|
| 547 |
+
|
| 548 |
Note: `capture_actor` is deliberately absent until the engine gains a
|
| 549 |
Capture order (pending S8 / task #11) — the bench never advertises a
|
| 550 |
tool the engine can't execute (1:1 parity).
|
docs/scenarios.html
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|