yxc20098 commited on
Commit
ca00016
·
1 Parent(s): 6d918ec

docs: whole-suite no-cheat pass summary + engine footguns; regenerate catalog

Browse files

All 21 active packs redesigned to the no-cheat bar (#1-#21 cadence
complete). Records the recurring defect classes eliminated, the
predicates used to enforce each capability, and the two engine
footguns surfaced for follow-up (power_surplus_gte inert, deploy
unimplemented). Catalog regenerated to the new YAMLs.

Files changed (2) hide show
  1. SCENARIO_QUALITY.md +49 -0
  2. docs/scenarios.html +0 -0
SCENARIO_QUALITY.md CHANGED
@@ -496,6 +496,55 @@ A pack no longer needs a static `.oramap` + static enemy actors:
496
  can't see fogged enemy ids or command the enemy player; the bench
497
  surface (`openra_bench/botgen.py`) validates the name at compile.
498
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
499
  Note: `capture_actor` is deliberately absent until the engine gains a
500
  Capture order (pending S8 / task #11) — the bench never advertises a
501
  tool the engine can't execute (1:1 parity).
 
496
  can't see fogged enemy ids or command the enemy player; the bench
497
  surface (`openra_bench/botgen.py`) validates the name at compile.
498
 
499
+ ## Whole-suite no-cheat pass (#1–#21, all 21 active packs)
500
+
501
+ Every active pack has been individually closer-looked + redesigned to
502
+ the **"no defect, no cheat"** bar: a scripted brute / stall / greedy /
503
+ shortest-path / sweep / wrong-route policy must LOSE on every level +
504
+ every hard seed; the intended capability policy must WIN; non-win is a
505
+ real reachable timeout LOSS, not a draw. Engine smoke clean across
506
+ seeds 1–4 each level (no panics, in-bounds, `map_supported`); hard
507
+ tiers in `tests/test_hard_tier.py::UPGRADED` keep ≥2 distinct seed-
508
+ driven spawns; reasoned `NOT_APPLICABLE` exceptions documented in the
509
+ test. Suite **414 passed, 1 skipped, 0 failed** on the consolidated
510
+ `main` after all 21 redesigns landed. Author-side artifacts: the
511
+ SCENARIO_REVIEW_CHECKLIST.md is the methodology; per-scenario commit
512
+ messages on `main` document each capability, the cheat path that was
513
+ closed, and the scripted before/after evidence.
514
+
515
+ Recurring defect classes the pass eliminated:
516
+ - **Inert deadlines** (`within_ticks` ≫ tick reachable at `max_turns`,
517
+ i.e. > `93+90·(max_turns-1)`) ⇒ draw degeneracy. Every level now
518
+ satisfies `within_ticks` AND `after_ticks` ≤ that bound.
519
+ - **Missing `fail_condition`** ⇒ non-win = draw. Every level now has
520
+ an `any_of` fail covering at least the timeout (plus force-loss /
521
+ attrition where applicable).
522
+ - **Capability not enforced by the win predicate** (the laziest play
523
+ satisfies it without doing the advertised skill). Fixed with the
524
+ right predicate per pack: `units_in_region_gte` (split / main body),
525
+ `waypoint_sequence` (ordered / latched), `units_lost_lte` (bait
526
+ budget), `buildings_discovered_gte` (fog-dependent discovery),
527
+ `enemy_key_buildings_destroyed` (faithful "destroy fact+proc"),
528
+ exact-cost `starting_cash` (binding allocation), etc.
529
+ - **Off-map actors** (engine panic). Every level's actors verified in
530
+ playable bounds across seeds 1–4.
531
+ - **Engine auto-`done` on enemy-elimination** terminating before the
532
+ intended win/fail evaluated → drew. Mitigation: an unarmed high-HP
533
+ enemy `fact` marker at the objective ensures the episode runs to a
534
+ real win or timeout LOSS.
535
+
536
+ Engine footguns surfaced during the pass (flagged for follow-up):
537
+ - **`power_surplus_gte` is currently inert**: the obs reports
538
+ `power_provided/power_drained = 0` even with placed `powr`, so the
539
+ predicate is effectively `0 ≥ N` and never fires meaningfully.
540
+ Affected packs (economy-investment, tech-production-planning, etc.)
541
+ intentionally do NOT rely on it as the sole discriminator.
542
+ - **`deploy` is not executed** by the installed wheel: an MCV reaches
543
+ the target cell but never becomes a Construction Yard. `building-
544
+ and-planning` hard was redesigned around build-radius creep instead.
545
+ - Both are tracked as engine-side follow-ups (separate from the
546
+ pre-existing pending task #11 sabotage/specials).
547
+
548
  Note: `capture_actor` is deliberately absent until the engine gains a
549
  Capture order (pending S8 / task #11) — the bench never advertises a
550
  tool the engine can't execute (1:1 parity).
docs/scenarios.html CHANGED
The diff for this file is too large to render. See raw diff