mapv10 Visual-Regression Scenarios

Deterministic camera-pose + mode + layer-state setups that the viewer can drive on demand. Each scenario produces the same screenshot every run, so a regression in the renderer shows up as a pixel diff against a known-good baseline — not as a vague "looks different" judgment from a validator agent.

TL;DR — running scenarios

In the dev console (or via mcp__Claude_in_Chrome__javascript_tool):

// List every scenario id defined in mapv10_scenarios.json
window.__mapv10ListScenarios();

// Run one — returns a Promise<Mapv10ScenarioResult>
window.__mapv10RunScenario("mapv10_continent_political_oblique").then(console.log);

// Geometry probe (used by the geometry validator + by Codex)
window.__mapv10TerrainGeometryProbe();
window.__mapv10TerrainSelectionProbe(); // Wave 3 per-role × per-metric SSE state

// Direct hooks if you want to drive things by hand
window.__mapv10SetCameraPose([1200, -1300, 1500], [1200, 900, 0], 50);
window.__mapv10WaitForTilesSettled(8000);
window.__mapv10WaitForFrame();
window.__mapv10Ready;          // { loaded, runId, errors }
window.__mapv10DebugState();   // full renderer telemetry

// Zoom/performance trace hooks for manual camera drives
window.__mapv10StartZoomTrace({ label: "manual-zoom", maxSamples: 5000 });
window.__mapv10MarkZoomTrace("before-close-zoom");
window.__mapv10StopZoomTrace(); // returns Mapv10ZoomTraceReport

From the viewer package, with the dev server already running:

npm run scenarios -- --url http://100.97.188.110:5443/?manifest=/mapv10/runs/continent-lod6/manifest.json
npm run scenarios -- --only location_geography --out-dir verification/scenarios/location-geography

The CLI waits for __mapv10Ready.loaded === true before each scenario, applies any scenario-local networkThrottle, runs each scenario in an isolated page, saves one screenshot per scenario, and writes scenario-results.json with:

• scenario runner duration (result.durationMs)
• observed wall-clock scenario duration
• viewer load time
• frame telemetry (frameP50Ms, frameP95Ms, frameP99Ms, smoothed frame time, long tasks)
• terrain/runtime cache counts, tile residency, visible route counts, visible label counts
• geometry and terrain-shading probes
• terrain-selection probe (primaryZ, the two documented SSE targets, active/underlay/preload tile counts, and the per-role × per-metric primaryGeometric/primaryRaster/underlayGeometric/underlayRaster max + 95th-percentile streams — Wave 3 replaced the aggregated maxSsePx/p95SsePx numbers because they could not be compared apples-to-apples against either documented target)
• interaction log entries for scenarios that drive renderer-owned hover, selection, empty-click, pointer-leave, or Escape clear commands
• optional per-frame zoom traces for scenarios that define zoomTrace
• optional performance-budget verdicts for scenarios that define performanceBudget

When a scenario has zoomTrace, the CLI also writes:

• <index>-<scenario-id>-zoom-trace.json: per-frame camera, frame-work, LOD, scheduler, cache, route, label, tile, frame-budget, render, lifecycle, and long-task samples.
• a compact metrics.zoomTrace summary inside scenario-results.json, including p50/p95/p99/max frame work, slow-frame counts and consecutive overruns, zoom-band transitions, cache hit rate, max pending work, render calls/triangles, route batch/build/upload proxies, label churn, lifecycle create/dispose counters, fallback/omitted slots, and the worst frame.

When a scenario has performanceBudget, the CLI writes performanceBudgetIssues and performanceBudgetWarnings per result. Fail issues make npm run scenarios exit non-zero; warnings are reported without flipping the exit code. Budgets are intentionally scenario-local so close overlays, clean terrain, overview routes, and future mobile/zoom cases can carry different limits.

Result shape (Mapv10ScenarioResult):

{
  id: "mapv10_continent_political_oblique",
  durationMs: 19967,
  tilesSettled: true,
  finalCameraState: {
    positionScene: [x, y, z],
    targetScene: [x, y, z]
  },
  geometryProbe: {
    activeMeshCount: 13,
    worldZRangeKm: 11.84,        // Z range across active terrain meshes (km)
    hasDisplacement: true        // false ⇒ flat-plane terrain bug
  },
  terrainSelectionProbe: {
    // Wave 3 LOD/Data-Coherence Foundation: dual targets + per-role
    // per-metric SSE telemetry. Values below are a realisable
    // continent-oblique example: 1500 km altitude over centre,
    // primaryZ=2, primary raster SSE just above the documented 6 px
    // target because z=2 raster sample footprint at this clearance
    // projects to ~7.5 px (geometric tracks the much tighter
    // height-displacement metric and stays well under 1.5 px).
    primaryZ: 2,
    geometricTargetSsePx: 1.5,
    rasterTargetSsePx: 6.0,
    primaryTileCount: 33,
    underlayTileCount: 17,
    preloadTileCount: 0,
    primaryGeometricSsePxMax: 0.93,
    primaryGeometricSsePxP95: 0.78,
    primaryRasterSsePxMax: 7.48,
    primaryRasterSsePxP95: 6.21,
    underlayGeometricSsePxMax: 1.62,
    underlayGeometricSsePxP95: 1.41,
    underlayRasterSsePxMax: 14.6,
    underlayRasterSsePxP95: 12.7
  },
  interactionLog: [
    {
      name: "select-location",
      action: "selectSample",
      payload: { state: "selected", target: { kind: "location", id: "..." } },
      debugState: { selectionMaskSelectedCount: 18, cursor: "pointer" }
    }
  ],
  newErrors: []                  // entries pushed to __mapv10Ready.errors during the scenario
}

The historical aggregated maxSsePx / p95SsePx fields (and their targetSsePx companion) were removed in Wave 3. They collapsed primary AND underlay tiles into one stream with max(geometric, raster), so a scenario's maxSsePx was usually dominated by the structural-parent underlay's raster footprint (one z coarser → ~2× the raster sample at any altitude) and could not be compared apples-to-apples against either documented target. Consumers must now explicitly choose which role and which metric they are gating against.

File layout

examples/map/mapv10/viewer/src/scenarios/
  mapv10_scenarios.json      ← THE SOURCE OF TRUTH. Edit this to add/remove scenarios.
  scenarioTypes.ts           ← TypeScript types for the JSON shape.
  scenarioRunner.ts          ← Driver: setMode → setLayerState → setCameraPose → settle → scripted interactions → frame.

Scenario file shape

{
  "schemaVersion": 1,
  "notes": "Free-form authoring notes for humans.",
  "scenarios": [ /* Mapv10Scenario[] */ ]
}

One scenario, fully specified

{
  "id": "mapv10_continent_political_oblique",
  "description": "Oblique view of the continent in political mode. Camera 1500 km up, 2200 km south of centre, looking north-northeast at 60° pitch.",
  "camera": {
    "positionKm": [1200, -1300, 1500],
    "targetKm":   [1200,   900,    0],
    "fovDeg": 50
  },
  "mode": "political",
  "layers": {
    "terrain": true, "water": true, "borders": false,
    "routes": false, "labels": false, "markers": false
  },
  "settleTimeoutMs": 10000,
  "expectations": [
    "Skyline silhouette is jagged (mountains visible against sky)",
    "Political coloring tracks the displaced terrain surface, not a flat plane"
  ],
  "failConditions": [
    { "type": "tilesSettled" },
    { "type": "noPageCrash" },
    { "type": "noNewReadyErrors" },
    { "type": "forbidZ0UnderlayAtPrimaryZ", "minPrimaryZ": 3 },
    { "type": "ancestorZDistanceMax", "max": 1 }
  ],
  "notes": [
    "Flat horizon line — terrain mesh is not displaced by heightmap",
    "Political colors drape on a flat plane while horizon is straight"
  ]
}

failConditions gate vocabulary

Every failConditions entry is a discriminated object the Node CLI evaluates with viewer/scripts/scenario-gates.mjs. Contract tests for the evaluator live at viewer/scripts/__tests__/scenario-gates.test.mjs.

Gate type	Evaluated against	Fails when
tilesSettled	result.tilesSettled	result is not true after the settle window.
noPageCrash	Playwright pageerror count	any pageerror was recorded during the scenario.
noNewReadyErrors	result.newErrors	any entry was appended to __mapv10Ready.errors during the scenario.
primaryGeometricSsePx {max}	terrainSelectionProbe.primaryGeometricSsePxMax	observed > max. Wave 3: gates the geometric (height-displacement) SSE on primary tiles only against the documented 1.5 px target.
primaryRasterSsePx {max}	terrainSelectionProbe.primaryRasterSsePxMax	observed > max. Wave 3: gates the raster/detail sidecar sample SSE on primary tiles only against the documented 6 px target. MZ makes close scenarios use generated z6/z7 products plus closeDetailNormal; do not raise these gates back to the old z5-only calibration.
underlayGeometricSsePx {max}	terrainSelectionProbe.underlayGeometricSsePxMax	observed > max. Wave 3: gates geometric SSE on underlay tiles only. The structural-parent's geometric error is mechanically ~2× the primary's; gate bounds should reflect that.
underlayRasterSsePx {max}	terrainSelectionProbe.underlayRasterSsePxMax	observed > max. Wave 3: gates raster SSE on underlay tiles only.
underlayTileCount {max}	terrainSelectionProbe.underlayTileCount	observed > max.
primaryTileCountMin {min}	terrainSelectionProbe.primaryTileCount	observed < min.
forbidZ0UnderlayAtPrimaryZ {minPrimaryZ}	debugState.retainedByLod.z0 while terrainSelectionProbe.primaryZ >= minPrimaryZ	a z=0 catch-all underlay appears at deep zoom; legitimate one-level z=0 underlays at primaryZ < minPrimaryZ are allowed.
ancestorZDistanceMax {max}	terrainSelectionProbe.ancestorDistanceMax	observed > max (default AAA target = 1).
sidecarReadyForActiveMeshes {required?}	result.sidecarReadyByMesh	any active primary terrain mesh is missing a required sidecar (default: all seven — materialWeights, materialWeightsB, biome, waterMask, borderSdf, normalRoughness, closeDetailNormal).
semanticPolicyLoaded	debugState.semanticDisplay.loaded	the generated semantic display policy was not loaded by the viewer.
semanticBandEquals {band}	debugState.semanticDisplay.zoomBand	current semantic zoom band differs from the expected band. Close-detail scenarios use band: "close" and must not infer display state from physical lodBand alone.
primaryZAtLeast {min}	terrainSelectionProbe.primaryZ	observed primary physical z is below min; MZ close scenarios require z6+ residency.
closeDetailScaleMin {min}	active primary tile metadata	any active primary tile reports closeDetailScale < min; MZ close scenarios require generated detail scale 8.
labelTextUniqueness {minRatio}	result.renderedLabelTexts	unique / total < minRatio. PASS-on-empty.
labelTextMinDistinctCount {min}	result.renderedLabelTexts	new Set(texts).size < min (catches "labels invisible" cases too).
labelMaxDuplicatesForText {max}	result.renderedLabelTexts	any single text appears more than max times.
interactionPayloadKind {interaction, kind, state?}	result.interactionLog	named interaction is missing, has no payload target, has the wrong target kind, or has the wrong state.
inspectorFieldPresent {interaction, fields}	result.interactionLog[].payload.inspector.fields	any requested inspector field key is missing.
selectionMaskSelectedCount {interaction, min?, max?}	interaction debug snapshot	selected-mask count falls outside the requested range after the named interaction.
hoverPayloadClearable {interaction, clearReason?}	named interaction payload	hover clear payload did not clear target or has the wrong clear reason.
selectionClearable {interaction, clearReason?}	named interaction payload	selection clear payload did not clear target or has the wrong clear reason.
labelEntityEmphasized {interaction, state?}	interaction debug snapshot	no label for the interacted entity is emphasized in hover or selected state.
cursorFeedback {interaction, cursor?}	interaction debug snapshot and payload cursor	cursor feedback is missing or differs from the expected cursor.
influenceInspectorFieldsPresent {interaction, fields?}	named interaction inspector	required influence fields are absent. Defaults cover active type, intensity, mask, source/rule ids, effective product keys, and preserve-base marker.

Authorial intent must be explicit in every scenario: tilesSettled, noPageCrash, and noNewReadyErrors are listed verbatim in each scenario rather than implicitly injected by the CLI. The Wave 1 verifier (viewer/scripts/verify-browser.mjs) inverts the matching z=0 underlay assertion: at primaryZ >= 2 the verifier requires retainedByLod.z0 === 0. Constant: minZForUnderlayLevel = 2.

Field semantics

Field	Meaning
id	MUST start with mapv10_. Used as the lookup key, in screenshots, and in validator reports.
description	One-paragraph human-readable description. Visible in validator output.
camera.positionKm	World km, [x, y, z]. World x/y are the map plane; z is altitude.
camera.targetKm	World km look-at point.
camera.fovDeg	Optional. Omit to keep current FOV.
mode	Map mode id (geography / political / routes / influence / height / slope / normal).
layers	Optional. Partial MapLayerState. Missing keys keep current state.
interactions	Optional scripted interaction commands run after camera/layer setup and tile settle. Actions include hoverSample, selectSample, clearHover, clearSelection, emptyClick, and keyEscape; sample actions ask the renderer for a product-backed target of a requested kind.
zoomTrace	Optional scripted camera path. The runner traces from zoomTrace.startCamera to camera, captures samples during motion plus settle, and emits zoomTrace in the scenario result.
performanceBudget	Optional CLI-enforced thresholds for scenario metrics and/or nested zoomTrace metrics. Failing budgets are written to performanceBudgetIssues and make npm run scenarios exit non-zero; warning budgets are written to performanceBudgetWarnings.
networkThrottle	Optional CLI-only network delay used by scripts/run-scenarios.mjs for slow-stream proofs. It is not applied when calling window.__mapv10RunScenario directly in an already-loaded page.
settleTimeoutMs	Optional. Default 8000. Increase for detail-zoom scenarios that must stream more tiles.
expectations	Free-form bullet list — the visual validator quotes these in its PASS report.
failConditions	Wave 1 schema, refined in Wave 3, MZ, and Wave U: Mapv10GateSpec[] evaluated by the Node CLI (viewer/scripts/scenario-gates.mjs). Each gate is a typed { type, ... } object: tilesSettled, noPageCrash, noNewReadyErrors, primaryGeometricSsePx, primaryRasterSsePx, underlayGeometricSsePx, underlayRasterSsePx, underlayTileCount, primaryTileCountMin, forbidZ0UnderlayAtPrimaryZ, ancestorZDistanceMax, sidecarReadyForActiveMeshes, semanticPolicyLoaded, semanticBandEquals, primaryZAtLeast, closeDetailScaleMin, influenceProductRequiredBeforeRendering, noPlaceholderTextureBound, coverageHoleMax, labelTextUniqueness, labelTextMinDistinctCount, labelMaxDuplicatesForText, interactionPayloadKind, inspectorFieldPresent, selectionMaskSelectedCount, hoverPayloadClearable, selectionClearable, labelEntityEmphasized, cursorFeedback, influenceInspectorFieldsPresent. A scenario fails when any gate in its list evaluates to a failure. The legacy free-form prose array was replaced; surviving archival prose lives in notes. Wave 3 removed the aggregated maxSsePx / p95SsePx gates — see the gate-vocabulary table above for why; the replacement gates are role-and-metric specific.
notes	Optional string[] archival prose carried over from the legacy failConditions strings. Validators do not parse this; it is human context for additional regressions the scenario also cares about.

zoomTrace shape:

{
  "label": "continent-to-location-clean-lowland",
  "startCamera": {
    "positionKm": [840, -1830, 2400],
    "targetKm": [840, 870, 0],
    "fovDeg": 50,
    "altitudeReference": "target-terrain",
    "bypassInteractiveLimits": true
  },
  "steps": 96,
  "maxSamples": 5000
}

performanceBudget shape:

{
  "zoomTrace": {
    "fail": {
      "frameP95Ms": 24,
      "frameP99Ms": 120,
      "frameMaxMs": 160,
      "slowFrameCount33Ms": 5,
      "slowFrameCount50Ms": 4,
      "longTaskCount": 4,
      "maxSchedulerPending": 160,
      "maxFrameBudgetPending": 48,
      "cacheEvictionsDelta": 0,
      "cacheHitRateMin": 0.9,
      "omittedSlotCountMax": 0
    },
    "warn": {
      "maxFrameMs": 75,
      "consecutiveFramesOver33Ms": 2,
      "routeBatchCountMax": 512,
      "renderCallsMax": 700,
      "trianglesMax": 3000000,
      "labelChurnPerFrameMax": 96,
      "meshCreateCountPerFrameMax": 96
    }
  }
}

Legacy flat performanceBudget.zoomTrace objects are still accepted as hard-fail groups. Use the nested fail/warn shape for new budgets so new metrics can start warn-only while baselines settle. Omitted terrain slots remain a hard-fail gate for clean/local scenarios; the retained slow-network scenario is the exception because its remote RTT throttle is non-product stress evidence.

Browser visual baselines

The browser verifier now owns the screenshot baseline gate for the canonical generated run. It captures the same evidence a human uses when judging the map: overview, slow zoom, constant-distance camera move, all current map modes (geography, political, routes, height, slope, normal), close/detail, tile-boundary, zoom-back, and required-failure UX.

For mapv10 wave validation, Browser MCP/manual screenshots remain preferred. When Browser MCP tools are unavailable in a CLI session, npm run verify:browser:continent may substitute only for the continent browser self-audit/gates, and only with real Playwright execution, screenshot/proof artifacts, final __mapv10Ready.loaded === true, empty readiness errors, the script's console/GL warning scan, and missing-required-product hard-failure proof. Report manual Browser MCP evidence as unavailable; do not fabricate it. This does not replace npm run scenarios or scenario-specific visual validation when those are in scope.

From examples/map/mapv10/viewer:

npm run fixture:continent:validate
npm run verify:browser:baseline:update
npm run verify:browser:baseline

verify:browser:baseline:update writes a local baseline under viewer/baselines/browser/<run-id>/manifest.json plus PNGs. verify:browser:baseline reruns the full proof, compares every screenshot against that manifest, and fails with structured metrics:

• changed pixel count and ratio
• RGB RMS delta
• maximum channel delta
• diff PNG path for failed comparisons

The default thresholds are intentionally small but not zero: per-channel pixel tolerance 18, max changed-pixel ratio 0.01, and max RMS 6. Override with --baseline-pixel-tolerance, --baseline-max-changed-ratio, and --baseline-max-rms when investigating headless GPU variance. Do not recapture baseline images to hide a renderer regression; update them only after the new image is visually accepted.

Baseline-in-git policy. The repository commits only the SHA-256 manifest under viewer/baselines/browser/<run-id>/manifest.json. PNG screenshots under viewer/baselines/browser/<run-id>/screenshots/ are gitignored (see viewer/.gitignore). This keeps the manifest reviewable in pull-request diffs as a small JSON of per-screenshot hashes plus metadata while keeping the repo free of large binary blobs. The workflow is:

1. npm run verify:browser:baseline:update regenerates manifest.json AND the local PNGs.
2. npm run verify:browser:baseline reruns the full proof and compares every screenshot's pixels against the committed manifest's hashes plus the tolerance thresholds.
3. Only the manifest is committed; PNGs stay local. CI baseline access is T3-3 work (LFS / artifact bundle / golden pack decision).
4. A regression is investigated and fixed in code or in the affected scenario expectations — never by refreshing the baseline to make the diff disappear. Refresh runs only after the new visual state has been accepted by the user.

T3-3 can later decide whether CI obtains the PNGs from LFS, an artifact bundle, or a checked-in golden pack. Until then, baselines are user-triggered and local-only beyond the committed manifest.

networkThrottle shape:

{
  "delayMs": 75,
  "urlIncludes": [
    "/mapv10/runs/continent-lod6/meshes/terrain/",
    "/mapv10/runs/continent-lod6/tiles/raster/",
    "/mapv10/runs/continent-lod6/tiles/semantic/",
    "/mapv10/runs/continent-lod6/tiles/vector/"
  ]
}

Use this only for harnessed browser scenarios such as mapv10_continent_to_location_slow_network, where the goal is to prove parent fallback under delayed child-tile streaming.

Current zoom evidence

The accepted R-25 route adaptive batching/upload pacing proof passed all 23 scenarios with 0 hard budget failures and 0 warnings. That table is historical and pre-dates the LOD/Data Coherence Foundation chain (Waves 1-4.5): Wave 1 grew the gate vocabulary (forbidZ0UnderlayAtPrimaryZ, ancestorZDistanceMax, sidecarReadyForActiveMeshes, noPlaceholderTextureBound, coverageHoleMax, labelTextUniqueness, labelTextMinDistinctCount, labelMaxDuplicatesForText) and additional scenarios were added. Wave U added interaction payload scenarios; the current scenario source now has 32 definitions.

The current committed evidence under viewer/verification/scenarios/latest/scenario-results.json was captured on 2026-05-11 against /mapv10/runs/continent-lod6/manifest.json, but it is not a complete proof packet: the artifact declares scenarioCount: 27 while storing 25 result entries. The missing recorded entries are mapv10_location_height_z5 and mapv10_location_label_uniqueness_z5. Of the 25 stored entries, 24 have no hard budget or fail-condition issues and mapv10_continent_to_location_slow_network has the old blocking omittedSlotCountMax issue with actual value 24 under remote RTT throttle. PROC reclassifies that throttled omitted-slot metric as non-blocking stress evidence for a local-only product; regenerate the full scenario evidence later rather than fabricating the two missing results.

Scenario	p95	max	lifecycle peak	fallback	warnings
mapv10_zoom_continent_to_location_clean_lowland	4.4 ms	10.2 ms	mesh create 48, mesh dispose 38, texture create 96, texture dispose 95	fallback 28, omitted 0, duration 2	none (R-25 historical)
mapv10_continent_to_location_slow_network	—	—	—	omitted 24 under old blocking gate	non-blocking remote-throttle stress evidence after the local-only decision
mapv10_zoom_realm_to_location_overlays_z5	9.0 ms	20.5 ms	mesh create 48, mesh dispose 34, texture create 96, texture dispose 80	fallback 24, omitted 0, duration 3	none; route upload 67,584 bytes, route build 4.6 ms (R-25 historical)

Terrain lifecycle, label churn, route upload/build remain closed. The next scenario-evidence work is a full rerun that records all 32 definitions under the accepted local-only gate classification; it is not a visual-baseline update.

Coordinate cheatsheet (continent-lod6 fixture)

• Continent extent: ~2400 km × 1800 km (x × y in world km)
• Centre point: [1200, 900, 0]
• A typical "top-down at 5000 km altitude" pose: positionKm = [1200, 900, 5000], targetKm = [1200, 900, 0]
• A typical "oblique from south" pose: positionKm = [1200, -1300, 1500], targetKm = [1200, 900, 0], fovDeg = 50
• Close terrain readability now has both a highland proof (mapv10_location_geography_clean_oblique_z5, target ~2148 m) and a lower-elevation proof (mapv10_location_geography_clean_lowland_oblique_z5, target ~997 m). Run both when changing terrain material, lighting, haze, or micro-detail.

How to add a scenario

1. Open examples/map/mapv10/viewer/src/scenarios/mapv10_scenarios.json.
2. Append a new object to the scenarios array. Use an id that starts with mapv10_ and is unique.
3. Save. Vite HMR picks the change up automatically — no rebuild.
4. Verify in the dev console: window.__mapv10ListScenarios() should now include your id.
5. Run it: await window.__mapv10RunScenario("your-id"). Inspect the returned result.
6. Take a screenshot via Chrome MCP / DevTools and check it matches your expectations.

No code change needed unless you're adding a new field to the scenario schema (in which case bump schemaVersion and update scenarioTypes.ts + scenarioRunner.ts).

How the scenario runner orchestrates a run

1. Snapshot __mapv10Ready.errors length (so the runner can return the new errors that fired during the scenario, not pre-existing ones).
2. Call __mapv10SetMode(mode) — swaps the unified shader's uModeId AND syncs the mode-tab UI.
3. Call __mapv10SetLayerState(layers) — updates layer panel checkboxes and renderer state in one shot.
4. If zoomTrace is absent, call renderer.setCameraPose(positionKm, targetKm, fovDeg) — deterministic pose write, no heuristic.
5. If zoomTrace is present, start renderer.startZoomTrace, interpolate from zoomTrace.startCamera to camera, record one frame per step, then keep tracing through settle.
6. await renderer.waitForTilesSettled(settleTimeoutMs) — resolves when no fetches are pending and the scheduler has drained, OR after the timeout.
7. await renderer.waitForFrame() — markFrameDirty + two requestAnimationFrames, guaranteeing one stable on-screen render.
8. Stop the zoom trace when active and attach the Mapv10ZoomTraceReport to the scenario result.
9. Read renderer.getCameraState() and renderer.getTerrainGeometryProbe().
10. Compute newErrors = errorsAfter.slice(errorsBefore.length).
11. Return the structured Mapv10ScenarioResult.

Step 6 is the only step that can fail "softly" (tiles didn't settle in time). The runner records tilesSettled: false in that case but still returns the screenshot-ready frame — the validator decides whether that's a PASS or FAIL.

How validators consume scenarios

The visual validator (Step 3, Sonnet, mapv10-validate-visual) walks every scenario id, runs each, screenshots after each runScenario resolves, then writes a per-scenario verdict:

=== mapv10_continent_political_oblique ===
durationMs:     19967
tilesSettled:   true
hasDisplacement: true   (worldZRangeKm = 11.84)
newErrors:      []
expectations:
  ✓ Skyline silhouette is jagged
  ✗ Political coloring tracks the displaced terrain surface — appears mostly flat at this exaggeration
failConditions:
  ✓ NOT flat horizon (geometry probe shows displacement)
  ✗ Political drapes on plane — but this looks like a vertical-exaggeration tuning issue, not a missing-displacement bug
verdict: WARN — geometry is correct but visual exaggeration may be insufficient

A verdict: FAIL on any scenario routes the wave back to Step 2.

Common gotchas

• The scenario runner now waits for __mapv10Ready.loaded === true. A scenario screenshot taken before the manifest and initial products load is a false proof: it can capture a blank loading frame with tilesSettled: false. If the viewer never reaches ready, the runner fails loudly and includes current readiness errors.
• Camera coordinate system is scene-space, not world-space, in finalCameraState. The runner converts your world-km input to scene units inside setCameraPose. Reading renderer.getCameraState() gives you the scene-space values back. To round-trip: worldKm.x = sceneX + worldWidth/2, worldKm.y = sceneZ + worldHeight/2, worldKm.z = sceneY.
• Don't await __mapv10RunScenario(...) directly in mcp__Claude_in_Chrome__javascript_tool — the CDP eval has a 45 s timeout and a long scenario can exceed it. Fire-and-poll: window.__mapv10RunScenario(id).then(r => window.__last = r) then read window.__last later.
• hasDisplacement: true does NOT mean "mountains are visually obvious." It means worldZRangeKm > 0.001. Visual exaggeration is a separate concern — the displacement is real, but the human eye can miss sub-1%-of-extent relief at strategic camera distances.