Files
llamamod/notes/mountain3-crash.md
T

10 KiB
Raw Blame History

"Mountain 3 crash" (speedrun community) – investigation 2026-11-08

Short version: Mountain 3 = t1l5m003 (world 5, chapter 3). Its boulder spawner (RollingBoulderMountain) can end while a boulder it spawned is still alive. When that boulder finishes its path, it decrements parent.v13 of the dead spawner. That write (-= 1.0, i.e. -0x1000) goes into freed or reused memory (use after free). If it hits a header of the game's local heap, a later FreeLocalMem crashes. That matches the community crash dump. The stale write is verified in game. A full crash was not reproduced. A one-word level-data patch removes the stale write (verified in game).

Evidence from the community

  • DebugDiag dump (English exe, 2000-10-17): access violation writing 0x0000340d at groove+0x1ad52 = VA 0x41AD52, thread 0, Argonaut name FreeLocalMem.
  • English 0x41AD10..0x41AD89 is byte-identical to German FUN_0041ada0 (only the rel32 of the "ITS NOT USED! %08x" debug printf differs). Crash instruction = German 0x41ADE2 mov [eax+4], esi.
  • The local heap (FUN_0041af00 init, FUN_0041ae20 alloc, FUN_0041ada0 free) holds blocks with a 12-byte header {next, prev, u16 size (dwords), u16 used}. The blocks are contiguous (next == hdr + (size+3)*4). On free, the routine merges with the next block if next.used == 0, then does next->next->prev = hdr. The crash means the header after the freed block had used == 0 and next == 0x3409. 0x3409 is not 4-aligned, so it can't be a real block pointer. The heap metadata was corrupted earlier, and the free only shows it.
  • Clips: index.mp4 is in a snowy canyon with a pushable boulder (red X), wall shards (SmashWallShrapnel), a ruby-idol pickup (HUD), and a crash at the cave about 6 s later. index2.mp4 is the Kronk ice race (KronkOnIceDoor/KronkOnIce, same WAD): black screen about 2 s after the race starts. Both are in t1l5m003. Our boot to (74.6, 14.8, 6.3) shows the same canyon and boulder as clip 1, so clip 1 is most likely the #47 push-boulder section (hypothesis, based on what the scenes look like).
  • Side note: the heap is operator_new(0x200000) but is initialised with size 0x800000 into a u16 field, so it is really 0xFFFD dwords (≈256 KB). Mountain 3 uses about 88 KB of it, so the heap is not exhausted. No leak was seen: block count stayed stable over minutes.

The bug (strat code of unit 1 of t1l5m003, verified by disassembly)

RollingBoulderMountain (6 MAP instances, params v0..v9). v13 is set to 0 and never incremented anywhere. The only other access is the boulder's parent.v13 -= 1.0 (pushParentVarAddr v13; dec) in L_0574, which runs when the boulder ends its path or falls:

spawner:  ... spawn loop (L_0c24: spawn<7,31,4,0x14c000,0>(…RollingBoulderForMountain)) ...
          freezeOffCamera 1
L_0d90:   pushVar v13 / pushShort 0 / eq / jzYield L_0d90 / end      ; ends at once when v13 == 0
boulder:  L_0574: parent.v13 -= 1.0 ; ... ; remove
  • With v1 == 0 (instances #17 and #35) or v4 != 0 (#22 and #47, the pushable "red X" boulders, which wait to be charged), the spawner spawns one boulder and ends in the same frame (v13 is still 0).
  • Instances #1 and #27 (v1 = 8.3 / 8.75) loop: a boulder about every 8–10 s, until the player is near the path start. They end the same way if no boulder has died yet when the loop exits.
  • Strat removal (strat update loop FUN_0054c4a0) frees the var block (+0xF4) and the collision block (+0xC0) and puts the struct on the free list (LIFO). It does not clear the children's +0xC parent pointers. The parent opcodes (0x553750/0x553770/ 0x553a40) have no check.
  • So the boulder's write goes to deadSpawner.vars + 0x34: either memory of the freed var block (now part of a free block or of another allocation, possibly a block header), or, if the struct was already reused, v13 of another strat. For a boulder, v13 is its sound address (v13 = sound:4, used by playSoundChannel).
  • "Predictable intervals" (runner's guess) fits: the stale writes happen a fixed time after you pass a spawner, or after you push a boulder, when that boulder reaches the end of its path. Whether a write hurts depends on the heap layout at that moment, so it depends on timing. It is not deterministic across runs.

In-game verification (apptest, Xvfb :26, sessions mtn3-*)

Tooling (throwaway, see notes/dev/mountain3-diag/): a test-only mod Mtn3DiagMod (loaded through APPTEST_CP + -Dllamamod.extraMods). It installs a code cave (cave.asm, nasm) that replaces the pushParentVarAddr and popParentVar handlers. They log every access, flag stale ones (parent +0xEC == 0, i.e. removed) together with the block that the target address falls in, and walk the whole heap on every alloc and free (next/prev/used consistency). rd.py reads the log by attaching winedbg for a moment (stdin script attach/x/detach; works under ptrace_scope 1). apptest.py in the agent copy got an env switch APPTEST_IGNORE_WINEDBG so the short attach is not taken for a crash (not copied back). A gdb-over-winedbg monitor with breakpoints was too slow: the game froze.

Boot-to-level start=29.5, 14, 23.2, 180 @ 5/3 (next to spawners #1 and #17), no input, 70–85 s:

runs runs with a stale write heap anomalies crash
original WAD 6 (1 manual + 5) 5 0 0
level fix (below) 3 0 0 0

Every stale record was the same and deterministic for this start (logged by the cave): RollingBoulderForMountain → parent RollingBoulderMountain (struct +0xEC == 0, removed), v13, target = freed var block + 0x34. At the time of the write this address was inside a used 16-dword block (a 0x40000 collision block, most likely the next boulder's), at offset +0x40 from its header, which is 12 bytes before the next block header. In this layout it corrupts collision data (u16 fields +0x34/+0x36 become 0xF000/0xFFFF), not the heap. Any other layout (other timing, other route, other spawns in between, e.g. an 11-dword allocation such as a 0x28000 pickup's collision block puts a remainder header's size/used word exactly there) turns the same write into header corruption. Not verified: a crash caused by this write. No natural run crashed, and the real runner route (from the chapter start at full speed, pushing #22/#47, then the race) could not be driven reliably by key scripts (the push needs speed and alignment; several attempts missed). Also seen in game: strat structs are recycled LIFO (the dead spawner's struct was later reused by a boulder).

Other checks in this WAD: every collisionOn user is spawned or placed with collision type flags (no typeFlags 0 + collisionOn case like in the arena), and all extra operands are 0. Other parent-variable writers (KronkOnIcePickup → parent.v7, CheckPointHead, RubyIdolStatue, AxeGuardAxe) have parents that do not end while they live, as far as the code shows. That was checked by reading the code only; the cave saw no stale access from them.

Side finding (not part of the crash): boot-to-level start=28.5, 10.5, 22.5, 0 @ 5/3 passes the floor check, but the game crashes while loading (page fault read 0x00000002 at 0x405527, floor/ tile lookup in FUN_004054c0). Out of scope here.

Fix / avoidance

  • Level data (tested): in RollingBoulderMountain, change pushShort 0 before eq to pushShort 0.5 (word 0x00000044 → 0x08000044, the only match of the 6-word pattern 0x1002f 0xd0004 0x44 0x4e 0xfffc0012 0xb4, file offset 0x4a0a27 in the German t1l5m003.wad). v13 only takes whole numbers (0, -1, -2 …), so the spawner never ends. It stays frozen off camera and costs nothing, and every boulder writes into a live spawner. The boulders still roll (in-game: 6–7 normal parent writes per run, 0 stale). notes/dev/mountain3-diag/wadfix.py IN OUT. The English CD's WAD is not checked; the patch looks for the pattern and refuses if it isn't unique.
  • Exe (general, untested): let removal (+0xE8 & 4 path) clear +0xC of all strats whose parent is the removed one (pool DAT_006d9dc0, stride 0x154), and let the parent opcodes redirect a NULL or removed parent to a dummy dword. This also covers similar bugs in other levels.
  • Runners (hypothesis): there's no reliable avoidance. The write happens when the boulders of #17 and #35 (spawned when you pass their trigger spots) and the pushed boulders #22 and #47 reach the end of their path. Not pushing #22/#47 removes two of the four guaranteed stale writes, if the route allows that.

Diag tooling notes: the files in notes/dev/mountain3-diag/ are as used, with absolute paths of the agent job dir. The mod writes addresses.txt and expects cave_<fix>_<CAVE>_<DATA>.bin (nasm -f bin -DCAVE=0x6ef0e0 -DDATA=0x70f010 -DFIX=0 cave.asm; these addresses are valid with boot-to-level + mtn3-diag on the German exe). This is only for the German exe; nothing is registered in the app.

Discord answer (draft)

We looked into the Mountain 3 crash. Your dump crashes in the game's own memory manager (FreeLocalMem, writing to 0x340d). That is the symptom: its bookkeeping was overwritten earlier. Verified: in Mountain 3 the rolling-boulder spawner can delete itself while its boulder is still rolling. When that boulder reaches the end of its path (or falls off), it writes into the spawner's memory, which has already been freed and reused. We saw this write in the game with a debug build, and a one-value level patch makes it go away. Not verified yet: that this exact write causes your crash. We never got the game to crash. Where the write lands depends on what else is in memory at that moment, so it breaks things only sometimes, and the crash can come seconds later somewhere else (e.g. in the Kronk race). That would explain why it's Mountain 3 only, at different spots, without an error message, and "every so often" around boulders. The likely triggers are the one-shot boulders near the start/top of the slope and the two red-X push boulders. We'll probably ship the fix in LlamaMod.