Windows, GCC: threaded programs can hang for ever in a deadlock between Boehm GC and the C heap #31518

Describe the bug

On Windows, a threaded V program built natively with GCC can hang for ever: no CPU used, no output, no error. It happens at random, and most often early in the run. It still happens on current master.

It is a deadlock between Boehm GC's stop-the-world and the C heap. The collector suspends the other threads, then calls signal() while marking. In msvcrt, signal() can allocate from the C heap. If a suspended thread was inside that heap, it holds the heap's lock and can never release it, so the collector waits for ever.

Reproduction Steps

// gc_hang.v
const workers = 16
const rounds = 300

fn work() int {
	mut total := 0
	for _ in 0 .. rounds {
		// the C heap, which holds its lock while it works
		for _ in 0 .. 200 {
			p := unsafe { C.malloc(4096) }
			unsafe { C.free(p) }
		}
		// garbage, so that collections start
		mut names := []string{cap: 8}
		for i in 0 .. 100 {
			names << 'name ${i}'
		}
		total += names.len
	}
	return total
}

fn main() {
	mut threads := []thread int{}
	for _ in 0 .. workers {
		threads << spawn work()
	}
	mut sum := 0
	for t in threads {
		sum += t.wait()
	}
	println(sum)
}

Build with GCC and run it repeatedly, since one run normally succeeds in about 0.4 s:

v -cc gcc -cflags "-fpermissive -w" -o gc_hang.exe gc_hang.v

(-fpermissive -w is only there because GCC 16 otherwise rejects the generated C here.)

Expected Behavior

Every run prints 480000 and exits.

Current Behavior

Some runs never exit: 4 of 352 on 92cf0f5, and 8 of 380 on 0535623. A hung process has many threads in Suspended, for example 14 of 17, and stays at the little CPU it had used.

Stacks from gdb, with addresses and paths removed. The collector, waiting for the heap's lock:

ntdll!RtlEnterCriticalSection ()
ntdll!LdrFindEntryForAddress ()
ntdll!RtlAllocateHeap ()
ntdll!RtlAllocateHeap ()
msvcrt!malloc ()
realloc ()
signal ()
GC_set_and_save_fault_handler ()
GC_setup_temporary_fault_handler ()
GC_mark_some ()
GC_stopped_mark ()
GC_try_to_collect_inner ()
GC_collect_or_expand ()

A suspended thread, inside the heap:

ntdll!RtlAllocateHeap ()
msvcrt!malloc ()
work ()
work_thread_wrapper ()
__v_windows_thread_start ()
GC_win32_start_inner ()

The other threads wait in GC_malloc or GC_register_my_thread for the collector's lock.

Possible Solution

thirdparty/libgc/gc.c defines WRAP_MARK_SOME on Windows, and NO_SEH_AVAILABLE under __GNUC__. So a GCC build has no __try, and GC_mark_some wraps marking in GC_setup_temporary_fault_handler, which calls signal() with the world stopped. MSVC builds should not take this path, though I have not tested one.

Possible directions, none of which I have tried:

• Do not call into the CRT with the world stopped, for example by installing the handler before stopping it, or with a vectored exception handler.
• Define NO_WRAP_MARK_SOME for GCC builds on Windows, though libgc's comment there says it guards against marking from memory that has gone away.

A workaround that does work is to disable collection while threads run. With gc_disable() at the start of main in the program above, there were 0 hangs in 554 runs.

Additional Information/Context

• This is separate from makev: self-hosted fallback and Boehm thread registration on Windows #30278, which registers threads with the collector. That fix is included in the version below, and the hang remains. It ends in a silent hang, not in Collecting from unknown thread.
• It seems to need a thread's first signal() call: msvcrt appears to allocate a per-thread table there, which would be why hangs come early in a run.
• It was first seen in a program whose threads parse files, with fread inside RtlAllocateHeap as the suspended thread.
• A program cross-compiled from Linux with -os windows -cc x86_64-w64-mingw32-gcc is not affected: it came out with no libgc in it at all. A native build with MSYS2 GCC has it.

V version

V 0.5.2 92cf0f5 (master of 2026-10-31), and V 0.5.2 0535623

Environment details (OS name and version, etc.)

Windows 11 x64, GCC 16.2.0 (MSYS2 mingw64), msvcrt.