Fix segfault in GC standalone large pages emulation mode (segments path)

[cshung]

In the non-regions (segments) path, GCLargePages=2 (emulation mode) was not handled by virtual_alloc. The function only had a bool use_large_pages_p parameter, so it could not distinguish between real large pages (VirtualReserveAndCommitLargePages) and emulation mode (which needs VirtualReserve + VirtualCommit). In emulation mode, memory was reserved but never committed, causing a segfault on access.

Fix: Change virtual_alloc to accept an int large_page_config parameter:

• 0 = no large pages (reserve only)
• 1 = real large pages (VirtualReserveAndCommitLargePages)
• 2 = emulation (VirtualReserve + VirtualCommit)

The caller computes large_page_config once and passes it through reserve_initial_memory, eliminating the need for reserve_initial_memory to access the static large_pages_emulation_mode_p field.

The test is updated to use per-object-heap hard limits (GCHeapHardLimitSOH/LOH/POH), which are required for the segments path with large pages since each segment is sized to the full hard limit (soh + loh + poh = 3x limit with a single combined GCHeapHardLimit).

Fixes #127668

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[mangod9]

I am not convinced whether we should do this for segments. In fact we should be looking at not shipping clrgc.dll given that regions have been live for 3+ releases already

[cshung]

I am not convinced whether we should do this for segments. In fact we should be looking at not shipping clrgc.dll given that regions have been live for 3+ releases already

I think we should still maintain this path for two reasons:

1. The segments path (#ifndef USE_REGIONS) is not just about standalone GC — it's the codepath 32-bit systems use in production. Standalone GC is currently the mechanism that exercises this path in CI. If we stop maintaining it, we're effectively letting the 32-bit segments codepath rot without any test coverage.

2. Even setting aside 32-bit, standalone GC provides a valuable vehicle for flexible testing and experimentation. We may not want to ship it, but that doesn't mean it isn't worth keeping functional.

The fix itself is small and straightforward — it passes the large page config as an integer through virtual_alloc so the emulation mode does Reserve+Commit correctly on both the regions and segments paths.

[mangod9]

is the x86 case broken due to the new changes here?

[cshung]

is the x86 case broken due to the new changes here?

No, x86 is good:

• x86 is not broken before this PR (large pages are not supported on x86, anything broken because of large page does not impact x86).
• x86 is not broken after this PR either (this PR impacts large page only)