Skip to content

Commit 49a7238

Browse files
jfrolichclaude
andauthored
Rewatch: replace wave scheduler with DAG + critical-path priority (#8374)
* Rewatch: replace wave scheduler with DAG + critical-path priority Instead of compiling modules in topologically-sorted waves (which stall on the slowest file per wave), schedule them from a priority queue on a work-stealing dispatcher. Priority is the length of the module's longest downstream dependency chain, so bottlenecks start first. Workers run inside a `rayon::in_place_scope` bounded to `rayon::current_num_threads()` so the heap ordering actually decides dispatch. Completions flow back over `std::sync::mpsc`; the dispatcher updates pending-dep counters, propagates the dirty flag, and pushes newly-ready modules back onto the heap. `BuildState` mutations happen after the scope exits so worker reads and main-thread writes stay disjoint. Cycle detection now triggers when the heap drains without completing the universe. Signed-Off-By: Jaap Frolich <jaap@tella.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Rewatch: tighten scheduler invariants and sort results by module name - Restore the `.unwrap()` on `package.namespace.to_suffix()` for mlmap dispatch so a missing namespace suffix panics loudly instead of being silently papered over. - Drop the defensive `.unwrap_or_default()` on the post-completion dependents lookup and the `if *count == 0 { continue }` guard on the pending-deps decrement; both can only fire when the graph itself is inconsistent, and hiding that would mask bugs. Align with the unwrap-style assertions the old scheduler used. - Sort `results_buffer` by module name before applying results so the returned `compile_errors` / `compile_warnings` strings and the per- package `.compiler.log` writes are deterministic across runs despite non-deterministic completion order. Signed-Off-By: Jaap Frolich <jaap@tella.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Rewatch: persist propagated dirtiness across failed builds When a predecessor's cmi changes the dispatcher only marks the local dirty_set; it used to not write compile_dirty = true back onto build_state. If the first compile error aborted dispatch before those dependents were scheduled, their compile_dirty stayed false in build_state, and the next incremental build would skip recompiling them even though an upstream .cmi had changed — producing stale output. Persist dirty_set onto build_state.modules[].compile_dirty right before applying compile results. The success path in the result loop still overrides compile_dirty = false for modules that actually recompiled cleanly in this run, so only the propagated-but-not-scheduled modules retain the flag. Signed-Off-By: Jaap Frolich <jaap@tella.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * CHANGELOG: DAG scheduler for rewatch Signed-Off-By: Jaap Frolich <jaap@tella.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent bcc333c commit 49a7238

2 files changed

Lines changed: 437 additions & 304 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
#### :nail_care: Polish
4141

4242
- Allow builds while watchers are running. https://github.com/rescript-lang/rescript/pull/8349
43+
- Rewatch: replace wave-based compile scheduler with a work-stealing DAG dispatcher ordered by critical-path priority, avoiding the per-wave stall on the slowest file. https://github.com/rescript-lang/rescript/pull/8374
4344

4445
#### :house: Internal
4546

0 commit comments

Comments
 (0)