Skip to content

test: add memory check for unit tests in ci#558

Merged
Miou-zora merged 7 commits into
mainfrom
447-feature-add-memory-checks-valgrind-leaks-for-unit-tests-in-ci
Apr 5, 2026
Merged

test: add memory check for unit tests in ci#558
Miou-zora merged 7 commits into
mainfrom
447-feature-add-memory-checks-valgrind-leaks-for-unit-tests-in-ci

Conversation

@Miou-zora
Copy link
Copy Markdown
Contributor

@Miou-zora Miou-zora commented Apr 5, 2026

Pull Request

Description

I added a way to check memory through leaks on macOS and put it in CI. For now it is not a mandatory to pass the CI.

Related Issues (Put "None" if there are no related issues)

close #447

Type of Change

Please delete options that are not relevant.

  • Build/CI configuration change

Changes Made

List the main changes in this PR:

  • Added a task to check memory on unit tests on macOS ("check_leaks")
  • Put this in the CI

Testing

Describe the tests you ran to verify your changes. Please delete options that are not relevant.

  • Unit tests pass (xmake test)
  • Manual testing performed (describe what you tested)

Test Environment

  • OS: macOS
  • Compiler: Clang

Screenshots/Videos (Put "None" if there are no related issues)

None

Documentation

Please delete options that are not relevant.

  • No documentation changes are required

Checklist (Don't delete any options)

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Breaking Changes (Put "None" if there are no related issues)

None

Additional Notes (Put "None" if there are no related issues)

None

Summary by CodeRabbit

  • Chores

    • Updated CI workflow tooling to latest versions
    • Reorganized build configuration management for better modularity
  • New Features

    • Enabled memory leak detection for macOS builds in CI pipeline
  • Improvements

    • Updated test execution to run in debug mode for enhanced diagnostics
    • Added automatic cleanup of test artifacts on Linux

@Miou-zora Miou-zora linked an issue Apr 5, 2026 that may be closed by this pull request
@Miou-zora Miou-zora self-assigned this Apr 5, 2026
@Miou-zora Miou-zora added the ci Everything related to continous integration label Apr 5, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 5, 2026

📝 Walkthrough

Walkthrough

This pull request introduces automated memory leak detection into the CI pipeline using macOS's native leaks tool. It extracts build group name constants from the root Xmake configuration into a dedicated module, updates CI workflow configurations with debug-mode test execution and platform-specific checks, and adds a new Xmake task to orchestrate leak checking across test binaries.

Changes

Cohort / File(s) Summary
CI Workflow Updates
.github/workflows/ci.yml, .github/workflows/commit_norm_check.yml
Enhanced CI pipelines with checkout action upgrade (v4→v6, full fetch depth), debug-mode test compilation, macOS-exclusive leak checking, and Linux-exclusive artifact cleanup.
Xmake Build Configuration
xmake.lua, src/engine/xmake.lua
Relocated group name constants (TEST_GROUP_NAME, PLUGINS_GROUP_NAME, UTILS_GROUP_NAME) from root config to shared module; minor whitespace formatting cleanup.
Memory Leak Checking Infrastructure
tools/xmake/groups.lua, tools/xmake/check_leaks.lua
Introduced group name constants module and new Xmake task for executing macOS leaks tool against test binaries, with target filtering and failure reporting.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Poem

🐰 A leak in the memory? Not here, I declare!
With leaks on macOS, we hunt with such care,
Debug builds and cleanups, our tests run so tight,
No rabbits have holes in their burrows tonight! 🕳️✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR partially addresses #447 by implementing macOS leak checking in CI, but does not implement Linux/Windows Valgrind checks or AddressSanitizer as specified in issue requirements. Complete the implementation by adding Valgrind support for Linux/Windows and consider AddressSanitizer as alternatives, or adjust the PR scope to explicitly target macOS-only support.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the main change: adding memory checks for unit tests in CI, specifically the macOS leak check implementation.
Out of Scope Changes check ✅ Passed Changes to CI workflows, Xmake configuration, and build system files are all directly related to implementing memory checks in CI as specified in #447.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 447-feature-add-memory-checks-valgrind-leaks-for-unit-tests-in-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Miou-zora Miou-zora added the test Additions or modifications to tests label Apr 5, 2026
@Miou-zora Miou-zora linked an issue Apr 5, 2026 that may be closed by this pull request
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Apr 5, 2026

@Miou-zora Miou-zora marked this pull request as ready for review April 5, 2026 12:38
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
tools/xmake/check_leaks.lua (1)

17-17: Suppressing leaks output prevents debugging.

Redirecting both stdout and stderr to /dev/null hides the actual leak information when leaks are detected. Consider showing output on failure or removing the suppression entirely.

♻️ Proposed fix: show output or capture for failures

Option 1 - Remove suppression to always show output:

-        local return_value = os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true, stdout = os.nuldev(), stderr = os.nuldev()})
+        local return_value = os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true})

Option 2 - Only show output when leaks are found (run twice):

-        local return_value = os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true, stdout = os.nuldev(), stderr = os.nuldev()})
+        local return_value = os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true, stdout = os.nuldev(), stderr = os.nuldev()})
         if return_value ~= 0 then
+            print("Leak details for " .. target_name .. ":")
+            os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true})
             table.insert(failing_targets, target_name)
         end
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tools/xmake/check_leaks.lua` at line 17, The exec call to run the leaks tool
(os.execv(leaks_tool, {"--atExit", "--", bin_path}, {try = true, stdout =
os.nuldev(), stderr = os.nuldev()})) suppresses all output and hides useful leak
diagnostics; modify the invocation in check_leaks.lua so stdout/stderr are not
discarded — either remove the {stdout = os.nuldev(), stderr = os.nuldev()}
options so leaks_tool prints directly, or capture the output (e.g., not
discarding streams), check return_value, and on non-zero/failed runs print or
log the captured stdout/stderr so leak details are visible; ensure you update
the os.execv call around the leaks_tool and bin_path usage and handle
return_value accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 149-153: The CI macOS step "Check memory leaks (macOS)" invokes
"xmake check_leaks" but the underlying script check_leaks.lua does not fail the
process when leaks are detected; update check_leaks.lua so that when a leak is
found it calls os.raise() (or otherwise raises an error) instead of merely
logging, ensuring the xmake check_leaks target returns a nonzero status and
makes the workflow step fail; locate the leak-detection branch/handler in
check_leaks.lua (the function or block that parses leak results) and replace the
silent/log-only behavior with os.raise() including an informative message.

In `@tools/xmake/check_leaks.lua`:
- Around line 57-64: The script currently only prints leaking targets but
doesn't signal failure; update the end of the check (the block that inspects
failing_targets) so that when `#failing_targets` > 0 it not only prints the list
but also exits non‑zero (e.g., call os.exit(1) or error with a clear message) to
fail the CI job; modify the code around the failing_targets handling (the block
that prints "Targets with memory leaks found:" and iterates over
failing_targets) to perform the non‑zero exit after printing.
- Line 9: The bin_path construction is incorrect because it appends a hardcoded
"debug" directory to target:targetdir() and target:filename(); replace that
logic to obtain the binary path from the target itself by using
target:targetfile() (or an equivalent API) instead of path.join(os.projectdir(),
target:targetdir(), "debug", target:filename()); update the code that sets
bin_path to call target:targetfile() so it works for debug/release and
platform-specific target layouts.

---

Nitpick comments:
In `@tools/xmake/check_leaks.lua`:
- Line 17: The exec call to run the leaks tool (os.execv(leaks_tool,
{"--atExit", "--", bin_path}, {try = true, stdout = os.nuldev(), stderr =
os.nuldev()})) suppresses all output and hides useful leak diagnostics; modify
the invocation in check_leaks.lua so stdout/stderr are not discarded — either
remove the {stdout = os.nuldev(), stderr = os.nuldev()} options so leaks_tool
prints directly, or capture the output (e.g., not discarding streams), check
return_value, and on non-zero/failed runs print or log the captured
stdout/stderr so leak details are visible; ensure you update the os.execv call
around the leaks_tool and bin_path usage and handle return_value accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a0daf330-1286-4ca4-b6e6-616143cfa6ab

📥 Commits

Reviewing files that changed from the base of the PR and between 3aaa52f and 8d51a1a.

📒 Files selected for processing (6)
  • .github/workflows/ci.yml
  • .github/workflows/commit_norm_check.yml
  • src/engine/xmake.lua
  • tools/xmake/check_leaks.lua
  • tools/xmake/groups.lua
  • xmake.lua
💤 Files with no reviewable changes (1)
  • xmake.lua

Comment thread .github/workflows/ci.yml
Comment thread tools/xmake/check_leaks.lua
Comment thread tools/xmake/check_leaks.lua
@Miou-zora Miou-zora merged commit 175321e into main Apr 5, 2026
59 of 60 checks passed
@Miou-zora Miou-zora deleted the 447-feature-add-memory-checks-valgrind-leaks-for-unit-tests-in-ci branch April 5, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Everything related to continous integration test Additions or modifications to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add memory checks (Valgrind / leaks) for unit tests in CI

1 participant