Add weekly testing digest workflow for #core-test#155
Add weekly testing digest workflow for #core-test#155juanmaguitar wants to merge 2 commits intoWordPress:trunkfrom
Conversation
Adds a scheduled GitHub Actions workflow that collects open "Needs Testing" counts from both Gutenberg (via GitHub Search API) and WordPress Trac (via headless browser to bypass bot protection), then posts a digest to #core-test via a Slack Workflow webhook. Refs WordPress#131 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
I tested this action and these are the results: ❌ Trigger workflow manually via Actions tab on the fork to verify data collectionI had to create a repo to test this GH Action. I ran into a couple of issues:
permissions:
contents: read
- name: Install Playwright
run: npm install playwrightPlace this step just before the After fixing all these, the workflow now runs flawlessly. Verify Gutenberg counts match❌ TotalReceived: 13 ❌ IssuesReceived: 11 ✅ PRReceived: 2 Verify Core counts match && milestone✅ CountsReceived: 8 ✅ MilestoneReceived: 7.1 Verify links in JSON payloadWith these, I might be wrong about what the expected URL should be. Could you confirm if this is right? ✅ gutenberg_issues_url
❌ gutenberg_prs_url
❌ trac_url
Test message deliveryI don't have the necessary permissions to test this on Slack, so I created a mock server for testing.
// mock-server.js
import express from "express";
const app = express();
app.use(express.json());
app.post("/", (req, res) => {
console.log("Received payload:", req.body);
res.sendStatus(200);
});
app.listen(3000, () => console.log("Mock server running on port 3000"));
Received payload: {
gutenberg_issue_count: '11',
gutenberg_pr_count: '2',
gutenberg_total: '13',
trac_milestones: [ { milestone: '7.1', count: 8 } ],
trac_total: '8',
gutenberg_issues_url: 'https://github.com/WordPress/gutenberg/issues?q=is%3Aopen+label%3A%22Needs%20Testing%22',
gutenberg_prs_url: 'https://github.com/WordPress/gutenberg/pulls?q=is%3Aopen+label%3A%22Needs%20Testing%22',
trac_url: 'https://core.trac.wordpress.org/tickets/needs-testing',
date: '2026-04-28'
}Other IssuesThe dry-run output has some differences when running locally and on GitHub Actions Could you also adjust the trac timeouts? To 60000ms and 45000ms, respectively. await page.goto( TRAC_QUERY_URL, { timeout: 45000 } );
await page.waitForSelector( 'table.listing', { timeout: 30000 } ); |
|
I finally managed to set up a workspace where I can test the Slack webhook message delivery. 
I guess Slack doesn't play well with Arrays. These are the changes I made to the code to get it to run (added: tracSummary, changed payload output, dry-run output): const gutenbergTotal = issueCount + prCount;
const tracTotal = tracMilestones.reduce( ( sum, m ) => sum + m.count, 0 );
const date = new Date().toISOString().split( 'T' )[ 0 ];
const tracSummary = tracMilestones
.map( ( m ) => `${ m.milestone }: ${ m.count }` )
.join( ' • ' );
const payload = {
gutenberg_issue_count: String( issueCount ),
gutenberg_pr_count: String( prCount ),
gutenberg_total: String( gutenbergTotal ),
trac_milestones_count: String( tracSummary ),
trac_total: String( tracTotal ),
gutenberg_issues_url: buildGitHubURL( 'issues' ),
gutenberg_prs_url: buildGitHubURL( 'pulls' ),
trac_url: TRAC_REPORT_URL,
date,
};
if ( mode === 'dry-run' ) {
// Human-readable summary
console.error( '' );
console.error( '=== Weekly Testing Digest ===' );
console.error( `Date: ${ date }` );
console.error( '' );
console.error(
`Gutenberg: ${ gutenbergTotal } open (${ issueCount } issues, ${ prCount } PRs)`
);
console.error( '' );
console.error(
`Trac: ${ tracTotal } tickets in versioned milestones`
);
for ( const m of tracMilestones ) {
console.error( ` Trac summary ${ m.milestone }: ${ m.count } tickets` );
}Then used 
This is ready for review by the whole team. I currently don't have permission in #core-test to create workflows. I'd like to create the workflow. |
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
@3kori I didn't understand where the express server will be running to fetch the trac ticket results? Does Slack handle that? |
I tested @3kori 's code and it works correctly with both Playwright and Puppeteer. |
|
@3kori @juanmaguitar @ozgursar I've tried using the mock-server.js. I've getting below in the console. |
|
This is the repo I made to test this workflow: Weekly Digest Test repo. |
That's the correct output, but it needs a few changes to work smoothly with Slack. |
Successfulsebunya
added
Discuss
Needs: 2 Reviews
and removed
Needs: 2 Reviews
Discuss
labels
5 participants
Summary
Adds a scheduled GitHub Actions workflow that posts a weekly testing backlog digest to
#core-test, addressing #131.workflow_dispatchHow it works
The workflow runs
.github/scripts/weekly-testing-digest.mjswhich:Sample output (dry-run)
What's needed to activate
#core-testneeds to create a Slack Workflow that accepts the webhook payload and formats the messageSLACK_WEEKLY_TESTING_DIGEST_WEBHOOK_URLto this repo's Actions secrets with the webhook URL from step 1Until the secret is configured, the workflow runs in dry-run mode and logs the payload.
Payload format (for the Slack Workflow)
{ "gutenberg_issue_count": "8", "gutenberg_pr_count": "0", "gutenberg_total": "8", "trac_milestones": [{"milestone": "7.0", "count": 2}, {"milestone": "7.1", "count": 8}], "trac_total": "10", "gutenberg_issues_url": "...", "gutenberg_prs_url": "...", "trac_url": "https://core.trac.wordpress.org/tickets/needs-testing", "date": "2026-04-16" }Test plan
🤖 Generated with Claude Code