feat(scan): add Workday API support

[thomasrlord]

Detect /wday/cxs/{tenant}/{site}/jobs URLs as a Workday provider via the explicit api: field. fetchWorkdayAll paginates POST requests at limit=20 (Workday's hard cap on most public job APIs) up to a 5000-job safety bound. parseWorkday extracts title and locationsText, and builds the public job URL as host + /en-US/{site} + externalPath.

Unlocks federal contractors that run Workday boards (Leidos, GDIT, Booz Allen Hamilton, RTX, Amentum, Parsons, DXC, Accenture Federal) for zero-token scanning. Backwards compatible — fetchJson takes an optional options parameter (default {}) so the same helper handles both Workday's POST + body and the existing GET semantics for Greenhouse / Ashby / Lever.

Closes #533

What does this PR do?

Related issue

Type of change

• Bug fix
• New feature
• Documentation / translation
• Refactor (no behavior change)

Checklist

• I have read CONTRIBUTING.md
• I linked a related issue above (required for features and architecture changes)
• My PR does not include personal data (CV, email, real names)
• I ran node test-all.mjs and all tests pass
• My changes respect the Data Contract (no modifications to user-layer files)
• My changes align with the project roadmap

---

Questions? Join the Discord for faster feedback.

Summary by CodeRabbit

• New Features

  • Added Workday detection and automatic ingestion of job postings with paginated fetching and a safety cap
  • Scan routing updated so Workday sources use the new paginated ingestion path

• Enhancements

  • Improved Lever detection via explicit API recognition
  • JSON fetching enhanced to accept optional request parameters

[coderabbitai]

📝 Walkthrough

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(scan): add Workday API support' directly and concisely summarizes the main change in the PR, which adds Workday API detection and support to scan.mjs.
Linked Issues check	✅ Passed	All five coding objectives from issue #533 are met: detectApi() recognizes Workday URLs, fetchWorkdayAll() paginates POST requests with correct parameters, parseWorkday() extracts and formats job data, wire-up registers the parser and routes Workday companies correctly, and fetchJson() accepts optional parameters for backward compatibility.
Out of Scope Changes check	✅ Passed	All changes in scan.mjs are directly scoped to the Workday API support feature: detection logic, fetch implementation, parsing, wire-up, and backwards-compatible fetchJson extension—no unrelated modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scan.mjs`:
- Around line 149-166: fetchWorkdayAll can silently return partial results when
the pagination cap is hit; update the function (fetchWorkdayAll) to detect when
the loop exits due to maxPages being reached and signal that the results were
truncated (e.g., add a returned flag like _truncated: true and include
_pagesFetched or _itemsFetched), and also emit a warning (console.warn or
existing logger) when postings.length === limit on the final iteration and page
=== maxPages - 1 so operators can see the cap was hit.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 90d96e3c-cdb5-4a92-8c7d-ac6ca20e74d7

📥 Commits

Reviewing files that changed from the base of the PR and between 62b767d and 43d365812cc298d44b992b7301ea34e13efedf1b.

📒 Files selected for processing (1)

• scan.mjs

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scan.mjs`:
- Around line 156-165: The loop that reads Workday responses currently uses
`const postings = json.jobPostings || []`, which masks unexpected payloads;
update the code (inside the loop where `fetchJson` is called) to validate that
`json.jobPostings` exists and is an Array and throw a descriptive Error if it is
missing or not an array so the failure surfaces into `errors` instead of
silently producing zero jobs; reference the `fetchJson` call and the
`postings`/`allPostings` handling to add this runtime check and throw (include
tenant/context in the error message) before pushing into `allPostings`.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2df78ce3-66f2-47f4-ab8f-b0165ac11c64

📥 Commits

Reviewing files that changed from the base of the PR and between 43d365812cc298d44b992b7301ea34e13efedf1b and 3df0671.

📒 Files selected for processing (1)

• scan.mjs

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Fail fast on unexpected Workday payloads.

json.jobPostings || [] turns any 200-level error payload or schema drift into a silent “0 jobs” result. That undercounts the tenant and hides the failure from the summary. Throw if jobPostings is not an array so this lands in errors instead.

💡 Proposed fix

   for (let page = 0; page < maxPages; page++) {
     const json = await fetchJson(url, {
       method: 'POST',
       headers: { 'Content-Type': 'application/json' },
       body: JSON.stringify({ appliedFacets: {}, limit, offset: page * limit, searchText: '' }),
     });
-    const postings = json.jobPostings || [];
+    if (!Array.isArray(json.jobPostings)) {
+      throw new Error(`Unexpected Workday response shape from ${url}: missing jobPostings array`);
+    }
+    const postings = json.jobPostings;
     pages = page + 1;
     if (postings.length === 0) break;

As per coding guidelines, **/*.mjs: Check for command injection, path traversal, and SSRF. Ensure scripts handle missing data/ directories gracefully.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@scan.mjs` around lines 156 - 165, The loop that reads Workday responses
currently uses `const postings = json.jobPostings || []`, which masks unexpected
payloads; update the code (inside the loop where `fetchJson` is called) to
validate that `json.jobPostings` exists and is an Array and throw a descriptive
Error if it is missing or not an array so the failure surfaces into `errors`
instead of silently producing zero jobs; reference the `fetchJson` call and the
`postings`/`allPostings` handling to add this runtime check and throw (include
tenant/context in the error message) before pushing into `allPostings`.

[thomasrlord]

Addressed in 3df0671. fetchWorkdayAll now tracks lastPageSize and pages and emits a stderr warning when we exit via the maxPages bound
with a still-full last page (the only case where truncation actually happened). Skipped the _truncated/_pagesFetched/_itemsFetched
metadata fields since there's no consumer for them yet — happy to add them if you want them surfaced in the summary line.