Skip to content

Bug 2024449 - Add importer for streamed profile format (JSON Lines)#5901

Open
fqueze wants to merge 1 commit intofirefox-devtools:mainfrom
fqueze:import-streamed-profile
Open

Bug 2024449 - Add importer for streamed profile format (JSON Lines)#5901
fqueze wants to merge 1 commit intofirefox-devtools:mainfrom
fqueze:import-streamed-profile

Conversation

@fqueze
Copy link
Copy Markdown
Contributor

@fqueze fqueze commented Mar 20, 2026

Add support for importing the "streamed profile" format, a JSON Lines format where each line is a JSON object: a meta line, a thread declaration line, then marker lines. This format is produced by tools like resourcemonitor.py for streaming resource-usage profiles (with the patches in bug 2024449).

The importer passes through meta and thread objects as-is and respects the input's preprocessedProfileVersion, so the standard profile upgraders handle all format migrations. stringArray placement is version-aware: per-thread for versions < 56, in profile.shared for versions >= 56.

Also adds early detection in _extractJsonFromArrayBuffer so that JSON Lines content served as application/json reaches the string-based format detection instead of failing JSON.parse.

Example profiles on the timed out xpcshell jobs of this try push.

@mstange
Copy link
Copy Markdown
Contributor

mstange commented Mar 20, 2026

Also adds early detection in _extractJsonFromArrayBuffer so that JSON Lines content served as application/json reaches the string-based format detection instead of failing JSON.parse.

😧

@fqueze fqueze force-pushed the import-streamed-profile branch from 2c0c6f0 to fb8a404 Compare March 20, 2026 14:37
@mstange
Copy link
Copy Markdown
Contributor

mstange commented Mar 20, 2026

I insist that if it's not valid JSON, it should not be served as application/json. I don't want to maintain workarounds for this.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026

Codecov Report

❌ Patch coverage is 92.30769% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.46%. Comparing base (8ac20f5) to head (bcf0a07).
⚠️ Report is 26 commits behind head on main.

Files with missing lines Patch % Lines
src/profile-logic/import/streamed-profile.ts 93.47% 3 Missing ⚠️
src/actions/receive-profile.ts 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5901      +/-   ##
==========================================
+ Coverage   85.44%   85.46%   +0.02%     
==========================================
  Files         321      322       +1     
  Lines       32061    32132      +71     
  Branches     8753     8858     +105     
==========================================
+ Hits        27393    27463      +70     
- Misses       4236     4237       +1     
  Partials      432      432              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add support for importing the "streamed profile" format, a JSON Lines
format where each line is a JSON object: a meta line, a thread
declaration line, then marker lines. This format is produced by tools
like resourcemonitor.py for streaming resource-usage profiles.

The importer passes through meta and thread objects as-is and respects
the input's preprocessedProfileVersion, so the standard profile
upgraders handle all format migrations. stringArray placement is
version-aware: per-thread for versions < 56, in profile.shared for
versions >= 56.

Also adds early detection in _extractJsonFromArrayBuffer so that JSON
Lines content reaches the string-based format detection instead of
failing JSON.parse.
@fqueze fqueze force-pushed the import-streamed-profile branch from fb8a404 to bcf0a07 Compare March 20, 2026 15:00
@fqueze
Copy link
Copy Markdown
Contributor Author

fqueze commented Mar 20, 2026

Also adds early detection in _extractJsonFromArrayBuffer so that JSON Lines content served as application/json reaches the string-based format detection instead of failing JSON.parse.

😧

Sorry, this part of the commit message was misleading, the check is not for application/json but for /\.json/ which matches both json and jsonl.

I don't want to maintain workarounds for this.

To me it's not a workaround, just a simple if check:

if (text.startsWith('{"type":"meta"')) {
  return text;
}

I don't see how that could grow into more workarounds in the future.

The part where I'm the least confident for the future is about how we'll handle profile format changes for the parts that aren't part of what's streamed as lines. The profiler.shared object specifically is difficult to stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants