Skip to content

fs: add c++ fast path for writeFileSync string utf8#49884

Merged
nodejs-github-bot merged 1 commit intonodejs:mainfrom
CanadaHonk:fast-writefilesync-utf8
Nov 27, 2023
Merged

fs: add c++ fast path for writeFileSync string utf8#49884
nodejs-github-bot merged 1 commit intonodejs:mainfrom
CanadaHonk:fast-writefilesync-utf8

Conversation

@CanadaHonk
Copy link
Member

@CanadaHonk CanadaHonk commented Sep 26, 2023

Summary

Added fast path in almost entirely C++ for writeFileSync with UTF8 encoding and string data. Also improves appendFileSync as it just uses writeFileSync under the hood. Only string data as Buffer seems questionable in benchmarks for this so I'll just leave it for strings for now.

TL;DR: This makes writeFileSync(path, data: string) UTF8 up to ~2.5x faster depending on data size, especially when using file descriptors

Bench results

Benchmark in this PR

Note: running locally on Linux/i9/SSD

                                                                                                                 confidence improvement accuracy (*)   (**)  (***)
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=1024 useFd='false' encoding='utf8'                   -0.37 %       ±4.40% ±5.90% ±7.77%
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=1024 useFd='true' encoding='utf8'            ***     49.15 %       ±5.11% ±6.82% ±8.91%
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=102400 useFd='false' encoding='utf8'         ***      6.69 %       ±1.67% ±2.22% ±2.89%
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=102400 useFd='true' encoding='utf8'          ***     74.52 %       ±4.35% ±5.85% ±7.74%
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=1048576 useFd='false' encoding='utf8'        ***     16.05 %       ±1.60% ±2.14% ±2.81%
fs/bench-writeFileSync.js n=1000 func='writeFile' useBuffer='false' length=1048576 useFd='true' encoding='utf8'         ***     85.82 %       ±3.58% ±4.77% ±6.23%

Benchmark CI (old): https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1421/

Alternative benchmark

Alternative benchmark using Bun's bench for fs.copyFileSync (string data, using paths)

Current (main)

benchmark           time (avg)             (min … max)       p75       p99      p995
------------------------------------------------------ -----------------------------
12 ascii          2.19 µs/iter     (2.07 µs … 2.67 µs)   2.21 µs   2.67 µs   2.67 µs
12 utf8           2.23 µs/iter     (2.08 µs … 2.32 µs)   2.26 µs   2.32 µs   2.32 µs
12288 ascii       6.14 µs/iter     (5.97 µs … 6.62 µs)   6.14 µs   6.62 µs   6.62 µs
18432 utf8       40.02 µs/iter  (36.13 µs … 204.16 µs)  40.05 µs   63.6 µs  69.01 µs

This PR

Long ascii: 6.14µs -> 3.46µs (~1.8x speedup)
Long utf8: 40.02µs -> 18.25µs (~2.2x speedup)

benchmark           time (avg)             (min … max)       p75       p99      p995
------------------------------------------------------ -----------------------------
12 ascii          1.97 µs/iter      (1.9 µs … 2.03 µs)   1.98 µs   2.03 µs   2.03 µs
12 utf8           1.95 µs/iter        (1.91 µs … 2 µs)   1.97 µs      2 µs      2 µs
12288 ascii       3.46 µs/iter      (3.39 µs … 3.6 µs)   3.47 µs    3.6 µs    3.6 µs
18432 utf8       18.25 µs/iter  (16.06 µs … 147.95 µs)  18.48 µs  21.77 µs  24.03 µs

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. performance Issues and PRs related to the performance of Node.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.