[Bug?]: Yarn Berry's internal shell component sometimes fails to flush output

[gjvoosten]

Self-service

• I'd be willing to implement a fix

Describe the bug

I have a script in my package.json like:

  "scripts": {
    "test-merge": "./combine-test-results.sh ./test-input/*.xml > ./test-output.xml"
  }

(fully runnable example below)

Repeatedly running:

yarn run test-merge ; tail test-output.xml

should always show:

................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>
</testsuites>

but sometimes the output is cut short.
(We first noticed this when our pipelines would intermittently fail because the generated XML files were invalid.)

If I add another script that forces to use bash:

  "scripts": {
    "test-merge": "./combine-test-results.sh ./test-input/*.xml > ./test-output.xml",
    "test-merge-bash": "bash -c './combine-test-results.sh ./test-input/*.xml > ./test-output.xml'"
  }

and repeatedly run:

yarn run test-merge-bash ; tail test-output.xml

the output is always complete.

This leads me to believe that this is a bug in Yarn Berry's internal shell component, which sometimes fails to flush the output.

To reproduce

berrybug.zip

Environment

System:
    OS: Linux 6.13 Fedora Linux 41 (Workstation Edition)
    CPU: (22) x64 Intel(R) Core(TM) Ultra 9 185H
  Binaries:
    Node: 20.10.0 - /tmp/xfs-49a54522/node
    Yarn: 4.7.0 - /tmp/xfs-49a54522/yarn
    npm: 10.2.3 - /tmp/xfs-49a54522/npm

[gjvoosten]

Still happens with Yarn 4.9.1 😒

berrybug-4.9.1.zip