Print `borg create --list --dry-run` to STDOUT to allow combination with `--paths-from-stdin`

[mlell]

Currently, borg seems to list files to STDERR, which prevents its use in UNIX pipes. Of course, I could redirect 2>&1 but it would also include error messages and password prompts. The primary purpose of borg create --list --dry-run ... is to list files, so one could argue that therefore this should go to STDOUT and not STDERR. Also, this allows to use the borg pattern matching, then processing these somehow and finally pipe them back to borg create ... --paths-from-stdin.

My use-case is the following: I am using the pcloud remote of rclone to back up my data. This is slow when many small files are involved. As a test I arbitrarily tried to upload the git repo of opendataeditor and saw from vnstat -l that upload speed is only a few kbit/s. As it was noted that multiple parallel borg2 processes can concurrently write to the same repo, I did exactly that.

However, these processes tended to "clump together" uploading the same files at the same time, which is why I wanted to shuffle the file order so that the different upload processes do not interfere as much. This is where borg create --list --dry-run ... | shuf would become handy.

[ThomasWaldmann]

borg uses logging a lot and all that output goes to stderr by default.

that is done like this because some commands / options have data flowing over stdout.

[bigntallmike]

What might be handy is to add optional formatting to that log output so the specifics the user wants to react to are easier to parse. Someone could implement an option like --log-json to 'tag' all the logging output, allowing programs to filter for what they need automatically.
< not a request />