!WIP! mkfs: copy verity metadata from the --rootdir

[allisonkarlitskaya]

Add a new --fs-verity option. If it's specified, then check if the regular files in the --rootdir have fs-verity enabled, and if they do, copy the metadata into the created filesystem.

We could have done this implicitly on creation of all filesystems, but I believe the new --fs-verity option is justified (over simply always checking) for a number of reasons:

• (most importantly) if querying the fs-verity metadata on a file fails, we want to hear about it, loudly. Querying fs-verity metadata on btrfs itself is only supported on kernel 6.14 and later, and on ext4 fs-verity isn't enabled by default. We want to catch those cases and treat them as errors instead of silently proceeding. If we were to always query fs-verity metadata then we'd have to ignore the unsupported cases.
• tools and scripts invoking mkfs.btrfs and expecting the created filesystem to actually have fs-verity enabled on the relevant files need to be sure of this. We want to fail if the new --fs-verity option isn't present.
• most people are probably not interested in fs-verity data, and querying every single file (when no file has fs-verity enabled) would slow down filesystem creation
• finally, doing this without an option would be a change in existing behaviour

Issue: #929

So far this works, but not for zero-size or inline files.

I'd love some feedback here on the implementation. In particular, the way this fits in with the extents copying code is really awkward. I'd like to share the fd that we open() but we don't open the file if it's empty. I'd also maybe like to share the buffer.

My first thought would be to declare a separate buffer as a global and use it from both data copying and fs-verity copying. This code is not multithreaded and already has a bunch of global variables, so it's not thread-safe anyway. This would save the continuous malloc()/free(). I'd also be happy to use a stack for this sort of thing: it's a large allocation (1MB) but it's in non-recursive application code, so it ought to be OK.

There's also a question if the fs-verity copy should happen inside of the extent-copying function or not. That's where the fd is opened and I'd like to avoid opening the same file twice. I think I might like to create a function which opens the fd and then calls the extent copying code and then the fs-verity copying code separately. It's all a matter of taste and I'd like some input there.

[allisonkarlitskaya]

@kdave ping? would love some help here...

[kdave]

@adam900710 Can you please have a look how the verity data can be added during the mkdir --rootdir? The rest (interface, conditional support, etc) can be dealt with later.

For the first implementation it does not need to be optimal, so if there's extra copying or opening files we can still add it, as it won't affect the default case.

[adam900710]

First thing is, why we need the rootdir to have fs-verity enabled in the first place?
If the host fs doesn't support fs-verity, we can still generate our btrfs specific verity data.

I know you want to be noisy if the host files do not have verity, but one can exactly want to generate a btrfs with verity, on a host fs without fs-verity support (e.g. to generate a distro image).

Secondly, is there any guarantee that the host verity metadata can be copied directly into the host btrfs?
E.g. the host fs is in 64K block size, but the target fs is in 4K block size.

So overall, my thought on the new fs-verity feature (although I'm not 100% sure about some details) is pretty simple, we just want to add the per-inode verity mkfs support, no matter if the host fs supports or not.
There are already the per-inode-flag options added recently, I guess it can be expanded to support per-inode verity.

With that said, I'm not a huge fan of copying verity metadata, even it means faster mkfs.
It's just not generic enough.

But if you want to implement a host-fs independent mkfs verity support, I'm totally fine with that and appreciate the contribute a lot.

[adam900710]

BTW, I didn't see anything docs (the on-disk format doc is full of WIP) related to btrfs' verity support.

Thus if anyone who is familiar with verity, can help enhance the btrfs verity docs in the first place, it will be a huge help.

[allisonkarlitskaya]

This goes to overall approach, so I wrote some comments in the issue: #929 (comment)