NC | NSFS | List Objects with `s3 ls` of a Bucket Shows an Hidden Directory (Probably not in Normal Flow)

[shirady]

Environment info

• NooBaa Version: 5.19
• Platform: NC

Actual behavior

1. When using the AWS CLI of list objects with s3 ls I could see the hidden internal directory (PRE .noobaa-nsfs_67d7ed5f51c5043305fee625/). Note that the bucket ID is 67e2accc57cf31f3f0d80f31.

Expected behavior

1. When using the AWS CLI of list objects with s3 ls and not see any hidden internal directory.

Steps to reproduce

I'm not sure what exactly I did but I can guess that I reused the same path for creating a bucket with the name bucket-01, and then deleting it (configuration-wise - delete with sudo rm -rf /etc/noobaa.conf.d/) and created the bucket again bucket-01 with noobaa-cli In the same path with the same name I had the previous hidden folder so it shows it as an item.

ls -al /Users/buckets/bucket-01

total 8
drwxrwxrwx   5 shiradymnik  admin  160 Mar 25 21:39 .
drwxrwxrwx  19 root         admin  608 Mar 26 17:39 ..
drwxrwx---   3 1001         admin   96 Mar 17 11:40 .noobaa-nsfs_67d7ed5f51c5043305fee625
drwxrwx---   3 1001         admin   96 Mar 25 15:36 .noobaa-nsfs_67e2accc57cf31f3f0d80f31
-rw-rw----@  1 1001         admin   13 Mar 25 21:39 hello_world1.txt

67e2accc57cf31f3f0d80f31 the bucket ID that was created with noobaa-cli in that time,
67d7ed5f51c5043305fee625 probably old bucket ID that was deleted from the config, but it's underlying storage was not deleted.

More information - Screenshots / Logs / Other output

nc-user-1-s3 s3 ls s3://bucket-01

                           PRE .noobaa-nsfs_67d7ed5f51c5043305fee625/
2025-03-25 21:39:28         13 hello_world1.txt

The outputs of list-objects-v2 and list-objects are as expected (no hidden folder appears):
nc-user-1-s3 s3api list-objects --bucket bucket-01

{
    "Contents": [
        {
            "Key": "hello_world1.txt",
            "LastModified": "2025-03-25T19:39:28+00:00",
            "ETag": "\"mtime-d8ploctaemf4-ino-3v0xtu\"",
            "Size": 13,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "shira-1001",
                "ID": "67e2acb6a091ac33941ee437"
            }
        }
    ],
    "RequestCharged": null
}

nc-user-1-s3 s3api list-objects-v2 --bucket bucket-01

{
    "Contents": [
        {
            "Key": "hello_world1.txt",
            "LastModified": "2025-03-25T19:39:28+00:00",
            "ETag": "\"mtime-d8ploctaemf4-ino-3v0xtu\"",
            "Size": 13,
            "StorageClass": "STANDARD"
        }
    ],
    "RequestCharged": null
}

[achouhan09]

If we create a bucket and then remove the configuration using the command sudo rm -rf /etc/noobaa.conf.d/, the folder associated with the bucket (e.g., bucket-01 in Shira's example) will not be deleted from the new_buckets_path.
As a result, if we try to recreate the bucket with the same name, the AWS CLI command returns an error such as BucketAlreadyExists or BucketNameNotAvailable.
Therefore, I believe we should also prevent bucket creation via the NooBaa CLI if a folder with the same bucket name already exists.

wdyt? @romayalon @nadavMiz

[romayalon]

Hey @achouhan09
The purpose of creation of buckets via NooBaa CLI is to export existing paths while the S3 create bucket flow will create the file system path.
You can read more about it here -
https://github.com/noobaa/noobaa-core/blob/master/docs/NooBaaNonContainerized/AccountsAndBuckets.md#s3-bucket-vs-cli-exported-bucket
IMO we don't want to create this kind of limitation.