Bug 57529 - Rsync files are missing the ACL's after successfully sync - Filesystem xfs to ext4
Summary: Rsync files are missing the ACL's after successfully sync - Filesystem xfs to...
Status: CLOSED FIXED
Alias: None
Product: UCS
Classification: Unclassified
Component: General
Version: UCS 5.0
Hardware: Other Linux
: P5 normal
Target Milestone: UCS 5.0-9-errata
Assignee: Arvid Requate
QA Contact: Julia Bremer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-08-20 07:47 CEST by Mirac Erdemiroglu
Modified: 2024-10-07 14:53 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.086
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2024080221000121
Bug group (optional): Workaround is available
Customer ID:
Max CVSS v3 score:


Attachments
bug57529-xfs-rsync-xattr.patch (1.24 KB, patch)
2024-08-28 12:53 CEST, Mirac Erdemiroglu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mirac Erdemiroglu univentionstaff 2024-08-20 07:47:29 CEST
A customer has updated his system and installed kernel 5.10. Since then he has had problems with the Sysvol sync and missing ACLs after the files have been successfully synced. Due to the rsync, the files lose their ACLs and can therefore not be used.

The following error message can be seen in the sysvol.log:
2024-08-18 00:20:48 ERROR [dc02] some files from dc02 don't have ACLs set. Will not sync to hot target!


The interesting thing about the customer environment is that the DC master has formatted the file system in xfs. 
With filesystems installed as ext4, these rsync errors do not occur with kernel 5.10.

In our tests, we therefore booted an earlier kernel 4.19 and performed another rsync from xfs to ext4 and from xfs to xfs. 
No errors occurred and all files had the ACL's set correctly.
Comment 3 Mirac Erdemiroglu univentionstaff 2024-08-20 07:53:44 CEST
Informations from our testings and my test environment.



It seems that the kernel is to blame for the ACL's being lost when rsyncing to the xfs filesystem.
On the primary I ran rsync -aAX /var/lib/samba/sysvol /mnt/ and the files no longer had ACL's set.

root@ucs5primary:/mnt# xfs_info /mnt/
meta-data=/dev/sdb1              isize=512    agcount=4, agsize=131072 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=524288, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =Internes Protokoll     bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =keine                  extsz=4096   blocks=0, rtextents=0


root@ucs5primary:/mnt# df -T
Dateisystem             Typ      1K-Blöcke Benutzt Verfügbar Verw% Eingehängt auf
udev                    devtmpfs   4063772       0   4063772    0% /dev
tmpfs                   tmpfs       817036   16056    800980    2% /run
/dev/mapper/vg_ucs-root ext4      19746772 6279096  12589636   34% /
tmpfs                   tmpfs      4085176       0   4085176    0% /dev/shm
tmpfs                   tmpfs         5120       0      5120    0% /run/lock
tmpfs                   tmpfs      4085176       0   4085176    0% /sys/fs/cgroup
/dev/sda1               ext4        472408  201974    241404   46% /boot
/dev/sdb1               xfs        2086912   35252   2051660    2% /mnt


Loaded Kernel 5.10.0-0.deb10.30-amd64.

root@ucs5primary:~# rsync -aAX /mnt/sysvol/univention.intranet/Policies/ root@ucs5backup:/mnt/
root@ucs5primary:~# cd /mnt/

root@ucs5primary:/mnt# ls -lah
insgesamt 8,0K
drwxr-xr-x  3 root          root             20 Aug 16 15:22 .
drwxr-xr-x 18 root          root           4,0K Aug 16 15:30 ..
drwxrwx---  3 Administrator Administrators   33 Aug 16 14:48 sysvol

root@ucs5primary:/mnt# rm -rf sysvol/

root@ucs5primary:/mnt# ls -lah
insgesamt 4,0K
drwxr-xr-x  2 root root    6 Aug 16 15:46 .
drwxr-xr-x 18 root root 4,0K Aug 16 15:30 ..


After reboot with and older kernel version:

root@ucs5primary:/mnt# uname -a
Linux ucs5primary 4.19.0-27-amd64 #1 SMP Debian 4.19.316-1 (2024-06-25) x86_64 GNU/Linux


root@ucs5primary:/mnt# ls -lah /var/lib/samba/sysvol/univention.intranet/
insgesamt 32K
drwxrwx---+ 4 Administrator Administrators 4,0K Aug 16 14:48 .
drwxrwx---+ 3 Administrator Administrators 4,0K Aug 16 14:48 ..
drwxrwx---+ 4 Administrator Administrators 4,0K Aug 16 14:48 Policies
drwxrwx---+ 2 Administrator Administrators 4,0K Aug 16 14:48 scripts

root@ucs5primary:/mnt# rsync -aAX /var/lib/samba/sysvol/univention.intranet/ /mnt/

root@ucs5primary:/mnt# ls -lah
insgesamt 16K
drwxrwx---+  4 Administrator Administrators   37 Aug 16 14:48 .
drwxr-xr-x  18 root          root           4,0K Aug 16 15:30 ..
drwxrwx---+  4 Administrator Administrators   98 Aug 16 14:48 Policies
drwxrwx---+  2 Administrator Administrators    6 Aug 16 14:48 scripts

root@ucs5primary:/mnt# ls -lah Policies/
insgesamt 16K
drwxrwx---+ 4 Administrator Administrators 98 Aug 16 14:48 .
drwxrwx---+ 4 Administrator Administrators 37 Aug 16 14:48 ..
drwxrwx---+ 4 Administrator Domain Admins  48 Aug 16 14:48 {31B2F340-016D-11D2-945F-00C04FB984F9}
drwxrwx---+ 4 Administrator Domain Admins  48 Aug 16 14:48 {6AC1786C-016F-11D2-945F-00C04FB984F9}

root@ucs5primary:/mnt# getfacl Policies/\{31B2F340-016D-11D2-945F-00C04FB984F9\}/
# file: Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/
# owner: Administrator 
# group: Domain\040Admins
user::rwx
user:Administrator:rwx 
user:5010:r-x
user:5035:rwx
group::rwx
group:Authenticated\040Users:r-x
group:System:rwx
group:Administrators:rwx
group:Server\040Operators:r-x
mask::rwx
other::---
default:user::rwx
default:user:Administrator:rwx
default:user:5010:r-x  
default:user:5035:rwx  
default:group::---
default:group:Authenticated\040Users:r-x
default:group:System:rwx
default:group:Administrators:rwx
default:group:Server\040Operators:r-x
default:mask::rwx
default:other::---
Comment 4 Mirac Erdemiroglu univentionstaff 2024-08-22 15:10:44 CEST
Workaround is to reboot an older kernel version on both systems where rsync will be used. 
In my case this was the kernel:
root@ucs5primary:/mnt# uname -a
Linux ucs5primary 4.19.0-27-amd64 #1 SMP Debian 4.19.316-1 (2024-06-25) x86_64 GNU/Linux
Comment 6 Arvid Requate univentionstaff 2024-08-28 11:44:49 CEST
Since updating from Kernel 4.19 to Kernel 5.10 the behavior of xfs
seems to have changed with respect to the handling of xattrs.
As a symptom, rsync -AX seems to remove trusted.SGI_ACL_FILE when
synchronizing from the sysvol from a system with an ext4 partition.
Looks similar to https://github.com/RsyncProject/rsync/issues/301
and the workaround listed there almost works, adjusted to our use
case to "--filter='-x! security.NTACL'". but it caused problems
in combination with --delete-excluded, which then removed all "excluded"
xattrs (i.e. everything except security.NTACL) on the recipient side.
According to the man page, the use of that --delete-excluded option
modifies the behavior of --filter to only apply server side, but not
recipient side any longer. The documentation of the --xattrs option
mentions that under certain circumstances, the --filter rules need
to be explicitly extended, so now we also add the xattr exclude filter
rule explicitly for the recipient side, like "--filter='-xr! security.NTACL'".
That seems to work, customer reported positive feedback.
Comment 7 Arvid Requate univentionstaff 2024-08-28 12:01:21 CEST
For explanation: The getfacl calls above only show a symptom. The fACLs are stored in extended attributes (aka xattrs) that seem to be file system specific. ext4 apparently stores them in system.posix_acl_access and system.posix_acl_default while xfs stores them in trusted.SGI_ACL_FILE and trusted.SGI_ACL_DEFAULT (see https://lore.kernel.org/all/20151027113133.GA26600@bfoster.bfoster/T/

OTOH Samba also uses the xattr security.NTACL to store the NTACLs (UCS configures smb.conf to do it that way) and sysvol-sync.sh uses rsync with the option -X/--xattrs to sync the extended attributes. It's still unclear to me how this can have worked without problems between ext4 and xfs under Linux Kernel 4.19, but apparently something changed in the interaction of xfs under Linux Kernel 5.10 and rsync and that causes rsync to delete the xfs specific xattrs trusted.SGI_ACL_{FILE,DEFAULT} which destroys the basis for the lookup of fACLs on the target xfs system.

In case other people face issues like this, it's helpful to look at the xattrs:

EXAMPLE_FILE="/var/lib/samba/sysvol/$(hostname -d)/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/MACHINE"

getfattr -m - "$EXAMPLE_FILE" # -d ## optionally show values

We now have an adjusted version of sysvol-sync.sh making use of the advanced rsync filter options, which will probably be released as an errata update.
Comment 8 Mirac Erdemiroglu univentionstaff 2024-08-28 12:53:36 CEST
Created attachment 11230 [details]
bug57529-xfs-rsync-xattr.patch

Patch is in the attachment.

For migration the patch, download it and use the following command:

patch -p0 -d / < bug57529-xfs-rsync-xattr.patch
Comment 9 Arvid Requate univentionstaff 2024-09-30 16:33:56 CEST
09f6e9551a | filter xattrs during sysvolsync

Package: univention-samba4
Version: 9.0.18-2
Branch: 5.0-0
Scope: errata5.0-9
Comment 10 Julia Bremer univentionstaff 2024-10-01 21:21:16 CEST
OK: Sysvol sync uses filter option to filter out every xattr that is not security.NTACL and thus inhibits the other xattr to be overwritten/deleted during sync.
OK: Sync between xfs and ext4 on primary backup
OK: Manual run of ucs-test on an xfs /ext4 mixed system
OK: Jenkins
OK: YAML 
Verified