Bug 54324 - TestEC2UCSAppliance: 00_checks/46_ntacl_sysvolcheck failing - sysvol-sync.sh fails
TestEC2UCSAppliance: 00_checks/46_ntacl_sysvolcheck failing - sysvol-sync.sh ...
Status: CLOSED FIXED
Product: UCS Test
Classification: Unclassified
Component: Samba
unspecified
Other Linux
: P5 normal (vote)
: ---
Assigned To: Samba maintainers
:
Depends on: 49042
Blocks:
  Show dependency treegraph
 
Reported: 2022-01-11 20:21 CET by Philipp Hahn
Modified: 2023-05-31 09:44 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2022-01-11 20:21:21 CET
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-1/view/Appliances/job/TestEC2UCSAppliance/cfg=master-slave-ec2/lastCompletedBuild/testReport/00_checks/46_ntacl_sysvolcheck/slave/

2022-01-11 19:34:44.20843393 DEBUG: 46_ntacl_sysvolcheck
2022-01-11 19:34:44.20965493 /var/lib/samba/sysvol
2022-01-11 19:34:44.20967993 /var/lib/samba/sysvol/masla.ec2
2022-01-11 19:34:44.20969593 /var/lib/samba/sysvol/masla.ec2/scripts
2022-01-11 19:34:44.20988193 DEBUG: 46_ntacl_sysvolcheck (done)
2022-01-11 19:34:44.60147541 ERROR(<class 'TypeError'>): uncaught exception - (2, 'No such file or directory')
2022-01-11 19:34:44.60365541   File "/usr/lib/python3/dist-packages/samba/netcmd/__init__.py", line 186, in _run
2022-01-11 19:34:44.60368141     return self.run(*args, **kwargs)
2022-01-11 19:34:44.60370141   File "/usr/lib/python3/dist-packages/samba/netcmd/ntacl.py", line 449, in run
2022-01-11 19:34:44.60372241     lp, mask_msad_differences)
2022-01-11 19:34:44.60374141   File "/usr/lib/python3/dist-packages/samba/provision/__init__.py", line 2174, in checksysvolacl
2022-01-11 19:34:44.60375541     direct_db_access, mask_msad_differences)
2022-01-11 19:34:44.60377041   File "/usr/lib/python3/dist-packages/samba/provision/__init__.py", line 2101, in check_gpos_acl
2022-01-11 19:34:44.60378441     direct_db_access=direct_db_access, service=SYSVOL_SERVICE)
2022-01-11 19:34:44.60379941   File "/usr/lib/python3/dist-packages/samba/ntacls.py", line 121, in getntacl
2022-01-11 19:34:44.60381141     xattr.XATTR_NTACL_NAME)
2022-01-11 19:34:44.62288541 error 2022-01-11 19:34:44	 samba-tool ntacl sysvolcheck failed
2022-01-11 19:34:44.62398241 error 2022-01-11 19:34:44	 **************** Test failed above this line (110) ****************


# pdb3 /usr/bin/samba-tool ntacl sysvolcheck
(Pdb) break /usr/lib/python3/dist-packages/samba/ntacls.py:122
(Pdb) run

(Pdb) p file
'/var/lib/samba/sysvol/masla.ec2/Policies'

(Pdb) bt
  /usr/lib/python3.7/bdb.py(585)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()
  /usr/bin/samba-tool(44)<module>()
-> retval = cmd._run("samba-tool", subcommand, *args)
  /usr/lib/python3/dist-packages/samba/netcmd/__init__.py(236)_run()
-> "%s %s" % (myname, subcommand), *args)
  /usr/lib/python3/dist-packages/samba/netcmd/__init__.py(236)_run()
-> "%s %s" % (myname, subcommand), *args)
  /usr/lib/python3/dist-packages/samba/netcmd/__init__.py(186)_run()
-> return self.run(*args, **kwargs)
  /usr/lib/python3/dist-packages/samba/netcmd/ntacl.py(449)run()
-> lp, mask_msad_differences)
  /usr/lib/python3/dist-packages/samba/provision/__init__.py(2174)checksysvolacl()
-> direct_db_access, mask_msad_differences)
  /usr/lib/python3/dist-packages/samba/provision/__init__.py(2101)check_gpos_acl()
-> direct_db_access=direct_db_access, service=SYSVOL_SERVICE)
> /usr/lib/python3/dist-packages/samba/ntacls.py(122)getntacl()
-> xattr.XATTR_NTACL_NAME)

(Pdb) p root_policy_path
'/var/lib/samba/sysvol/masla.ec2/Policies'

2098        root_policy_path = os.path.join(sysvol, dnsdomain, "Policies")

root@slave:~# tree /var/lib/samba/sysvol/masla.ec2/
/var/lib/samba/sysvol/masla.ec2/
└── scripts
1 directory, 0 files


root@master:~# tree /var/lib/samba/sysvol/masla.ec2/
/var/lib/samba/sysvol/masla.ec2/
├── Policies
│   ├── {31B2F340-016D-11D2-945F-00C04FB984F9}
│   │   ├── GPT.INI
│   │   ├── MACHINE
│   │   └── USER
│   └── {6AC1786C-016F-11D2-945F-00C04FB984F9}
│       ├── GPT.INI
│       ├── MACHINE
│       └── USER
└── scripts
8 directories, 2 files


root@slave:~# ucr set samba4/sysvol/sync/debug=yes
root@slave:~# /usr/share/univention-samba4/scripts/sysvol-sync.sh
2022-01-11 20:12:33 DEBUG [master] placing triggerfile.
2022-01-11 20:12:33 DEBUG [master] rsync check for changes on upstream DC
2022-01-11 20:12:34 DEBUG [master] rsync pull from upstream DC
2022-01-11 20:12:34 DEBUG [master] trying to get remote read lock
2022-01-11 20:12:36 DEBUG [master] checking ACL's
2022-01-11 20:12:36 DEBUG [master] local sync from importdir to sysvol
2022-01-11 20:12:36 DEBUG [master] trying to get exclusive (write) lock on local sysvol


- failed first time
- reported changed SSH key second time
- worked 3rd time

> + log ERROR '[master] placing triggerfile with ssh failed with 255. (
> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
> @       WARNING: POSSIBLE DNS SPOOFING DETECTED!          @
> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
> The ECDSA host key for master has changed,
> and the key for the corresponding IP address 10.210.91.203
> is unchanged. This could either mean that
> DNS SPOOFING is happening or the IP address for the host
> and its host key have changed at the same time.
> Offending key for IP in /root/.ssh/known_hosts:2
>   remove with:
>   ssh-keygen -f "/root/.ssh/known_hosts" -R "10.210.91.203"


root@slave:~# wc -l ~/.ssh/known_hosts 
3 /root/.ssh/known_hosts

root@slave:~# ssh-keygen -F master.masla.ec2
# Host master.masla.ec2 found: line 1 
|1|Wcj+PcXRg4nbkwsteP3OTZMb1r4=|JSNyt5DHwcj8J5TI0EObC0pC/ps= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxfBeLPdm+2oEVVfP3jfDzSTAdUSCxG/dhX+qqFsiwIH2RcmPvurP6a6xFlF3FCNTCMob5xdMYnfO/9A//sRE8=

root@slave:~# ssh-keygen -F 10.210.91.203
# Host 10.210.91.203 found: line 2 
|1|+H/DFML8WaFkV/XS5TKE6/LkE6s=|kNvh9HVSz3dQZgjk3IrJ0DWQCxk= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxfBeLPdm+2oEVVfP3jfDzSTAdUSCxG/dhX+qqFsiwIH2RcmPvurP6a6xFlF3FCNTCMob5xdMYnfO/9A//sRE8=

root@slave:~# ssh-keygen -F master
# Host master found: line 3 
|1|frvGwROxp3ugMjlpiBIOkyPyyWw=|kKeST+9ueGJ28qLFNjjj8OaduY4= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBxfBeLPdm+2oEVVfP3jfDzSTAdUSCxG/dhX+qqFsiwIH2RcmPvurP6a6xFlF3FCNTCMob5xdMYnfO/9A//sRE8=
Comment 1 Julia Bremer univentionstaff 2022-01-12 14:04:32 CET
I think we interfered with our debugging yesterday.

I noticed something about the DNS entry being off. 
Being on the slave machine, executing "ssh master" reconnected me to the slave again. 
This ucr variable was set:
hosts/static/127.0.1.1=master.unknown.kvm master

which made it so that the sysvol sync connected to localhost and checked if there was anything to sync. That's why the Policies folder was never synced from master and missing on the slave and samba-tool ntacl sysvolcheck was failing. 

I don't know why that ucr var is set. But it is like that on my machines too. But there the master server is not called "master", so I didn't experience any problems.
Comment 2 Philipp Hahn univentionstaff 2022-01-12 18:10:22 CET
(In reply to Julia Bremer from comment #1)
> I noticed something about the DNS entry being off. 
> Being on the slave machine, executing "ssh master" reconnected me to the
> slave again. 
> This ucr variable was set:
> hosts/static/127.0.1.1=master.unknown.kvm master
...
> I don't know why that ucr var is set. But it is like that on my machines
> too. But there the master server is not called "master", so I didn't
> experience any problems.

That is Bug #49042: Debian added the 127.0.1.1 "hack" to make things work when the hostname has no public IP address associated: <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=316099>
This lead to Problems with kFreeBSD: <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=649747> contains more information.

The explanation for the need is documented here: <https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution>
> The IP address 127.0.1.1 in the second line of this example may not be found on some other Unix-like systems. The Debian Installer creates this entry for a system without a permanent IP address as a workaround for some software (e.g., GNOME) as documented in the bug <https://bugs.debian.org/719621>.

Back than I had to patch "netcfg" for UCS-4.0-0 to keep that "127.0.1.1=$FQHN $HOSTNAME" entry even after "univention-base-files" replaced "/etc/hosts" with the UCR templated version for some reason, but I no longer remember the details.

Maybe we should just try to do a `ucr unset hosts/static/127.0.1.1` and see what breaks?
At least my internal VM is plain wrong:
> # ucr get hosts/static/127.0.1.1 
> master.unknown.kvm master
Comment 3 Philipp Hahn univentionstaff 2023-05-31 08:24:19 CEST
(In reply to Philipp Hahn from comment #2)
> Maybe we should just try to do a `ucr unset hosts/static/127.0.1.1` and see what breaks?

The UCRV has been removed for UCS 5.0-2 via Bug #49042.
Has the test failed since then or can we close this bug now?
Comment 4 Julia Bremer univentionstaff 2023-05-31 09:33:43 CEST
Test has not failed again: We can close the bug :)