Univention Bugzilla – Bug 38868
sysvol-sync.sh need error handling
Last modified: 2015-09-23 17:12:46 CEST
Created attachment 7014 [details] sysvol-sync.sh 2015070221000354 I've seen lots of problems when debugging sysvol-sync issues in a customer environment with > 8 downstream DCs with slow and unsteady connections. In different cases the rsync jobs fail for numerous reasons like: rsync: change_dir "/var/lib/samba/sysvol" failed: Permission denied (13) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1526) [Receiver=3.0.7] 2015-07-09 11:55:16: [dcxyz] rsync pull from downstream DC: dcxyz rsync: send_files failed to open "/var/lib/samba/sysvol/xx-net.local/scripts/user/.jniklas.vbs.6jBdib": Permission denied (13) rsync: send_files failed to open "/var/lib/samba/sysvol/xx-net.local/scripts/user/.smichelle.vbs.6EnvLj": Permission denied (13) rsync: send_files failed to open "/var/lib/samba/sysvol/xx-net.local/scripts/user/.tjasmin.vbs.bu4XqG": Permission denied (13) rsync: send_files failed to open "/var/lib/samba/sysvol/xx-net.local/scripts/user/.uregina.vbs.SjlU63": Permission denied (13) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1526) [generator=3.0.7] Jul 09 03:02:49 ssh: connect to host dcxyz port 22: Connection timed out Jul 09 03:02:49 rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] Jul 09 03:02:49 rsync error: unexplained error (code 255) at io.c(601) [Receiver=3.0.7] Mostly for debugging purposes I've created an extended sysvol-sync.sh with the follwing changes/additions: * Improve/add logging (with timestamps) * Does not trash stderr/stdout from rsync * downstream DC's also sync into /var/cache/univention-samba4/sysvol-sync first * check error codes from rsync and don't write to SYSVOL when != 0 * don't write to SYSVOL if a file from remote DC does not have POSIX ACLs For me it seems as if ACLs may get lost when rsync fails because of connection timeout (maybe even because of permission denied) as permissions and ALCs get set _after_ the files have been transferred. With the attached sysvol-sync.sh files with broken ACLs are no longer written to SYSVOL on $samba4_sysvol_sync_host and therefore they don't get synced to downstream DCs.
This sysvol-sync.sh version would be a great improvement. Would have made things much at easier at Ticket#2015080621000363 / Issue#2809
I've added a "--delete" switch to the "pull from parent s4dc" univention-ssh-rsync call so that files removed from upstream dc are removed from the downstream dc's "cold target/importdir" as well. In concrete case files without ACLs had been synced to downstream dc's cold target and that stopped the sync to hot target (as expected). Deleting those files from upstream dc does not re-enable the sync to hot target on downstream dc as the files still exist there. The "don't delete files from downstream dc"-behaviour should stay intact. As we said one should probably also add a check that prevents the script from running in parallel as that may confuse rsync.
To refresh you knowledge about sysvol-sync's wizardry skills see: https://hutten.knut.univention.de/mediawiki/index.php/Samba4_Debugging#Sysvol
*** Bug 33238 has been marked as a duplicate of this bug. ***
* adopted patch (with rsync --delete for "pull from parent s4dc") * error handling (stop/continue if rsync fails or files with no ACL's ) * more logging (log rsync/ssh errors) * use flock to prevent script from running twice * added samba4/sysvol/sync/debug switch to enable additional debug messages * merge to 4.1-0 YAML: 2015-09-21-univention-samba4.yaml
Looks good, but please fix the quoting in log(), maybe like this: log() { local msg="${2//$'\r'/}" builtin echo $(date +"%F %T") "$1" "${msg//$'\n'/}" }
(In reply to Arvid Requate from comment #6) > Looks good, but please fix the quoting in log(), maybe like this: > > log() { > local msg="${2//$'\r'/}" > builtin echo $(date +"%F %T") "$1" "${msg//$'\n'/}" > } OK, updates errata4.0-1, 4.1-0 YAML: 2015-09-21-univention-samba4.yaml
Ok, built and yaml updated.
<http://errata.software-univention.de/ucs/4.0/330.html>