Bug 39539 - timeout for univention-dnsedit - maximum recursion depth exceeded - join hangs
timeout for univention-dnsedit - maximum recursion depth exceeded - join hangs
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM - CLI
UCS 5.0
Other Linux
: P5 normal (vote)
: UCS 5.0-0-errata
Assigned To: Philipp Hahn
Florian Best
:
: 45110 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-14 17:07 CEST by Philipp Hahn
Modified: 2021-08-04 16:25 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.206
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2015121821000547
Bug group (optional): Error handling
Max CVSS v3 score:
hahn: Patch_Available+


Attachments
patch (9.58 KB, patch)
2017-08-22 17:02 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2015-10-14 17:07:03 CEST
slapd failed to start on DC Master, provisioning hangs since 5 days:

/usr/share/univention-admin-tools/univention-dnsedit: timeout while trying to contact LDAP server master.XXX
...
/usr/share/univention-admin-tools/univention-dnsedit: timeout while trying to contact LDAP server master.XXX
Traceback (most recent call last):
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 400, in <module>
    main()
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 351, in main
    lo, position = bind()
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 155, in bind
    return bind()  # recursion
...
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 155, in bind
    return bind()  # recursion
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 141, in bind
    lo, position = uldap.getAdminConnection(start_tls)
  File "/usr/lib/pymodules/python2.6/univention/admin/uldap.py", line 70, in getAdminConnection
    lo=univention.uldap.getAdminConnection(start_tls, decode_ignorelist=decode_ignorelist)
  File "/usr/lib/pymodules/python2.6/univention/uldap.py", line 73, in getAdminConnection
    lo=access(host=ucr['ldap/master'], port=port, base=ucr['ldap/base'], binddn='cn=admin,'+ucr['ldap/base'], bindpw=bindpw, start_tls=start_tls, decode_ignorelist=decode_ignorelist, reconnect=reconnect)
  File "/usr/lib/pymodules/python2.6/univention/uldap.py", line 184, in __init__
    self.__open(ca_certfile)
  File "/usr/lib/pymodules/python2.6/univention/uldap.py", line 216, in __open
    self.lo = ldap.ldapobject.ReconnectLDAPObject(self.uri, trace_stack_limit=None, retry_max=self.client_connection_attempt, retry_delay=1)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 692, in __init__
    SimpleLDAPObject.__init__(self,uri,trace_level,trace_file,trace_stack_limit)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 71, in __init__
    self.protocol_version = ldap.VERSION3
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 118, in __setattr__
    self.set_option(self.CLASSATTR_OPTION_MAPPING[name],value)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 777, in set_option
    SimpleLDAPObject.set_option(self,option,invalue)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 568, in set_option
    return self._ldap_call(self._l.set_option,option,invalue)
RuntimeError: maximum recursion depth exceeded
Comment 1 Stefan Gohmann univentionstaff 2015-12-22 09:24:07 CET
A similar situation during a rejoin of a school slave. At least the join got stuck for about 15 or 20 minutes because the LDAP server was not reachable.

We should use the default uldap timeout.

Ticket #2015121821000547
Comment 2 Philipp Hahn univentionstaff 2017-08-18 10:21:35 CEST
*** Bug 45110 has been marked as a duplicate of this bug. ***
Comment 3 Philipp Hahn univentionstaff 2017-08-18 10:25:20 CEST
Again <http://jenkins.knut.univention.de:8080/job/UCSschool%204.2/job/UCSschool%204.2%20Singleserver/ImportTests=NoImportTests,SambaVersion=s4-with-slave/160/>

/var/log/univention/join-secondary-samba4.log:
**************************************************************************
* Join failed!                                                           *
* Contact your system administrator                                      *
**************************************************************************
* Message:  Warning: UCS version on master201.autotest201.local is lower (4.20) than local version (4.21). This constellation is not supported.
**************************************************************************
**************************************************************************
* INFO: In case 98univention-samba4slavepdc-dns.inst failed, this is ok
*       for rejoins and will be retried in a couple of seconds below.
**************************************************************************
Restarting samba-ad-dc (via systemctl): samba-ad-dc.service.
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
ldap_start_tls: Can't contact LDAP server (-1)
Checking my SID '' in the remote Samba4:
ERROR: SID '' not found associated with slave202 in Samba4 on 10.210.79.92
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server None
...
Traceback (most recent call last):
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 403, in <module>
    main()
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 354, in main
    lo, position = bind()
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 155, in bind
    return bind()  # recursion
...
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 138, in bind
    start_tls=start_tls)
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 330, in __init__
    self.lo = univention.uldap.access(host, port, base, binddn, bindpw, start_tls, follow_referral=follow_referral)
  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 152, in __init__
    self.__open(ca_certfile)
  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 187, in __open
    self.lo = ldap.ldapobject.ReconnectLDAPObject(self.uri, trace_stack_limit=None, retry_max=self.client_connection_attempt, retry_delay=1)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 773, in __init__
    SimpleLDAPObject.__init__(self,uri,trace_level,trace_file,trace_stack_limit)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 79, in __init__
    self.protocol_version = ldap.VERSION3
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 125, in __setattr__
    self.set_option(self.CLASSATTR_OPTION_MAPPING[name],value)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 871, in set_option
    return SimpleLDAPObject.set_option(self,option,invalue)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 614, in set_option
    return self._ldap_call(self._l.set_option,option,invalue)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 92, in _ldap_call
    self._ldap_object_lock.acquire()
RuntimeError: maximum recursion depth exceeded


Cost:
 - 2 EC2 VMS running 1d11h
 - /me ½h for debugging that again
Comment 4 Florian Best univentionstaff 2017-08-22 17:02:45 CEST
Created attachment 9137 [details]
patch

Attached a patch which fixes the evaluation of the timeout/retry-loop (which is set to two minutes). I will apply it when the next UDM errata update is released.
Comment 5 Stefan Gohmann univentionstaff 2017-08-22 18:42:38 CEST
I don't think the bug flags are correct. If this issue is fixed, the join still fails. Right?
Comment 6 Philipp Hahn univentionstaff 2017-08-25 16:40:43 CEST
(In reply to Stefan Gohmann from comment #5)
> I don't think the bug flags are correct. If this issue is fixed, the join
> still fails. Right?

Yes, but a failing command with an error message is better than no response in 15h:

>univention-join: joins a computer to an ucs domain
>copyright (c) 2001-2017 Univention GmbH, Germany
>
>Search DC Master: ESC[60Gdone
>Check DC Master: 
>
>**************************************************************************
>* Join failed!                                                           *
>* Contact your system administrator                                      *
>**************************************************************************
>* Message:  Warning: UCS version on master201.autotest201.local is lower (4.20) than local version (4.21). This constellatio
>**************************************************************************
>**************************************************************************
>* INFO: In case 98univention-samba4slavepdc-dns.inst failed, this is ok
>*       for rejoins and will be retried in a couple of seconds below.
>**************************************************************************
>Restarting samba-ad-dc (via systemctl): samba-ad-dc.service.
>ldap_start_tls: Can't contact LDAP server (-1)
...
>Checking my SID '' in the remote Samba4:
>ERROR: SID '' not found associated with slave202 in Samba4 on 10.210.65.154
>/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server None


# cat /var/log/univention/join.log
>Thu Aug 24 18:42:13 EDT 2017: starting /usr/share/univention-join/univention-join -dcaccount Administrator -dcpwd /tmp/tmp.WoQjEqQXy3
>Warning: Permanently added 'master201.autotest201.local,10.210.65.154' (RSA) to the list of known hosts.
>running version check
>Thu Aug 24 18:42:15 EDT 2017: finish /usr/share/univention-join/univention-join
>Thu Aug 24 18:50:07 EDT 2017: starting /usr/sbin/univention-join -dcaccount Administrator -dcpwd /tmp/tmp.ZaWMcaOd1J/dcpwd
>running version check
>Thu Aug 24 18:50:08 EDT 2017: finish /usr/sbin/univention-join


Slave is stuck running
 /root/ucs-school-join-secondary-samba4 --dcaccount Administrator --dcpwd /tmp/univention --rootpassword /tmp/univention 10.210.65.154

Running it with "bash -x":
...
>+ univention-ssh /tmp/tmp.E1tImpjEHg/rootpwd root@10.210.65.154 '/usr/sbin/ucr set samba4/service/drepl=true; /usr/share/univention-samba4/scripts/register_ldb_module.py -H /var/lib/samba/private/sam.ldb --ignore-exists --remove=univention_samaccountname_ldap_check && /etc/init.d/samba-ad-dc restart; /etc/init.d/univention-s4-connector restart'
>Setting samba4/service/drepl
>Multifile: /etc/samba/smb.conf
>Restarting samba-ad-dc (via systemctl): samba-ad-dc.service.
>Restarting univention-s4-connector (via systemctl): univention-s4-connector.service.
>+ univention-join -dcaccount Administrator -dcpwd /tmp/tmp.E1tImpjEHg/dcpwd
>univention-join: joins a computer to an ucs domain
>copyright (c) 2001-2017 Univention GmbH, Germany
>
>Search DC Master: ^[[60Gdone
>Check DC Master:
>
>**************************************************************************
>* Join failed!                                                           *
>* Contact your system administrator                                      *
>**************************************************************************
>* Message:  Warning: UCS version on master201.autotest201.local is lower (4.20) than local version (4.21). This constellation is not supported.


root@slave202:~# ucr search --brief ^version/
version/erratalevel: 52
version/patchlevel: 1
version/releasename: Lesum
version/version: 4.2

root@master201:~# ucr search --brief ^version/
version/erratalevel: 0
version/patchlevel: 0
version/releasename: Lesum
version/version: 4.2


master:/var/log/univention/updater.log
>Checking for package status: rH  linux-image-4.1.0-ucs174-amd64                      4.1.6-1.174.201602110938                       amd64        Linux 4.1 for 64-bit PCs
>failed
>ERROR: The package state on this system is inconsistent.
>       Please run 'dpkg --configure -a' manually
>Error: Update aborted by pre-update script of release 4.2-1


master:/var/log/apt/term.log
>Removing linux-image-4.1.0-ucs174-amd64-signed (2.0.0-6.15.201602111457) ...
>Removing linux-image-4.1.0-ucs174-amd64 (4.1.6-1.174.201602110938) ...
>/etc/kernel/postrm.d/initramfs-tools:
>update-initramfs: Deleting /boot/initrd.img-4.1.0-ucs174-amd64
>/etc/kernel/postrm.d/zz-update-grub:
>Generating grub configuration file ...
...
>  No volume groups found
>done
>dpkg: error processing package linux-image-4.1.0-ucs174-amd64 (--remove):
> subprocess installed post-removal script returned error exit status 128
Comment 7 Philipp Hahn univentionstaff 2017-08-25 17:30:53 CEST
And yet another one:

# find /etc/univention/ssl -type f -ls
# ldapsearch -LLL -o ldif-wrap=no -ZZ -h $(ucr get ldap/master) -p 7389 -x -D $(ucr get ldap/hostdn) -y /etc/machine.secret -b $(ucr get ldap/base) -s base
ldap_start_tls: Connect error (-11)
# less /var/log/univention/join.log
...
2017-08-24 17:40:21.949210011-04:00 (in joinscript_init)
Starting ldap server(s): slapd ...failed.
599f47c6 /etc/ldap/slapd.conf: line 110: unknown attr "@univentionVirtualMachine" in to clause 599f47c6 <access clause> ::= 
invoke-rc.d: initscript slapd, action "start" failed.
...
RUNNING 00ucs-school-app-version-check.inst
2017-08-24 19:47:56.170429901-04:00 (in joinscript_init)
Traceback (most recent call last):
  File "/usr/share/ucs-school-metapackage/scripts/app_version.py", line 51, in <module>
    info = domain.to_dict([app])[0]
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/domain.py", line 97, in to_dict
    lo, pos = self._get_ldap_connection(args=None, allow_machine_connection=True)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/credentials.py", line 177, in _get_ldap_connection
    return self._get_machine_connection()
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/credentials.py", line 146, in _get_machine_connection
    return get_machine_connection()
  File "/usr/lib/pymodules/python2.7/univention/appcenter/udm.py", line 139, in get_machine_connection
    return getMachineConnection()
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 147, in getMachineConnection
    lo = univention.uldap.getMachineConnection(start_tls, decode_ignorelist=decode_ignorelist, ldap_master=ldap_master)
  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 84, in getMachineConnection
    return access(host=ucr['ldap/master'], port=port, base=ucr['ldap/base'], binddn=ucr['ldap/hostdn'], bindpw=bindpw, start
  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 150, in __init__
    self.__open(ca_certfile)
  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 185, in __open
    self.lo.start_tls_s()
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 884, in start_tls_s
    res = self._apply_method_s(SimpleLDAPObject.start_tls_s,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 860, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 571, in start_tls_s
    return self._ldap_call(self._l.start_tls_s)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
    result = func(*args,**kwargs)
ldap.CONNECT_ERROR: {'desc': 'Connect error'}
****************************************************************************
The version of the UCS@school app on the DC master is lower than the version
on this host. Please consult the Univention Support Database on how to
proceed: http://sdb.univention.de/1374
****************************************************************************
EXITCODE=1
...
RUNNING 03univention-directory-listener.inst
2017-08-24 19:48:01.537302593-04:00 (in joinscript_init)
Setting ldap/database/ldbm/dbsync
Multifile: /etc/ldap/slapd.conf
24.08.17 19:48:02.652  DEBUG_INIT
24.08.17 19:48:02.663  LDAP        ( ERROR   ) : start_tls: Connect error
24.08.17 19:48:02.663  LISTENER    ( WARN    ) : can not connect to LDAP server master208.autotest208.local:7389
24.08.17 19:48:02.663  LISTENER    ( ERROR   ) : can not connect any server, exit
Setting ldap/database/ldbm/dbsync
Multifile: /etc/ldap/slapd.conf
Restarting ldap server(s).
Stopping ldap server(s): slapd ...done.
Starting ldap server(s): slapd ...failed.
.
EXITCODE=1
...
RUNNING 05univention-bind.inst
2017-08-24 19:48:19.299937951-04:00 (in joinscript_init)
wait for named ?
/usr/share/univention-admin-tools/univention-dnsedit: timeout while trying to contact LDAP server master208.autotest208.loca
...


# less /var/log/univention/listener.log
...
24.08.17 18:12:06.391  LISTENER    ( ERROR   ) : connection to notifier was closed
24.08.17 18:12:06.391  LISTENER    ( ERROR   ) : failed to recv result
24.08.17 18:12:06.391  LISTENER    ( ERROR   ) : listener: 1
Reloading /etc/samba/smb.conf: smbd.
Exporting directories for NFS kernel daemon....
Starting NFS kernel daemon: nfsd mountd.
Re-exporting directories for NFS kernel daemon....
UNIVENTION_DEBUG_BEGIN  : uldap.__open host=slave2081.autotest208.local port=7389 base=dc=autotest208,dc=local
UNIVENTION_DEBUG_END    : uldap.__open host=slave2081.autotest208.local port=7389 base=dc=autotest208,dc=local
Multifile: /etc/samba/smb.conf
24.08.17 18:12:12.553  DEBUG_INIT
24.08.17 18:12:12.572  LDAP        ( ERROR   ) : ldap_simple_bind: Invalid credentials
24.08.17 18:12:12.572  LISTENER    ( WARN    ) : can not connect to LDAP server master208.autotest208.local:7389
24.08.17 18:12:12.572  LISTENER    ( WARN    ) : can not connect any server, retrying in 30 seconds
24.08.17 18:12:43.019  LISTENER    ( WARN    ) : chosen server: master208.autotest208.local:7389
24.08.17 18:12:43.085  LDAP        ( ERROR   ) : start_tls: Can't contact LDAP server
24.08.17 18:12:43.085  LISTENER    ( WARN    ) : can not connect to LDAP server master208.autotest208.local:7389
24.08.17 18:12:43.085  LISTENER    ( WARN    ) : can not connect any server, retrying in 30 seconds
24.08.17 18:12:46.700  LISTENER    ( WARN    ) : received signal 15


# univention-directory-listener-ctrl status
Listener status:
 down: univention-directory-listener: 60292s
Current Notifier ID on "master208.autotest208.local"
 6943
cat: /var/lib/univention-directory-listener/notifier_id: Datei oder Verzeichnis nicht gefunden
Last Notifier ID processed by local Listener:
Last transaction processed:
Modules:
0       bind    /usr/lib/univention-directory-listener/system/bind.py
0       cups-pdf        /usr/lib/univention-directory-listener/system/cups-pdf.py
0       cups-printers   /usr/lib/univention-directory-listener/system/cups-printers.py
0       faillog /usr/lib/univention-directory-listener/system/faillog.py
0       gencertificate  /usr/lib/univention-directory-listener/system/gencertificate.py
0       hosteddomains   /usr/lib/univention-directory-listener/system/hosteddomains.py
0       keytab-member   /usr/lib/univention-directory-listener/system/keytab-member.py
0       keytab  /usr/lib/univention-directory-listener/system/keytab.py
0       ldap_extension  /usr/lib/univention-directory-listener/system/ldap_extension.py
0       ldap_server     /usr/lib/univention-directory-listener/system/ldap_server.py
0       license_uuid    /usr/lib/univention-directory-listener/system/license_uuid.py
0       nagios-client   /usr/lib/univention-directory-listener/system/nagios-client.py
0       nfs-homes       /usr/lib/univention-directory-listener/system/nfs-homes.py
0       nfs-shares      /usr/lib/univention-directory-listener/system/nfs-shares.py
0       nscd_update     /usr/lib/univention-directory-listener/system/nscd.py
0       nss     /usr/lib/univention-directory-listener/system/nss.py
0       pkgdb-watch     /usr/lib/univention-directory-listener/system/pkgdb-watch.py
0       pupilgroups     /usr/lib/univention-directory-listener/system/pupilgroups.py
0       quota   /usr/lib/univention-directory-listener/system/quota.py
0       remove-old-homedirs     /usr/lib/univention-directory-listener/system/remove-old-homedirs.py
0       remove-old-sharedirs    /usr/lib/univention-directory-listener/system/remove-old-sharedirs.py
0       replication     /usr/lib/univention-directory-listener/system/replication.py
0       s4-connector    /usr/lib/univention-directory-listener/system/s4-connector.py
0       samba4-idmap    /usr/lib/univention-directory-listener/system/samba4-idmap.py
0       samba-shares    /usr/lib/univention-directory-listener/system/samba-shares.py
0       ucsschool-s4-branch-site        /usr/lib/univention-directory-listener/system/ucsschool-s4-branch-site.py
0       ucs-school-user-logonscript     /usr/lib/univention-directory-listener/system/ucs-school-user-logonscript.py
0       udm_extension   /usr/lib/univention-directory-listener/system/udm_extension.py
0       umc-service-providers   /usr/lib/univention-directory-listener/system/umc-service-providers.py
0       univention-saml-servers /usr/lib/univention-directory-listener/system/univention-saml-servers.py
0       well-known-sid-name-mapping     /usr/lib/univention-directory-listener/system/well-known-sid-name-mapping.py



# ps www 2739
  PID TTY      STAT   TIME COMMAND
 2739 ?        S      0:00 runsvdir -P /etc/service log: v univention-reload-service: fatal: unable to start ./run: file does not exist runsv univention-reload-service: fatal: unable to start ./run: file does not exist runsv univention-reload-service: fatal: unable to start ./run: file does not exist runsv univention-reload-service: fatal: unable to start ./run: file does not exist ...................................................................


# sv status univention-directory-listener
down: univention-directory-listener: 60460s


# /var/log/univention/join.log
>Thu Aug 24 18:12:13 EDT 2017: starting /usr/sbin/univention-join -dcaccount Administrator -dcpwd /tmp/tmpcbEpvQ
>running version check
...
>univention-server-join: joins a server to an univention domain
>copyright (c) 2001-2017 Univention GmbH, Germany
>        
>ldap_dn="cn=slave2081,cn=dc,cn=server,cn=computers,ou=School1,dc=autotest208,dc=local"
>Setting hostname
>Setting ldap/hostdn
>Multifile: /etc/postfix/ldap.distlist
>Multifile: /etc/ldap/slapd.conf
>File: /etc/welcome.msg
>Multifile: /etc/postfix/ldap.virtualwithcanonical
>File: /etc/pam_ldap.conf
>File: /etc/issue
>Multifile: /etc/postfix/ldap.virtual
>Multifile: /etc/hosts
>Multifile: /etc/postfix/ldap.groups
>File: /etc/dhcp/dhclient.conf
>File: /etc/apache2/conf.d/ucs.conf
>Multifile: /etc/postfix/ldap.canonicalrecipient
>Multifile: /etc/postfix/ldap.transport
>File: /etc/libnss-ldap.conf
>Multifile: /etc/postfix/ldap.canonicalsender
>Multifile: /etc/postfix/ldap.saslusermapping
>Multifile: /etc/postfix/ldap.virtualdomains
>Module: ucs-school-webproxy
>Delayed reload triggered
>File: /etc/cups/cupsd.conf
>Multifile: /etc/postfix/ldap.sharedfolderlocal
>File: /etc/logrotate.d/univention-samba4
>File: /etc/logrotate.d/winbind
>File: /etc/cron.d/univention-directory-policy
>File: /etc/mailname
>File: /etc/pam.d/smtp
>Multifile: /etc/apache2/sites-available/default-ssl
>Multifile: /etc/postfix/main.cf
>Multifile: /etc/postfix/ldap.sharedfolderremote
>File: /etc/hostname
>Multifile: /etc/samba/smb.conf
>File: /etc/default/squid3
>File: /var/www/ucs-overview/entries.json
>File: /etc/squid3/squid.conf
>File: /var/www/proxy.pac
>Multifile: /etc/pam.d/univention-management-console

sv force-stop univention-directory-listener

>ok: down: univention-directory-listener: 0s

univention-scp "$DCPWD" -r "${DCACCOUNT}@${DCNAME}:/var/lib/heimdal-kdc/*" /var/lib/heimdal-kdc/ >>/var/log/univention/join.log 2>&1

>ssh: Could not resolve hostname master208.autotest208.local: Name or service not known

/usr/sbin/univention-join:821
>Setting ldap/server/name
>Setting ldap/server/ip
>Not updating ldap/server/port
>Setting ldap/master
>Not updating ldap/master/port
>Setting ldap/server/type
>Multifile: /etc/ldap/slapd.conf
>Multifile: /etc/postfix/ldap.virtualwithcanonical
>File: /etc/pam_ldap.conf
>File: /etc/krb5.conf
>File: /etc/cups/client.conf
>Multifile: /etc/postfix/ldap.transport
>Multifile: /etc/postfix/ldap.canonicalrecipient
>Multifile: /etc/postfix/ldap.virtual
>File: /etc/libnss-ldap.conf
>Multifile: /etc/postfix/ldap.saslusermapping
>Multifile: /etc/postfix/ldap.virtualdomains
>Multifile: /etc/postfix/ldap.distlist
>Multifile: /etc/postfix/ldap.groups
>File: /etc/pam.d/smtp
>Multifile: /etc/postfix/ldap.sharedfolderlocal
>File: /etc/init.d/slapd
>File: /etc/default/ntpdate
>File: /etc/nagios/nrpe.cfg
>Multifile: /etc/postfix/ldap.sharedfolderremote
>File: /etc/ntp.conf
>Multifile: /etc/postfix/ldap.canonicalsender
>File: /etc/squid3/squid.conf
>File: /etc/ldap/ldap.conf
>Clearing symlinks in /etc/ssl/certs...done.
>Updating certificates in /etc/ssl/certs... 174 added, 0 removed; done.
>Running hooks in /etc/ca-certificates/update.d....done.

/usr/sbin/univention-join:830 setup_ssl force
/usr/sbin/univention-join:127 setup_ssl () {
        if [ ! -d "/etc/univention/ssl/$hostname" ] &&  [ ! -d "/etc/univention/ssl/$hostname.$domainname" ]; then
                echo "failed to get host certificate"
                failed_message "failed to get host certificate"

trapOnExit()

>Thu Aug 24 18:13:10 EDT 2017: finish /usr/sbin/univention-join
...
>RUNNING 00ucs-school-app-version-check.inst
>2017-08-24 19:47:56.170429901-04:00 (in joinscript_init)
>Traceback (most recent call last):
>  File "/usr/share/ucs-school-metapackage/scripts/app_version.py", line 51, in <module>
>    info = domain.to_dict([app])[0]
>  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/domain.py", line 97, in to_dict
>    lo, pos = self._get_ldap_connection(args=None, allow_machine_connection=True)
>  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/credentials.py", line 177, in _get_ldap_connection
>    return self._get_machine_connection()
>  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/credentials.py", line 146, in _get_machine_connection
>    return get_machine_connection()
>  File "/usr/lib/pymodules/python2.7/univention/appcenter/udm.py", line 139, in get_machine_connection
>    return getMachineConnection()
>  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 147, in getMachineConnection
>    lo = univention.uldap.getMachineConnection(start_tls, decode_ignorelist=decode_ignorelist, ldap_master=ldap_master)
>  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 84, in getMachineConnection
>    return access(host=ucr['ldap/master'], port=port, base=ucr['ldap/base'], binddn=ucr['ldap/hostdn'], bindpw=bindpw, start_tls=start_tls, decode_ignorelist=decode_ignorelist, reconnect=reconnect)
>  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 150, in __init__
>    self.__open(ca_certfile)
>  File "/usr/lib/pymodules/python2.7/univention/uldap.py", line 185, in __open
>    self.lo.start_tls_s()
>  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 884, in start_tls_s
>    res = self._apply_method_s(SimpleLDAPObject.start_tls_s,*args,**kwargs)
>  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 860, in _apply_method_s
>    return func(self,*args,**kwargs)
>  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 571, in start_tls_s
>    return self._ldap_call(self._l.start_tls_s)
>  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
>    result = func(*args,**kwargs)
>ldap.CONNECT_ERROR: {'desc': 'Connect error'}




# cat /var/log/univention/config-registry.replog
...
2017-08-24 18:12:15: set dns/backend=ldap old:samba4
2017-08-24 18:12:25: set ldap/base=dc=autotest208,dc=local old:dc=autotest208,dc=local
2017-08-24 18:12:32: set hostname=slave2081 old:slave2081
2017-08-24 18:12:32: set ldap/hostdn=cn=slave2081,cn=dc,cn=server,cn=computers,ou=School1,dc=autotest208,dc=local old:cn=slave2081,cn=dc,cn=computers,dc=autotest208,dc=local
2017-08-24 18:12:51: set ldap/server/type=slave old:slave
2017-08-24 18:12:51: set ldap/master=master208.autotest208.local old:master208.autotest208.local
2017-08-24 18:12:51: set ldap/server/ip=10.210.250.0 old:10.210.250.0
2017-08-24 18:12:51: set ldap/server/name=slave2081.autotest208.local old:slave2081.autotest208.local
...


# less ./univention/management-console-module-schoolinstaller.log
24.08.17 18:13:10.387  MODULE      ( PROCESS ) : **************************************************************************
24.08.17 18:13:10.387  MODULE      ( PROCESS ) : * Join failed!                                                           *
24.08.17 18:13:10.388  MODULE      ( PROCESS ) : * Contact your system administrator                                      *
24.08.17 18:13:10.388  MODULE      ( PROCESS ) : **************************************************************************
24.08.17 18:13:10.388  MODULE      ( PROCESS ) : * Message:  Establishing a TLS connection with master208.autotest208.local failed. Maybe you didn't specify a FQDN.
Comment 8 Philipp Hahn univentionstaff 2018-10-15 13:57:11 CEST
Again: Customer 26 while upgrading from UCS-4.2 to UCS-4.3
Comment 9 Philipp Hahn univentionstaff 2021-07-19 14:02:43 CEST
Jenkins U@S test is blocked for 7 days:
https://jenkins.knut.univention.de:8181/job/UCSschool-5.0/job/Upgrade%20Multiserver/Config=s4-all-components,TestGroup=base1/13/console

# ps axfu
root      1902  0.0  0.0 144196  3836 ?        Ss   Jul12   0:13 sshd: root@notty
root      1913  0.0  0.0  13564  1552 ?        Ss   Jul12   0:00  \_ bash -c . utils.sh; run_setup_join_on_non_master
root      1921  0.0  0.0  13696  1604 ?        S    Jul12   0:00      \_ /bin/bash /usr/lib/univention-system-setup/scripts/setup-join.sh --dcaccount Administrator --password_file /tmp/univention
root     27882  0.0  0.0  13696   836 ?        S    Jul12   0:00      |   \_ /bin/bash /usr/lib/univention-system-setup/scripts/setup-join.sh --dcaccount Administrator --password_file /tmp/univention
root     27886  0.0  0.0  14276  1368 ?        S    Jul12   0:00      |   |   \_ /bin/bash /usr/share/univention-join/univention-join -dcaccount Administrator -dcpwd /tmp/tmp.VFYuNaW8bS
root      5814  0.0  0.0  14168  1424 ?        S    Jul12   0:00      |   |       \_ /bin/bash /usr/lib/univention-install/98univention-samba4-dns.inst --binddn uid=Administrator,cn=users,dc=autotest208,dc=local --bindpwdfile /tmp/tmp.KXKa3PLJf3/dcpwd
root      6141  0.0  0.0  13756  2208 ?        S    Jul12   0:00      |   |           \_ /bin/bash /usr/share/univention-samba4/scripts/setup-dns-in-ucsldap.sh --binddn uid=Administrator,cn=users,dc=autotest208,dc=local --bindpwdfile /tmp/tmp.KXKa3PLJf3/dcpwd --dc
root      7003  0.0  3.5 617600 142180 ?       S    08:54   0:08      |   |               \_ /usr/bin/python2.7 /usr/share/univention-directory-manager-tools/univention-dnsedit --binddn uid=Administrator,cn=users,dc=autotest208,dc=local --bindpwdfile /tmp/tmp.KXKa3PLJf3/
root     27883  0.0  0.0  13696  1084 ?        S    Jul12   0:00      |   \_ /bin/bash /usr/lib/univention-system-setup/scripts/setup-join.sh --dcaccount Administrator --password_file /tmp/univention
root      1922  0.0  0.0   7368   352 ?        S    Jul12   0:01      \_ tee -a /var/log/univention/setup.log


# lsof -p 7003 -a -d 0-255
COMMAND    PID USER   FD   TYPE  DEVICE SIZE/OFF    NODE NAME
univentio 7003 root    0r  FIFO    0,10      0t0   22664 pipe
univentio 7003 root    1w   REG   253,0  5312837 2097275 /var/log/univention/join.log
univentio 7003 root    2w   REG   253,0  5312837 2097275 /var/log/univention/join.log
univentio 7003 root    3u   REG   253,0    88353 2113158 /var/log/univention/directory-manager-cmd.log
univentio 7003 root    4u  IPv4 4632970      0t0     TCP slave2081.autotest208.local:35386->master208.autotest208.local:7389 (SYN_SENT)
univentio 7003 root    7r   CHR     1,9      0t0    1033 /dev/urandom



The master crashed and is no longer running:
> qemu: qemu_thread_create: Resource temporarily unavailable
> 2021-07-11 22:55:17.241+0000: shutting down, reason=crashed


# apt-get install python2.7-dbg libc6-dbg gdb
# gdb -p 7003
(gdb) frame 2
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 169, in bind
    time.sleep(10)
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 170, in bind
    return bind()  # recursion
...
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 404, in main
    lo, position = bind()
  File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 455, in <module>
    main()
Comment 11 Philipp Hahn univentionstaff 2021-07-22 13:55:15 CEST
[5.0-0] 00b5c4288d refactor[dns-edit] Use ArgumentParser features
 management/univention-directory-manager-modules/univention-dnsedit | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

[5.0-0] b236b08057 style[dns-edit] Remove exit() after raise
 management/univention-directory-manager-modules/univention-dnsedit | 2 --
 1 file changed, 2 deletions(-)

[5.0-0] 957564e384 fix[dns-edit] Timeout handling
 doc/errata/staging/univention-directory-manager-modules.yaml    | 29 ++++++-------
 .../univention-directory-manager-modules/debian/changelog       |  6 +++
 .../univention-directory-manager-modules/univention-dnsedit     | 63 ++++++++++++++---------------
 3 files changed, 52 insertions(+), 46 deletions(-)

[5.0-0] e73d7dd206 refactor[dns-edit] Use UCR singleton
 management/univention-directory-manager-modules/univention-dnsedit | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

[5.0-0] f24d8d2700 feat[dns-edit] Log exception message
 management/univention-directory-manager-modules/univention-dnsedit | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

[5.0-0] 49ac2fe6af style[dns-edit] Convert to Python if-else
 management/univention-directory-manager-modules/univention-dnsedit | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)


Package: univention-directory-manager-modules
Version: 15.0.11-13A~5.0.0.202107221344
Branch: ucs_5.0-0
Scope: errata5.0-0

[5.0-0] b11b3a2d1c Bug #51776: ssl, Bug #53339: udm
 doc/errata/staging/univention-directory-manager-modules.yaml | 2 +-
 doc/errata/staging/univention-ssl.yaml                       | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Comment 12 Florian Best univentionstaff 2021-07-23 13:26:00 CEST
OK: default timeout handling of 120 (always +10/20) seconds has been repaired:
root@master80:~# time /usr/share/univention-directory-manager-tools/univention-dnsedit --binddn cn=admin,l=school,l=dev --bindpwd univention school.dev add txt                                                                               
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}         
/usr/share/univention-directory-manager-tools/univention-dnsedit: timeout while trying to contact LDAP server master80.school.dev: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'}

real    2m10,640s
user    0m0,446s
sys     0m0,070s
root@master80:~# echo $?
1
OK: new --timeout parameter
OK: code review
~OK: YAML