Bug 45033 - installing a schema and ACL in one ucs_registerLDAPExtension call will create error with the ACL on slave
installing a schema and ACL in one ucs_registerLDAPExtension call will create...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Join (univention-join)
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.3-0-errata
Assigned To: Felix Botner
Arvid Requate
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-18 13:27 CEST by Daniel Tröder
Modified: 2018-05-16 17:03 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.343
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
qa-feedback.patch (1.14 KB, patch)
2018-05-15 13:20 CEST, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2017-07-18 13:27:21 CEST
When running this on a DC master:

ucs_registerLDAPExtension "$@" \
    --schema   /usr/share/univention-ox-user-migration/univention-ox-user-migration.schema \
    --udm_hook /usr/share/univention-ox-user-migration/ox_pim_migration_udm_hook.py
    --acl      /usr/share/univention-ox-user-migration/60univention-ox-user-migration.acl || die

.. the join script succeeds and the slapd on the DC master restarts correctly with schema and ACL applied.

But when this is replicated to a DC slave, the following is in the listener.log:

=========================================================================
12.07.17 21:19:00.497  LISTENER    ( PROCESS ) : updating 'cn=univention-ox-user-migration,cn=services,cn=univention,dc=quality,dc=ucs' command a
12.07.17 21:19:00.588  LISTENER    ( PROCESS ) : updating 'cn=master141,cn=dc,cn=computers,dc=quality,dc=ucs' command m
12.07.17 21:19:02.091  LISTENER    ( PROCESS ) : updating 'cn=univention-ox-user-migration,cn=ldapschema,cn=univention,dc=quality,dc=ucs' command a
12.07.17 21:19:02.243  LISTENER    ( PROCESS ) : updating 'cn=60univention-ox-user-migration,cn=ldapacl,cn=univention,dc=quality,dc=ucs' command a
12.07.17 21:19:03.170  LISTENER    ( ERROR   ) : ldap_extension: slapd.conf validation failed:
59667627 /etc/ldap/slapd.conf: line 114: unknown attr "oxMigrationStatus" in to clause
59667627 <access clause> ::= access to <what> [ by <who> [ <access> ] [ <control> ] ]+ 
<what> ::= * | dn[.<dnstyle>=<DN>] [filter=<filter>] [attrs=<attrspec>]
<attrspec> ::= <attrname> [val[/<matchingRule>][.<attrstyle>]=<value>] | <attrlist>
<attrlist> ::= <attr> [ , <attrlist> ]
<attr> ::= <attrname> | @<objectClass> | !<objectClass> | entry | children
<who> ::= [ * | anonymous | users | self | dn[.<dnstyle>]=<DN> ]
        [ realanonymous | realusers | realself | realdn[.<dnstyle>]=<DN> ]
        [dnattr=<attrname>]
        [realdnattr=<attrname>]
        [group[/<objectclass>[/<attrname>]][.<style>]=<group>]
        [peername[.<peernamestyle>]=<peer>] [sockname[.<style>]=<name>]
        [domain[.<domainstyle>]=<domain>] [sockurl[.<style>]=<url>]
        [dynacl/<name>[/<options>][.<dynstyle>][=<pattern>]]
        [ssf=<n>] [transport_ssf=<n>] [tls_ssf=<n>] [sasl_ssf=<n>]
<style> ::= exact | regex | base(Object)
<dnstyle> ::= base(Object) | one(level) | sub(tree) | children | exact | regex
<attrstyle> ::= exact | regex | base(Object) | one(level) | sub(tree) | children
<peernamestyle> ::= exact | regex | ip | ipv6 | path
<domainstyle> ::= exact | regex | base(Object) | sub(tree)
<access> ::= [[real]self]{<level>|<priv>}
<level> ::= none|disclose|auth|compare|search|read|{write|add|delete}|manage
<priv> ::= {=|+|-}{0|d|x|c|s|r|{w|a|z}|m}+
<control> ::= [ stop | continue | break ]
dynacl:
        <name>=ACI      <pattern>=<attrname>

slaptest: bad configuration file!
.
12.07.17 21:19:03.171  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /etc/univention/templates/files/etc/ldap/slapd.conf.d/60univention-ox-user-migration.acl.
=========================================================================

Rerunning the join script on the master doesn't fix it, because no change to the data happens.

As a workaround, the join script has been changed to first install the schema and the install the ACL:

ucs_registerLDAPExtension "$@" \
    --schema   /usr/share/univention-ox-user-migration/univention-ox-user-migration.schema \
    --udm_hook /usr/share/univention-ox-user-migration/ox_pim_migration_udm_hook.py || die

ucs_registerLDAPExtension "$@" \
    --acl      /usr/share/univention-ox-user-migration/60univention-ox-user-migration.acl || die


And that works.
Comment 1 Stefan Gohmann univentionstaff 2017-09-14 07:01:55 CEST
@Daniel, did it happen in a school customer environment or happened it during the development?
Comment 2 Florian Best univentionstaff 2017-09-15 09:22:45 CEST
So the order in which the objects are created is wrong?
Comment 3 Daniel Tröder univentionstaff 2017-09-18 09:57:09 CEST
(In reply to Stefan Gohmann from comment #1)
> @Daniel, did it happen in a school customer environment or happened it
> during the development?
I cannot remember the exact scenario. But the most likely one is that it was in a UCS@school domain and the slave was a non-school slave, so it can serve mail for the whole domain.

Yes, it happend during development for a OX-user migration script that needed an extended attribute and ACL:
svn/dev/branches/ucs-4.1/customers/*/univention-ox-user-migration/
Comment 4 Stefan Gohmann univentionstaff 2017-09-18 10:04:06 CEST
(In reply to Daniel Tröder from comment #3)
> Yes, it happend during development for a OX-user migration script that
> needed an extended attribute and ACL:

Thanks, I'll remove the school customer affected flag.
Comment 5 Florian Best univentionstaff 2017-09-20 14:51:25 CEST
Shouldn't we just say WONTFIX and force developers to use 2 bash commands if they want to configure things which depend on each other?
Comment 6 Daniel Tröder univentionstaff 2017-09-20 22:18:10 CEST
The syntax of ucs_registerLDAPExtension (see http://docs.software-univention.de/developer-reference-4.2.html#join:libraries:shell) clearly allows this. Either change the syntax (and require 2 calls) or fix the function. I think the function should be fixed.
It would need to check if both --schema and --acl are present and then split the command.
Comment 7 Daniel Tröder univentionstaff 2018-03-01 17:30:05 CET
*** Bug 46453 has been marked as a duplicate of this bug. ***
Comment 8 Felix Botner univentionstaff 2018-04-24 17:01:57 CEST
OK, can be reproduced with 

# ucs master
-> ucs_registerLDAPExtension --schema /opt/ox.schema  \
   --acl /opt/acl.acl --packagename pa --packageversion 1

# ucs backup/slave
listener.log:
...
12.07.17 21:19:03.170  LISTENER    ( ERROR   ) : ldap_extension: slapd.conf validation failed:
59667627 /etc/ldap/slapd.conf: line 114: unknown attr "attr" in to clause
59667627 <access clause> ::= access to <what> [ by <who> [ <access> ] [ <control> ] ]+ 
<what> ::= * | dn[.<dnstyle>=<DN>] [filter=<filter>] [attr
...

Problem:

During the registration with "ucs_registerLDAPExtension" two objects are created, one schema and one acl extension object. 

On the master the 
* listener for ldap extensions registers a new schema object
* writes a new schema file and includes this file in the slapd.conf
* then the listener registers the new acl object, creates/includes the new
  config in the slapd.conf and checks the slapd config with slaptest. 
* this works even without a slapd restart (which has not happened at this point)
  because the master uses the previously generated schema file directly
* so both the schema and acl are now correctly registered, last
  step is to restart the local slapd and to mark the extensions as 
  active in ldap

But, on a backup/slave this fails
* the listener registers a new schema, but due to our schema replication
  and replica ldap server config the local slapd never uses this local 
  schema file, the schema is automatically generated from the ldap master's 
  schema during the replication
  (the backup does not get the new schema until the slapd is restarted
  on the master)
* now the new acl object is registered, included in the slapd.conf and checked
  with slapdtest, which fails because the schema is unknown at this point
* and this is the problem, acl extensions on non master systems are included
  in the slapd.conf before the slapd is restarted  on the master 
  (before the schema replication)

fix in univention-lib/python/ldap_extension.py UniventionLDAPACL:
* on non-master systems ignore all new acl extensions, every extensions is 
  marked as active on the master and so an additional modification is
  replicated to non-master systems for these acl objects, and this is the right
  time to locally register the extension (now we can be sure the slapd on the
  master has been restarted and a possible schema replication has happened)
* something similar happens during update, the extension is mark as
  non-active, checked and included on the master, slapd is restarted on the
  master and the extension is marked as active again.
  On non-master we now ignore changes (old and new) if new is not 
  activated

->  ucs_registerLDAPExtension --schema /opt/ox.schema  --acl /opt/acl.acl --packagename pa --packageversion 1

this worked for me now, on master/backup, ucs-test-ldap also green

univention-lib
4b6c5a2d06ae0028dd742756b07b4cd2b413678c

yaml:
4b6c5a2d06ae0028dd742756b07b4cd2b413678c

one additional commit to let the listener module restart the slapd on non-master systems after an acl extension has been removed

univention-lib
07674f0c7e72b75dd83e5d29c260f38e4272ccd7

yaml
673ad904e54219464c77131b1212d183963702aa
Comment 9 Quality Assurance univentionstaff 2018-05-04 16:43:46 CEST
--- mirror/ftp/4.3/unmaintained/component/4.3-0-errata/source/univention-lib_7.0.0-4A~4.3.0.201803222006.dsc
+++ apt/ucs_4.3-0-errata4.3-0/source/univention-lib_7.0.0-6A~4.3.0.201804241649.dsc
@@ -1,6 +1,14 @@
-7.0.0-4A~4.3.0.201803222006 [Thu, 22 Mar 2018 20:06:26 +0100] Univention builddaemon <buildd@univention.de>:
+7.0.0-6A~4.3.0.201804241649 [Tue, 24 Apr 2018 16:49:25 +0200] Univention builddaemon <buildd@univention.de>:
 
   * UCS auto build. No patches were applied to the original source package
+
+7.0.0-6 [Tue, 24 Apr 2018 16:48:22 +0200] Felix Botner <botner@univention.de>:
+
+  * Bug #45033: reload slapd on non-master system after acl remove
+
+7.0.0-5 [Tue, 24 Apr 2018 14:29:31 +0200] Felix Botner <botner@univention.de>:
+
+  * Bug #45033: Fixed ldap acl registration for non-master systems
 
 7.0.0-4 [Thu, 22 Mar 2018 19:53:02 +0100] Arvid Requate <requate@univention.de>:
Comment 10 Arvid Requate univentionstaff 2018-05-15 13:20:30 CEST
Created attachment 9530 [details]
qa-feedback.patch

The commit 07674f0c7e72b75dd83e5d29c260f38e4272ccd7 makes things really hard to understand. That code block probably is a useless optimization anyway, so let's rather remove that.
Comment 11 Felix Botner univentionstaff 2018-05-15 13:53:30 CEST
(In reply to Arvid Requate from comment #10)
> Created attachment 9530 [details]
> qa-feedback.patch
> 
> The commit 07674f0c7e72b75dd83e5d29c260f38e4272ccd7 makes things really hard
> to understand. That code block probably is a useless optimization anyway, so
> let's rather remove that.

done
Comment 12 Arvid Requate univentionstaff 2018-05-16 13:43:08 CEST
Ok, works and advisory looks good too.
Comment 13 Arvid Requate univentionstaff 2018-05-16 17:03:55 CEST
<http://errata.software-univention.de/ucs/4.3/37.html>