Bug 49596

Summary: ldap_extension: validation failed for univention-app.schema
Product: UCS Reporter: Felix Botner <botner>
Component: LDAPAssignee: Felix Botner <botner>
Status: CLOSED FIXED QA Contact: Arvid Requate <requate>
Severity: normal    
Priority: P5 CC: best, damrose, hahn, requate
Version: UCS 4.4   
Target Milestone: UCS 4.4-0-errata   
Hardware: Other   
OS: Linux   
What kind of report is it?: Development Internal What type of bug is this?: ---
Who will be affected by this bug?: --- How will those affected feel about the bug?: ---
User Pain: Enterprise Customer affected?:
School Customer affected?: ISV affected?:
Waiting Support: Flags outvoted (downgraded) after PO Review:
Ticket number: Bug group (optional): Error handling, Troubleshooting
Max CVSS v3 score:
Bug Depends on: 49500    
Bug Blocks:    

Description Felix Botner univentionstaff 2019-06-05 10:13:39 CEST
+++ This bug was initially created as a clone of Bug #49500 +++

Recently the ldap schema extension validation failed in some jenkins test (http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-0/job/AutotestJoin/SambaVersion=s4,Systemrolle=master-part-II/ws/test/autotest-091-master-s4-part-II.log/*view*/).

Unfortunately the output seems to be broken see, python/ldap_extension.py

 # Slapschema doesn't fail on schema errors, errors are 
 # printed to stdout (Bug #45571)
 ud.debug(ud.LISTENER, ud.ERROR, '%s: validation failed:\n%s.' % (name, stdout))

but:

-> slapschema >/dev/null
5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
slapschema: bad configuration file!
-> slapschema 2>/dev/null

So first step is to also print stderr to the log.
Comment 1 Felix Botner univentionstaff 2019-06-05 10:17:10 CEST
That did not help, seen again in the jenkins tests

updating 'cn=apps,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:47.297  LISTENER    ( PROCESS ) : updating 'cn=univention-app,cn=ldapschema,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.
05.06.19 01:01:50.352  LISTENER    ( PROCESS ) : updating 'cn=66univention-appcenter_app,cn=ldapacl,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:50.353  LISTENER    ( PROCESS ) : ldap_extension: cn=66univention-appcenter_app,cn=ldapacl,cn=univention,dc=AutoTest091,dc=local active? ['FALSE']
05.06.19 01:01:51.120  LISTENER    ( ERROR   ) : ldap_extension: slapd.conf validation failed:
5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to clause
5cf6f85f <access clause> ::= access to <what> [ by <who> [ <access> ] [ <control> ] ]+ 
<what> ::= * | dn[.<dnstyle>=<DN>] [filter=<filter>] [attrs=<attrspec>]
<attrspec> ::= <attrname> [val[/<matchingRule>][.<attrstyle>]=<value>] | <attrlist>
<attrlist> ::= <attr> [ , <attrlist> ]
<attr> ::= <attrname> | @<objectClass> | !<objectClass> | entry | children
<who> ::= [ * | anonymous | users | self | dn[.<dnstyle>]=<DN> ]
	[ realanonymous | realusers | realself | realdn[.<dnstyle>]=<DN> ]
	[dnattr=<attrname>]
	[realdnattr=<attrname>]
	[group[/<objectclass>[/<attrname>]][.<style>]=<group>]
	[peername[.<peernamestyle>]=<peer>] [sockname[.<style>]=<name>]
	[domain[.<domainstyle>]=<domain>] [sockurl[.<style>]=<url>]
	[dynacl/<name>[/<options>][.<dynstyle>][=<pattern>]]

No error message, next idea is to enable debug for the slapschema tool in univention-lib/python/ldap_extension.py.

Or has anybody a better idea?
Comment 2 Erik Damrose univentionstaff 2019-06-05 10:30:32 CEST
We briefly discussed it. We could add -v (verbose) and/or -d <level> to increase the debuglevel.

Some interesting points: The slapschema manpage states that errors are already logged to stdout, the previous fix to include stderr will not help add anything useful. In addition, the manpage states
"LIMITATIONS For some backend types, your slapd(8) should not be running (at least, not in read-write mode) when you do this to ensure consistency of the database. It is always safe to run slapschema with the slapd-bdb(5), slapd-hdb(5), and slapd-null(5) backends."
Comment 3 Arvid Requate univentionstaff 2019-06-05 12:54:16 CEST
Can we log the temporary (broken) /etc/ldap/slapd.conf to see line what 249 is?
Comment 4 Arvid Requate univentionstaff 2019-06-05 13:02:01 CEST
Ist this this the exact error message?

> 5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
slapschema: bad configuration file!

That's what I get when I edit /var/lib/univention-ldap/local-schema/univention-app.schema and change the first occurance of "attributetype" to "ttributetype".
Comment 5 Felix Botner univentionstaff 2019-06-05 13:05:33 CEST
(In reply to Arvid Requate from comment #3)
> Can we log the temporary (broken) /etc/ldap/slapd.conf to see line what 249
> is?

The problem is not in line 249. This is the error from the failed ACL registration (because the schema registration failed).

The problem is

app,cn=ldapschema,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.

and we need more information at this point. 
But logging the slapd.conf in this case is maybe a good idea anyway
Comment 6 Felix Botner univentionstaff 2019-06-05 13:06:43 CEST
(In reply to Arvid Requate from comment #4)
> Ist this this the exact error message?
> 
> > 5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
> slapschema: bad configuration file!
> 
> That's what I get when I edit
> /var/lib/univention-ldap/local-schema/univention-app.schema and change the
> first occurance of "attributetype" to "ttributetype".

We have no error message from the fails slapschema.
Comment 7 Florian Best univentionstaff 2019-06-06 09:11:18 CEST
(In reply to Felix Botner from comment #1)
> That did not help, seen again in the jenkins tests

> 5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to
> clause

> No error message, next idea is to enable debug for the slapschema tool in
> univention-lib/python/ldap_extension.py.
> 
> Or has anybody a better idea?

Hm, isn't the error message that the attribute univentionApp is not known? (because the schema is not yet registered?)
Comment 8 Felix Botner univentionstaff 2019-06-06 10:32:06 CEST
(In reply to Florian Best from comment #7)
> (In reply to Felix Botner from comment #1)
> > That did not help, seen again in the jenkins tests
> 
> > 5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to
> > clause
> 
> > No error message, next idea is to enable debug for the slapschema tool in
> > univention-lib/python/ldap_extension.py.
> > 
> > Or has anybody a better idea?
> 
> Hm, isn't the error message that the attribute univentionApp is not known?
> (because the schema is not yet registered?)

No, 

1) register schema

ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.

2) this fails, no univention-app schema

3) register ACL

this fails, with a proper error message  because the schema is missing, but this is a consequence of first error
Comment 9 Felix Botner univentionstaff 2019-06-19 16:01:52 CEST
7e7a28a8337330242279f0d15bb1c65231574b91 - univention-lib
Comment 10 Philipp Hahn univentionstaff 2019-06-19 17:03:21 CEST
(In reply to Felix Botner from comment #9)
> 7e7a28a8337330242279f0d15bb1c65231574b91 - univention-lib

There seems to be one case where "slapschema" exits with != 0 and no output:

servers/slapd/slapschema.c
  161 »···if ( slap_tool_destroy() )
  162 »···»···rc = EXIT_FAILURE;
servers/slapd/slapcommon.c
   939 »···»···»···if ( slap_shutdown( be ))
   940 »···»···»···»···rc = EXIT_FAILURE;
servers/slapd/init.c
  235 »···return backend_shutdown( be );·
servers/slapd/backend.c
   363 »···»···»···rc = be->bd_info->bi_db_close( be, NULL );
   364 »···»···»···if ( rc ) return rc;
...
   368 »···»···»···rc = be->bd_info->bi_close( be->bd_info );
   369 »···»···»···if ( rc ) return rc;

The other case always returns 0:

servers/slapd/slapcommon.c
   942 »···»···if ( slap_destroy())
   943 »···»···»···rc = EXIT_FAILURE;
servers/slapd/init.c
  255 »···rc = backend_destroy();
servers/slapd/backend.c
   529 »···return 0;

If "slapschema" terminates by a signal, it would be logged with a negative return code:

# python -c 'import subprocess,time,os,signal;p=subprocess.Popen(["sh","-c","sleep 10"]);time.sleep(1);os.kill(p.pid,signal.SIGSEGV);out,err=p.communicate();print(p.wait())'
-11
Comment 11 Felix Botner univentionstaff 2019-06-24 11:20:30 CEST
we decided to use slaptest -u instead of slapschema for schema checking. slapschema also checks the actual database and this is not necessary (and error-prone) at this point.

Successful build
Package: univention-lib
Version: 8.0.1-23A~4.4.0.201906241115
Branch: ucs_4.4-0
Scope: errata4.4-0
Comment 13 Arvid Requate univentionstaff 2019-06-26 17:42:57 CEST
<http://errata.software-univention.de/ucs/4.4/168.html>