Bug 49596 - ldap_extension: validation failed for univention-app.schema
ldap_extension: validation failed for univention-app.schema
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: LDAP
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4-0-errata
Assigned To: Felix Botner
Arvid Requate
:
Depends on: 49500
Blocks:
  Show dependency treegraph
 
Reported: 2019-06-05 10:13 CEST by Felix Botner
Modified: 2019-06-26 17:42 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional): Error handling, Troubleshooting
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Botner univentionstaff 2019-06-05 10:13:39 CEST
+++ This bug was initially created as a clone of Bug #49500 +++

Recently the ldap schema extension validation failed in some jenkins test (http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-0/job/AutotestJoin/SambaVersion=s4,Systemrolle=master-part-II/ws/test/autotest-091-master-s4-part-II.log/*view*/).

Unfortunately the output seems to be broken see, python/ldap_extension.py

 # Slapschema doesn't fail on schema errors, errors are 
 # printed to stdout (Bug #45571)
 ud.debug(ud.LISTENER, ud.ERROR, '%s: validation failed:\n%s.' % (name, stdout))

but:

-> slapschema >/dev/null
5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
slapschema: bad configuration file!
-> slapschema 2>/dev/null

So first step is to also print stderr to the log.
Comment 1 Felix Botner univentionstaff 2019-06-05 10:17:10 CEST
That did not help, seen again in the jenkins tests

updating 'cn=apps,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:47.297  LISTENER    ( PROCESS ) : updating 'cn=univention-app,cn=ldapschema,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.
05.06.19 01:01:50.352  LISTENER    ( PROCESS ) : updating 'cn=66univention-appcenter_app,cn=ldapacl,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:50.353  LISTENER    ( PROCESS ) : ldap_extension: cn=66univention-appcenter_app,cn=ldapacl,cn=univention,dc=AutoTest091,dc=local active? ['FALSE']
05.06.19 01:01:51.120  LISTENER    ( ERROR   ) : ldap_extension: slapd.conf validation failed:
5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to clause
5cf6f85f <access clause> ::= access to <what> [ by <who> [ <access> ] [ <control> ] ]+ 
<what> ::= * | dn[.<dnstyle>=<DN>] [filter=<filter>] [attrs=<attrspec>]
<attrspec> ::= <attrname> [val[/<matchingRule>][.<attrstyle>]=<value>] | <attrlist>
<attrlist> ::= <attr> [ , <attrlist> ]
<attr> ::= <attrname> | @<objectClass> | !<objectClass> | entry | children
<who> ::= [ * | anonymous | users | self | dn[.<dnstyle>]=<DN> ]
	[ realanonymous | realusers | realself | realdn[.<dnstyle>]=<DN> ]
	[dnattr=<attrname>]
	[realdnattr=<attrname>]
	[group[/<objectclass>[/<attrname>]][.<style>]=<group>]
	[peername[.<peernamestyle>]=<peer>] [sockname[.<style>]=<name>]
	[domain[.<domainstyle>]=<domain>] [sockurl[.<style>]=<url>]
	[dynacl/<name>[/<options>][.<dynstyle>][=<pattern>]]

No error message, next idea is to enable debug for the slapschema tool in univention-lib/python/ldap_extension.py.

Or has anybody a better idea?
Comment 2 Erik Damrose univentionstaff 2019-06-05 10:30:32 CEST
We briefly discussed it. We could add -v (verbose) and/or -d <level> to increase the debuglevel.

Some interesting points: The slapschema manpage states that errors are already logged to stdout, the previous fix to include stderr will not help add anything useful. In addition, the manpage states
"LIMITATIONS For some backend types, your slapd(8) should not be running (at least, not in read-write mode) when you do this to ensure consistency of the database. It is always safe to run slapschema with the slapd-bdb(5), slapd-hdb(5), and slapd-null(5) backends."
Comment 3 Arvid Requate univentionstaff 2019-06-05 12:54:16 CEST
Can we log the temporary (broken) /etc/ldap/slapd.conf to see line what 249 is?
Comment 4 Arvid Requate univentionstaff 2019-06-05 13:02:01 CEST
Ist this this the exact error message?

> 5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
slapschema: bad configuration file!

That's what I get when I edit /var/lib/univention-ldap/local-schema/univention-app.schema and change the first occurance of "attributetype" to "ttributetype".
Comment 5 Felix Botner univentionstaff 2019-06-05 13:05:33 CEST
(In reply to Arvid Requate from comment #3)
> Can we log the temporary (broken) /etc/ldap/slapd.conf to see line what 249
> is?

The problem is not in line 249. This is the error from the failed ACL registration (because the schema registration failed).

The problem is

app,cn=ldapschema,cn=univention,dc=AutoTest091,dc=local' command a
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.

and we need more information at this point. 
But logging the slapd.conf in this case is maybe a good idea anyway
Comment 6 Felix Botner univentionstaff 2019-06-05 13:06:43 CEST
(In reply to Arvid Requate from comment #4)
> Ist this this the exact error message?
> 
> > 5cddc2bc /var/lib/univention-ldap/local-schema/univention-app.schema: line 18: unknown directive <ttributetype> outside backend info and database definitions.
> slapschema: bad configuration file!
> 
> That's what I get when I edit
> /var/lib/univention-ldap/local-schema/univention-app.schema and change the
> first occurance of "attributetype" to "ttributetype".

We have no error message from the fails slapschema.
Comment 7 Florian Best univentionstaff 2019-06-06 09:11:18 CEST
(In reply to Felix Botner from comment #1)
> That did not help, seen again in the jenkins tests

> 5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to
> clause

> No error message, next idea is to enable debug for the slapschema tool in
> univention-lib/python/ldap_extension.py.
> 
> Or has anybody a better idea?

Hm, isn't the error message that the attribute univentionApp is not known? (because the schema is not yet registered?)
Comment 8 Felix Botner univentionstaff 2019-06-06 10:32:06 CEST
(In reply to Florian Best from comment #7)
> (In reply to Felix Botner from comment #1)
> > That did not help, seen again in the jenkins tests
> 
> > 5cf6f85f /etc/ldap/slapd.conf: line 249: unknown attr "@univentionApp" in to
> > clause
> 
> > No error message, next idea is to enable debug for the slapschema tool in
> > univention-lib/python/ldap_extension.py.
> > 
> > Or has anybody a better idea?
> 
> Hm, isn't the error message that the attribute univentionApp is not known?
> (because the schema is not yet registered?)

No, 

1) register schema

ldap_extension: validation failed:

.
05.06.19 01:01:49.771  LISTENER    ( ERROR   ) : ldap_extension: Removing new file /var/lib/univention-ldap/local-schema/univention-app.schema.

2) this fails, no univention-app schema

3) register ACL

this fails, with a proper error message  because the schema is missing, but this is a consequence of first error
Comment 9 Felix Botner univentionstaff 2019-06-19 16:01:52 CEST
7e7a28a8337330242279f0d15bb1c65231574b91 - univention-lib
Comment 10 Philipp Hahn univentionstaff 2019-06-19 17:03:21 CEST
(In reply to Felix Botner from comment #9)
> 7e7a28a8337330242279f0d15bb1c65231574b91 - univention-lib

There seems to be one case where "slapschema" exits with != 0 and no output:

servers/slapd/slapschema.c
  161 »···if ( slap_tool_destroy() )
  162 »···»···rc = EXIT_FAILURE;
servers/slapd/slapcommon.c
   939 »···»···»···if ( slap_shutdown( be ))
   940 »···»···»···»···rc = EXIT_FAILURE;
servers/slapd/init.c
  235 »···return backend_shutdown( be );·
servers/slapd/backend.c
   363 »···»···»···rc = be->bd_info->bi_db_close( be, NULL );
   364 »···»···»···if ( rc ) return rc;
...
   368 »···»···»···rc = be->bd_info->bi_close( be->bd_info );
   369 »···»···»···if ( rc ) return rc;

The other case always returns 0:

servers/slapd/slapcommon.c
   942 »···»···if ( slap_destroy())
   943 »···»···»···rc = EXIT_FAILURE;
servers/slapd/init.c
  255 »···rc = backend_destroy();
servers/slapd/backend.c
   529 »···return 0;

If "slapschema" terminates by a signal, it would be logged with a negative return code:

# python -c 'import subprocess,time,os,signal;p=subprocess.Popen(["sh","-c","sleep 10"]);time.sleep(1);os.kill(p.pid,signal.SIGSEGV);out,err=p.communicate();print(p.wait())'
-11
Comment 11 Felix Botner univentionstaff 2019-06-24 11:20:30 CEST
we decided to use slaptest -u instead of slapschema for schema checking. slapschema also checks the actual database and this is not necessary (and error-prone) at this point.

Successful build
Package: univention-lib
Version: 8.0.1-23A~4.4.0.201906241115
Branch: ucs_4.4-0
Scope: errata4.4-0
Comment 13 Arvid Requate univentionstaff 2019-06-26 17:42:57 CEST
<http://errata.software-univention.de/ucs/4.4/168.html>