Bug 45042 - slapd segfault in libxmlsec1.so.1.2.20
slapd segfault in libxmlsec1.so.1.2.20
Product: UCS
Classification: Unclassified
Component: SAML
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2-1-errata
Assigned To: Florian Best
Jürn Brodersen
Depends on:
  Show dependency treegraph
Reported: 2017-07-19 11:28 CEST by Florian Best
Modified: 2017-08-09 16:57 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.160
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2017072621000074
Bug group (optional): External feedback, Security
Max CVSS v3 score:
best: Patch_Available+

patch (440 bytes, patch)
2017-07-26 13:27 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2017-07-19 11:28:22 CEST
We got a report that slapd segfaults in libxmlsec1.so, which is the crudesaml SASL plugin for SAML authentication at the LDAP server.

kernel: [71889.842511] slapd[5969]: segfault at f ip 00007f8081c07c72 sp 00007f7ffee63210 error 4 in libxmlsec1.so.1.2.20[7f8081bcb000+5d000]

The reporter sais:
I seem to be able to pretty reliably trigger the segfault when logging into a joined backup or member server with SAML SSO and trying to load any module that hits the LDAP on the master (DNS/DHCP/LDAP).
Comment 1 Florian Best univentionstaff 2017-07-19 11:32:04 CEST
The user hit also Bug #44764.
Comment 2 Florian Best univentionstaff 2017-07-19 11:32:23 CEST
So maybe the certificates are expired.
Comment 3 Florian Best univentionstaff 2017-07-26 08:50:18 CEST
The UMC-Webserver is also segfaulting:

python2.7: Loaded metadata from "/usr/share/univention-management-console/saml/idp/ucs-sso.ourdomain-snipped.com.au.xml"
python2.7: SAML assertion issuer is https://ucs-sso.ourdomain-snipped.com.au/simplesamlphp/saml2/idp/metadata.php
kernel: [161928.412566] univention-mana[31896]: segfault at f ip 00007f27d2611c72 sp 00007f27d06f1690 error 4 in libxmlsec1.so.1.2.20[7f27d25d5000+5d000]

We have got a core dump: upload_oGStDb.unknown
Comment 4 Florian Best univentionstaff 2017-07-26 11:50:59 CEST
#0  0x00007f83408e0c72 in xmlSecGetNodeNsHref () from /usr/lib/libxmlsec1.so.1
#1  0x00007f8341b6ffa0 in lasso_provider_verify_saml_signature () from /usr/lib/liblasso.so.3
#2  0x00007f8342893f45 in saml_check_assertion_signature (ctx=<optimized out>, doc=0x7f82a0113070, issuer=0x7f82a011f000 "https://ucs-sso***********.au/simplesamlphp/saml2/idp/metadata.php", node=0x7f82a0113070, params=0x7f82a0000f70) at saml.c:389
#3  saml_check_one_assertion (doc=0x7f82a0113070, assertion=<optimized out>, userid=0x7f82ad1f9800, params=0x7f82a0000f70, ctx=0x7f82a00010e0) at saml.c:442
#4  saml_check_all_assertions (ctx=ctx@entry=0x7f82a00010e0, params=params@entry=0x7f82a0000f70, userid=userid@entry=0x7f82ad1f9800, saml_msg=<optimized out>, 
    saml_msg@entry=0x7f82a010b2e0 "<samlp:Response xmlns:samlp=\"urn:oasis:names:tc:SAML:2.0:protocol\" xmlns:saml=\"urn:oasis:names:tc:SAML:2.0:assertion\" ID=\"_c419dd5e2319c9e7f3837513670f61dc4d01f3ad5a\" Version=\"2.0\" IssueInstant=\"2017-"..., flags=<optimized out>)
    at saml.c:562
#5  0x00007f8342892c1d in saml_server_mech_step (conn_context=0x7f82a00010e0, params=0x7f82a0000f70, clientin=0x7f82a010844a "", clientinlen=<optimized out>, serverout=<optimized out>, serveroutlen=<optimized out>, oparams=0x7f82a0107a90) at cy2_saml.c:277
#6  0x00007f8346f3e1e2 in sasl_server_step () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
#7  0x00007f8346f3e6f1 in sasl_server_start () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
#8  0x00005634c8079cee in slap_sasl_bind (op=op@entry=0x7f82a811d980, rs=rs@entry=0x7f82ad1f9aa0) at ../../../../servers/slapd/sasl.c:1525
#9  0x00005634c8043b1f in fe_op_bind (op=0x7f82a811d980, rs=0x7f82ad1f9aa0) at ../../../../servers/slapd/bind.c:280
#10 0x00005634c8043317 in do_bind (op=0x7f82a811d980, rs=0x7f82ad1f9aa0) at ../../../../servers/slapd/bind.c:205
#11 0x00005634c8024ccc in connection_operation (ctx=0x7f82ad1f9c10, arg_v=0x7f82a811d980) at ../../../../servers/slapd/connection.c:1155
#12 0x00005634c8024fb7 in connection_read_thread (ctx=0x7f82a0113070, argv=0x7f82a0113070) at ../../../../servers/slapd/connection.c:1291
#13 0x00007f8347ba4a32 in ldap_int_thread_pool_wrapper (xpool=0x5634ca1fdc40) at ../../../../libraries/libldap_r/tpool.c:696
#14 0x00007f8345bc7064 in start_thread (arg=0x7f82ad1fa700) at pthread_create.c:309
#15 0x00007f83458fc62d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Comment 7 Florian Best univentionstaff 2017-07-26 13:26:59 CEST
It seems that there is an uninitialized(/pseudo?) struct which is the XML root:

(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.type
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.name
$48 = (const xmlChar *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.children
$49 = (struct _xmlNode *) 0x7f82a01044a0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.last
$50 = (struct _xmlNode *) 0x7f82a01044a0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.parent
$51 = (struct _xmlNode *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.next
$52 = (struct _xmlNode *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.prev
$53 = (struct _xmlNode *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.doc
$54 = (struct _xmlDoc *) 0x7f82a0113070
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.ns
$55 = (xmlNs *) 0xffffffffffffffff
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.content
$56 = (xmlChar *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.properties
$57 = (struct _xmlAttr *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.nsDef
$58 = (xmlNs *) 0x0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.psvi
$59 = (void *) 0x7f82a0112f70
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.line
$60 = 0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.parent.extra
$61 = 0
(gdb) print xpobj->nodesetval->nodeTab[0].parent.name
$62 = (const xmlChar *) 0x7f82a01042c0 "Response"

So there you see, that parent.parent is the child of the <Response> element, which is the actual parent (See the XML in comment 6).

Some notes:

In frame 4 the XML is parsed (saml_check_all_assertions in crudesaml-1.5/saml.c):

xpobj = xmlXPathEvalExpression((const xmlChar *) "//saml:Assertion[@ID]", xpctx)
node = xpobj->nodesetval->nodeTab[0];

In frame 2 "node" is a iteration over "node->parent" from frame 4.
Comment 8 Florian Best univentionstaff 2017-07-26 13:27:22 CEST
Created attachment 9059 [details]
Comment 9 Florian Best univentionstaff 2017-07-26 13:46:20 CEST
Comment 11 Emmanuel Dreyfus 2017-07-28 01:20:15 CEST
This bug seems to have been fixed upstream in crudesaml-1.7

Latest release is 1.8:
Comment 12 Florian Best univentionstaff 2017-07-28 11:04:11 CEST
I upgraded crudesaml to version 1.8.
All of our patches are integrated upstream except:

The patch 10_fix-varargs.patch was fixed upstream in a different way.

r81521 | YAML Bug #45042

crudesaml (1.8.0-1):
r81520 | Bug #45042: upgrade to crudesaml 1.8
Comment 13 Jürn Brodersen univentionstaff 2017-08-04 12:22:14 CEST
I added a test case that triggered the segfault by renewing the ucs-sso certificate.

r81785: Added test for saml certificate renewal.

If the test does not fail on jenkins I will set this to verified on monday.
Comment 14 Jürn Brodersen univentionstaff 2017-08-08 12:11:11 CEST
Looks good to me.

I tested saml with the umc.
All saml tests are looking good.

-> Verified
Comment 15 Arvid Requate univentionstaff 2017-08-09 16:57:24 CEST