Bug 41247 - connector.samr.OpenUser: Insufficient system resources exist to complete the API.
connector.samr.OpenUser: Insufficient system resources exist to complete the ...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: AD Connector
UCS 4.1
Other Linux
: P3 normal (vote)
: UCS 4.1-2-errata
Assigned To: Felix Botner
Stefan Gohmann
:
Depends on: 40745
Blocks:
  Show dependency treegraph
 
Reported: 2016-05-10 19:11 CEST by Arvid Requate
Modified: 2016-09-21 20:11 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.274
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2016042621000438
Bug group (optional): Error handling
Max CVSS v3 score:


Attachments
lsass.exe (107.34 KB, image/png)
2016-05-19 12:59 CEST, Michel Smidt
Details
untested patch for univention-ad-connector (4.1-2) (3.02 KB, patch)
2016-05-19 15:36 CEST, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2016-05-10 19:11:59 CEST
At Ticket #2016042621000438 we had the case where the AD Connector failed to set the password in AD for some of the users during user import (yes, it's a school environment with a peculiar AD-Sync setup). The AD-Connector used the method introduced via Bug 40745. A typical traceback looked like this:

=========================================================================
09.05.2016 10:35:54,24 LDAP        (PROCESS): sync from ucs: [ 
 user] [       add] cn=schueler1,cn=schueler,cn=users,ou="$school","$connector_ad_ldap_basedn"
09.05.2016 10:35:54,97 LDAP        (WARNING): sync failed, saved as rejected
09.05.2016 10:35:54,98 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/connector/__init__.py", line 733, in __sync_file_from_ucs
    or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn))):
  File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 2358, in sync_from_ucs 
    f(self, property_type, object)
  File "/usr/lib/pymodules/python2.7/univention/connector/ad/password.py", line 287, in password_sync_ucs
    res = set_password_in_ad(connector, object['attributes']['sAMAccountName'][0], pwd)
  File "/usr/lib/pymodules/python2.7/univention/connector/ad/password.py", line 139, in set_password_in_ad
    user_handle = connector.samr.OpenUser(connector.dom_handle, security.SEC_FLAG_MAXIMUM_ALLOWED, rid)
RuntimeError: (-1073741670, 'Insufficient system resources exist to complete the API.')
=========================================================================

After restarting the AD Connector the rejects resolved, so I guess that it might be a timeout issue (or other linit) for the SAMR RPC session. We should detect this and retry.

After all this the majority of synchronized users accounts where still disabled in AD (userAccountControl: 546). No clue if this is related or just accidental. I fixed in manually with a small shell script.
Comment 1 Felix Botner univentionstaff 2016-05-12 16:43:42 CEST
Seem that 'Insufficient system resources exist to complete the API.' indicates a hardware/windows kernel failure:

http://mikemstech.blogspot.de/2011/12/troubleshooting-0x0000007a.html
this one says - error code that indicates serious file system corruption

https://support.microsoft.com/en-us/kb/909095
here it is a Windows kernel power manager problem 

http://answers.microsoft.com/en-us/windows/forum/windows8_1-hardware/this-device-cannot-start-code-10-insufficient/4d84d097-2da5-4701-bdcf-04c5b02cd2a9
driver issue here

...

So i guess if this error pops up there are some serious windows kernel/hardware issue's involved, nothing we can do about.

I also checkd if the ad connector can sync password after a restart of windows (ucs/windows running, connector start and samr connection initialized, sync ok, restart windows, sync OK).

It is still suspicious though that the sync in the customer environment worked after restarting the connector.
Comment 2 Stefan Gohmann univentionstaff 2016-05-18 08:35:00 CEST
(In reply to Felix Botner from comment #1)
> It is still suspicious though that the sync in the customer environment
> worked after restarting the connector.

But that could be a client side issue for example a timeout issue.

Do we have more debug? How much time has passed from the first password sync in this connector session? How much time has passed from the last password sync in this connector session?
Comment 3 Michel Smidt 2016-05-19 12:59:13 CEST
Created attachment 7670 [details]
lsass.exe
Comment 4 Michel Smidt 2016-05-19 13:01:33 CEST
Seems that the lsass.exe on the AD continuously increase during:
(PROCESS): sync from ucs: [          user] [       add] dn-xy

See attached screenshot (the actual load is low but continuously increase!). A restart of the univention-ad-connector decrease the load of the lsass.exe
Comment 5 Arvid Requate univentionstaff 2016-05-19 15:35:58 CEST
I guess you are referring to the number of threads? Ok, this might lead to
"Insufficient system resources". Probably we need to close the connection(s) properly.

This is what the old pwdump6/LsaExt.c did:
===========================================================
        if (hPipe != NULL && hPipe != INVALID_HANDLE_VALUE)
                CloseHandle(hPipe);

        if(hUser) 
                pSamrCloseHandle(&hUser);
        if(hDomain) 
                pSamrCloseHandle(&hDomain);
        if(hSam) 
                pSamrCloseHandle(&hSam);
        if(hLsa) 
                LsaClose(hLsa);
===========================================================
Comment 6 Arvid Requate univentionstaff 2016-05-19 15:36:35 CEST
Created attachment 7672 [details]
untested patch for univention-ad-connector (4.1-2)
Comment 7 Felix Botner univentionstaff 2016-05-19 16:28:55 CEST
I would prefer to close only the user_handle (not self.dom_handle and self.samr_handle) in set_password_in_ad. I think this global samr connection was introduced with regard to the resource consumption of the local samba daemon.
see Bug #40745

Opinions?
Comment 8 Arvid Requate univentionstaff 2016-05-20 14:42:18 CEST
No objections, we can just go ahead and test. Would be great if a minimal adjustment would avoid the issue.
Comment 9 Felix Botner univentionstaff 2016-06-06 13:12:02 CEST
univention-ad-connector r69844
 * connector.samr.Close(user_handle) after password sync to ad

univention-ad-connector.yaml
Comment 10 Stefan Gohmann univentionstaff 2016-06-08 06:48:12 CEST
Code review: OK

Jenkins tests: OK (http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/ADConnectorMultiEnv/7/)

Manual tests: OK

YAML: OK (r69933 s/AD/Active Directory)
Comment 11 Janek Walkenhorst univentionstaff 2016-06-08 14:28:34 CEST
<http://errata.software-univention.de/ucs/4.1/197.html>