Univention Bugzilla – Bug 49838
Wireless/Wired group GPO not replicated from Master Samba/AD to School Slave via OpenLDAP
Last modified: 2019-12-18 13:33:09 CET
When a UCS@school admin wants to configure a Wifi autologin profile via GPO (see URL abouve), this currently only works when it's done on each of the UCS@school Slaves separately. If the admin creates this GPO in the UCS Master Samba/AD instead, then the affected Microsoft Windows Clients fail to apply the GPO, because the S4-Connector ignores the essential GPO sub-object for the Wifi autologin profile: ============================================================================== # record 5 dn: CN=WiFi-Autologin-Radius-USC,CN=IEEE80211,CN=Windows,CN=Microsoft,CN=Machine,CN={6F5794D6-DFCA-48E5-89B3-4DEA576BE704},CN=Policies,CN=System,DC=school,DC=intranet objectClass: top objectClass: ms-net-ieee-80211-GroupPolicy instanceType: 4 whenCreated: 20190707054723.0Z uSNCreated: 4157 showInAdvancedViewOnly: TRUE objectGUID: cfb69081-c6b1-417a-af40-ce4f85ea2501 objectCategory: CN=ms-net-ieee-80211-GroupPolicy,CN=Schema,CN=Configuration,DC =school,DC=intranet ms-net-ieee-80211-GP-PolicyGUID: {410c9b3b-eb20-4e62-82f6-93f7aa93b49a} description: WiFi Autologin cn: WiFi-Autologin-Radius-USC name: WiFi-Autologin-Radius-USC ms-net-ieee-80211-GP-PolicyData:: <Binary-blob-displayed-in-base64> whenChanged: 20190711143114.0Z uSNChanged: 4189 distinguishedName: CN=WiFi-Autologin-Radius-USC,CN=IEEE80211,CN=Windows,CN=Mic rosoft,CN=Machine,CN={6F5794D6-DFCA-48E5-89B3-4DEA576BE704},CN=Policies,CN=Sy stem,DC=school,DC=intranet ============================================================================== The technical UCS issue to solve here is the same as in Bug #40298.
I'd say that's a key scenario for school customers (districts).
Ok, if that is the case, then you need to put this issue onto the UCS@school board and flag it as "important" to address this issue. I did this now.
This is specified in [MS-GPWL]: Group Policy: Wireless/Wired Protocol Extension https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpwl/4ea37f20-c3b7-402b-ab25-bb82389198d1 says: " The following class names are used while constructing various LDAP messages: BLOB-based wireless Group Policy is stored as an Active Directory object that MUST be an instance of class msieee80211-Policy. [..] XML-based wireless Group Policy is stored as an Active Directory object that MUST be an instance of class ms-net-ieee-80211-GroupPolicy. XML-based wired Group Policy is stored as an Active Directory object that MUST be an instance of class ms-net-ieee-8023-GroupPolicy. " For details see: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpwl/70cca0b1-92c2-40fa-80c8-abdeff9e7f9e
Created attachment 10198 [details] patch (git:fbest/hackathon-s4-connector-windows-policies) (In reply to Michel Smidt from comment #1) > I'd say that's a key scenario for school customers (districts). (In reply to Arvid Requate from comment #2) > Ok, if that is the case, then you need to put this issue onto the UCS@school > board and flag it as "important" to address this issue. I did this now. Or ask Florian to do this in a Hackathon.
Created attachment 10199 [details] Screenshot how it looks in UMC The patch adds 3 new UDM modules (ms/gpwl-wired, ms/gpwl-wireless, ms/gpwl-wireless-blob) and adds them to the S4-Connector mapping.
The git branch contains enhanced patches for some more policy types, adds translations, fixes some data types/UMC layout things.
(In reply to Florian Best from comment #4) > > Or ask Florian to do this in a Hackathon. Great! :D
We should support all of the following: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpwl/70cca0b1-92c2-40fa-80c8-abdeff9e7f9e https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpdpc/6882716a-9936-47c3-b609-0d592697762b - already implemented https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpipsec/9fc15fcc-bc74-4e55-9e70-ce953cdf969d https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpsi/616e4dd2-d9ec-4f5d-a76a-aea9c6720e08 (https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpcap/6cd109b2-4acd-49bc-bf65-f2b49411cf8d - not supported yet by Samba)
Information for the QA: 1. In the Samba4 LDAP there exists already some default IPsec policies. To let them be synced do: diff --git services/univention-s4-connector/conffiles/etc/univention/s4connector/s4/mapping.py services/univention-s4-connector/conffiles/etc/univention/s4connector/s4/mapping.py index f7a111e037..578af9e541 100644 --- services/univention-s4-connector/conffiles/etc/univention/s4connector/s4/mapping.py +++ services/univention-s4-connector/conffiles/etc/univention/s4connector/s4/mapping.py @@ -63,7 +63,7 @@ global_ignore_subtree = [ 'CN=Server,CN=System,@%@connector/s4/ldap/base@%@', 'CN=ComPartitionSets,CN=System,@%@connector/s4/ldap/base@%@', 'CN=ComPartitions,CN=System,@%@connector/s4/ldap/base@%@', - 'CN=IP Security,CN=System,@%@connector/s4/ldap/base@%@', + #'CN=IP Security,CN=System,@%@connector/s4/ldap/base@%@', 'CN=DFSR-GlobalSettings,CN=System,@%@connector/s4/ldap/base@%@', 'CN=DomainUpdates,CN=System,@%@connector/s4/ldap/base@%@', 'CN=Password Settings Container,CN=System,@%@connector/s4/ldap/base@%@', ucr set connector/s4/mapping/msgpipsec=yes service univention-s4-connector restart /usr/share/univention-s4-connector/resync_object_from_s4.py --filter 'CN=IP Security' 2. In Samba4 LDAP is also already a Class Store for the Software installation policy: We currently don't map/sync "CN=Default Domain Policy,CN=System,$ldap_base". So either we can decide to sync this as well(?), or you can just create something like this: udm containers/cn create --set name='Default Domain Policy' --position "CN=System,$ldap_base" ucr set connector/s4/mapping/msgpsi=yes service univention-s4-connector restart /usr/share/univention-s4-connector/resync_object_from_s4.py --filter '(|(objectClass=classStore)(objectClass=categoryRegistration)(objectClass=packageRegistration))' Then the class store gets synced. For the creation of the Package registration or Category registration objects the AD tools on a Windows must be used. So far, I couldn't find any way to create a category registration object :-( 3. For the Wireless/Wired Group Policy objects the Windows tools must be used as well. ucr set connector/s4/mapping/msgpwl=yes service univention-s4-connector restart
Created attachment 10247 [details] Screenshot Software-Policy Package
Created attachment 10248 [details] Screenshot Software-Policy Category
Created attachment 10249 [details] Screenshot IPsec policies
Created attachment 10250 [details] Screenshot Wired Policy
Merged the branch: univention-s4-connector (13.0.2-59) 4ec687c51a24 | Bug #49838: Merge branch 'fbest/hackathon-s4-connector-windows-policies' into 4.4-3 674870663584 | Bug #49838: Add UDM syntax classes 5540e1503bf3 | Bug #49838: add translations 5d8d4059206d | Bug #49838: fix stopping of s4-connector in debug mode 30856d80a4e5 | Bug #49838: execute compare_function for multivalue attributes 1b56eed93090 | Bug #49838: Add icons for UMC 2258e20c3f6a | Bug #49838: add binary attributes to DECODE_IGNORELIST c033c808c8d7 | Bug #49838: Add MS Software installation policy 20d383c3c94c | Bug #49838: Add MS IPsec policy ec50a4b1abf1 | Bug #49838: add MS Wireless/Wired Group Policy 53b1c3b5a973 | Bug #49838: use dh-python and univention-l10n univention-s4-connector.yaml 4ec687c51a24 | Bug #49838: Merge branch 'fbest/hackathon-s4-connector-windows-policies' into 4.4-3 TBD: should we enable the new UCR variables? Or only for new installations? TODO: Opening objects with binary data in UMC causes a module crash. Should we fix this now? Or let the Python 3 migration fix this automatically where we have explicit encoding in UDM.
> TBD: should we enable the new UCR variables? Or only for new installations? We'll have to discuss this. Check out the current implementation of pushedprinterconnections, gpos and wmi filters. One thing to keep in mind is that the S4-Connector always first synchronizes from OpenLDAP to Samba/AD. IIRC we implemented one or two scripts called from joinscript to sync the objects from Samba/AD to OpenLDAP before starting the Connector with the new modules enabled. That way we try to avoid overwriting / removing stuff in Samba/AD, in particular in UCS@school scenarios where we have multiple connectors and new School slaves are joined and re-joined at any time. > TODO: Opening objects with binary data in UMC causes a module crash. Can we just hide those properties in UMC? That's useless/irritating information anyway.
I see this in our Jenkins tests (s4 slave https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-3/job/AutotestJoin/SambaVersion=s4,Systemrolle=slave/ws/test/join.log/*view*/) Multifile: /etc/ldap/slapd.conf Warning: slapd.service changed on disk. Run 'systemctl daemon-reload' to reload units. SRV record _ldap._tcp for port 7389 not created because Samba4 DCs are present: master095 wait for named ? Traceback (most recent call last): File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 455, in <module> main() File "/usr/share/univention-directory-manager-tools/univention-dnsedit", line 406, in main univention.admin.modules.update() File "/usr/lib/python2.7/dist-packages/univention/admin/modules.py", line 123, in update os.path.walk(dir, _walk, p) File "/usr/lib/python2.7/posixpath.py", line 239, in walk walk(name, func, arg) File "/usr/lib/python2.7/posixpath.py", line 231, in walk func(arg, top, names) File "/usr/lib/python2.7/dist-packages/univention/admin/modules.py", line 108, in _walk m = __import__(mod, globals(), locals(), name) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/ms/gpsi-package-registration.py", line 101, in <module> syntax=univention.admin.syntax.octetstring, AttributeError: 'module' object has no attribute 'octetstring' And please keep in mind, these new UDM modules have to work with older UCS versions (4.3).
And i see this in the s4connector tests aster091c] 2019-12-04T23:23:12.741071 Object created: cn=Policies,cn=System,dc=AutoTest091c,dc=local [master091c] 2019-12-04T23:23:12.860853 Object created: ou=Domain Controllers,dc=AutoTest091c,dc=local [master091c] 2019-12-04T23:23:12.967560 Object created: cn=WMIPolicy,cn=System,dc=AutoTest091c,dc=local [master091c] 2019-12-04T23:23:13.076967 Object created: cn=SOM,cn=WMIPolicy,cn=System,dc=AutoTest091c,dc=local [master091c] 2019-12-04T23:23:13.613340 Usage: ucs_registerLDAPExtension [options] [master091c] 2019-12-04T23:23:13.614290 ucs_registerLDAPExtension: error: --udm_syntax: file does not exist: usr/lib/python2.7/dist-packages/univention/admin/syntax.d/s4_connector.py [master091c] 2019-12-04T23:23:14.202458 Object exists: cn=ldapschema,cn=univention,dc=AutoTest091c,dc=local [master091c] 2019-12-04T23:23:14.848970 Object exists: cn=udm_module,cn=univention,dc=AutoTest091c,dc=local
(In reply to Felix Botner from comment #16) > And please keep in mind, these new UDM modules have to work with older UCS > versions (4.3). No, I did set "--versionstart 4.4-0" when registering them as extensions.
(In reply to Felix Botner from comment #17) > And i see this in the s4connector tests > [master091c] 2019-12-04T23:23:13.613340 Usage: ucs_registerLDAPExtension > [options] > [master091c] 2019-12-04T23:23:13.614290 ucs_registerLDAPExtension: error: > --udm_syntax: file does not exist: > usr/lib/python2.7/dist-packages/univention/admin/syntax.d/s4_connector.py Thanks, fixed the missing leading slash: univention-s4-connector (13.0.2-60) d056af32f139 | Bug #49838: fix typo registering syntax class
(In reply to Arvid Requate from comment #15) > > TODO: Opening objects with binary data in UMC causes a module crash. > > Can we just hide those properties in UMC? That's useless/irritating > information anyway. They are already hidden, but the problem is that UMC-UDM sends all values (obj.info) to the frontend. When trying to json.dumps(obj.info) this fails because the values are bytestring (str, not unicode) and JSON is UTF-8 based so it tries to decode every value as UTF-8, which fails for some binary sequences: MODULE ( ERROR ) : Traceback (most recent call last): File "/usr/sbin/univention-management-console-module", line 121, in <module> notifier.loop() File "/usr/lib/pymodules/python2.7/notifier/nf_generic.py", line 287, in loop step() File "/usr/lib/pymodules/python2.7/notifier/nf_generic.py", line 279, in step __min_timer = dispatch.dispatcher_run() File "/usr/lib/pymodules/python2.7/notifier/dispatch.py", line 72, in dispatcher_run if not disp(): File "/usr/lib/pymodules/python2.7/notifier/threads.py", line 154, in _simple_threads_dispatcher task.announce() File "/usr/lib/pymodules/python2.7/notifier/threads.py", line 135, in announce self._callback( self, self._result ) File "/usr/lib/pymodules/python2.7/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 297, in thread_finished_callback self.finished(request.id, result) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 478, in finished self.result(res) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 482, in result self.signal_emit('success', response) File "/usr/lib/pymodules/python2.7/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "/usr/lib/pymodules/python2.7/notifier/signals.py", line 41, in emit cb( *args ) File "/usr/lib/pymodules/python2.7/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 130, in _reply self.response(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 361, in response self.__queue += str(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 305, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 111, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/json/encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xac in position 1: invalid start byte It's a conceptual problem we have since years, see Bug #33522 (Bug #40485, Bug #48061). The Python-3 Migration must fix it nevertheless.
The UMC module crashes have been fixed by decoding everything as base64 in the UDM properties - while keeping all attributes binary data. univention-s4-connector (13.0.2-61) 06dbb0ba1d8c | Bug #49838: add base64 presentation in UMC
Question: Should we rename the UDM modules from ms/gp* to msgp/* ?
What I tested: Wired and Wireless Policies are synced to ldap -> OK Policies are shown in udm and umc -> OK Policies, created on the master in a multi school env, are synced to a school -> OK Changes in ldap are synced back to s4 -> OK Test are green on my test machine -> OK, Thanks for adding them! :) Note: QA for IPsec Policies has been done by Julia QA for Software Policies has been done by Johannes Please add your results :) Waiting for jenkins results
What I tested: GPIPSEC* Policies are synced to ldap -> OK GPIPSEC* are shown in udm and umc -> OK Policies, created on the master in a multi school env, are synced to a school -> OK Changes in ldap are synced back to s4 -> OK Changes in the Policies are synced only to Windows members of that school and apply -> OK Yaml incomplete -> Fail Why are the CN=IP Security Objects in the ignore_subtree?
I just saw that the code in 97univention-s4-connector.inst uses resync_object_from_s4.py to trigger an initial sync from Samba/AD-> UDM. IIRC that tool just adds the DNs to the corresponding reject list, but the S4-Connector only considers that *after* the initial sync from OpenLDAP. Please check that again, I'm unsure. I know that Felix code for msprintconnectionpolicy used the same tool, but just in case...
The branch git:fbest/49838-domainpolicy contains a patch for synconizing also the "CN=Default Domain Policy,CN=System,$ldap_base" object / domainPolicy object class.
(In reply to Julia Bremer from comment #24) > Why are the CN=IP Security Objects in the ignore_subtree? I don't know. I created Bug #50642 for this. (In reply to Florian Best from comment #26) > The branch git:fbest/49838-domainpolicy contains a patch for synconizing > also the "CN=Default Domain Policy,CN=System,$ldap_base" object / > domainPolicy object class. → Moved to Bug #50641
(In reply to Arvid Requate from comment #25) > I just saw that the code in 97univention-s4-connector.inst uses > resync_object_from_s4.py to trigger an initial sync from Samba/AD-> UDM. > IIRC that tool just adds the DNs to the corresponding reject list, but the > S4-Connector only considers that *after* the initial sync from OpenLDAP. > Please check that again, I'm unsure. I know that Felix code for > msprintconnectionpolicy used the same tool, but just in case... > Diese --write2ucs oder resync_object_from_s4.py Aufrufe in dem Joinscript sind ja dafür da, zu verhindern, dass der S4-Connector, der beim Start immer zuerst von sync_from_ucs durchläuft, Objekte im Samba/AD überbügelt. Die Skripte, die dort mit der Option --write2ucs aufgerufen werden, machen das alles direkt. Aber das resync_object_from_s4.py Skript sucht einfach nur die DNs raus und schreibt sie dem S4-Connecgtor in seine Reject-Tabellen. Ich befürchte, dass er die erst anschaut, wenn er mit dem Initialen sync_from_ucs fertig ist. So, you would do it like it's done in the joinscript with the upgrade_msWMI-Som.py msGPOWQLFilter.py calls? This adds the objects also to the reject table but prior it modifies it once (with same values). I can add a tool which "touch"es every object once before adding it to the sqlite reject table.
> So, you would do it like it's done in the joinscript with the upgrade_msWMI-Som.py msGPOWQLFilter.py calls? From reading the code, I see that msGPOWQLFilter with --write2ucs manually synchronizes a new attribute from Samba/AD to OpenLDAP. I don't know if that's required. Let's keep it the way your iplemented it, this worked also for msPrint-ConnectionPolicy.
Sounds good. Added version number to advisory: univention-s4-connector.yaml 5dcc37c60de9 | YAML Bug #49838
Tested: Software-Policy Packages Attributes are synced: OK IPsec policies are synced if taken out of the ignore_subtree : OK Wired policies are synced: OK Wireless policies are synced: OK UCR-Variables: OK Works in ucs@chool environment: OK Manual Windows Tests: OK Automated tests: OK Default Domain Policy is not yet synced: OK YAML: OK -> Verified
<http://errata.software-univention.de/ucs/4.4/407.html>