Univention Bugzilla – Bug 43485
Pilot study: Windows Search Protocol (MS-WSP)
Last modified: 2022-10-14 10:53:53 CEST
An implementation of the Windows Search Protocol (MS-WSP) has been started upstream. We should check it out, build it, assess the state of the implementation, see if we can help and learn what would be required to integrate it in the product. Indexer backend configuration is required for this too. This is the current state: * https://lists.samba.org/archive/samba-technical/2017-January/117990.html https://sambaxp.org/archive_data/SambaXP2015-SLIDES/wed/track1/sambaxp2015-wed-track1-Noel_Power-WindowSearchProtocolAndSamba.pdf https://wiki.samba.org/index.php/SoC/Ideas#Windows_Search_Protocol_WSP_client_library_and_torture_tests
Noel manages the WSP code on Github in his repository https://github.com/noelpower/samba. Several branches exist, `WSP-WIP` seems to be the main development branch. The current patch-sets to `samba-technical` are in `WSP-WIP-NO_RAWPIPE` and `WSP-WIP-CLIENT-ONLY`. The implementation seems advanced. Everything is integrated in the Samba build-system, the WSP-daemon is configured via `smb.conf`. The configuration of tracker/systemd is as documented in [0], some paths must be corrected from `/usr/libexec/ to `/usr/lib/tracker`. The branch `WSP-WIP` requires little effort to rebase on the samba `4.6.rc2` release branch. A little patch was necessary to get the daemon to work with Windows (see [1]). The WSP-daemon was tested with Windows 7, Windows 8.1 and Windows 10. There are some quirks: - The share to be searched must be selected. Just selecting a remote host is not enough. - The search yields slightly different results than the native Windows search. This is due to different semantics and capabilities to search e.g. in files (Windows searches PDF content, the WSP-daemon does not. The WSP-daemon searches MP3-metadata, the Windows search does not.). As hoped, the WSP-daemon speeds the search up with large shares. Users without the proper read/access permissions can't search the shares. This might be due to the fact, that a share must be selected and Windows denies that. Open questions: - How to package the WSP-daemon? - How to package auxiliary configuration (tracker/systemd)? - How to automatically add the samba-shares to the tracker-configuration? - Remove `/etc/xdg/autostart/tracker-store.desktop` to prevent tracker for normal users? Further links: - WSP patches from 01.17 [2] - WSP patches from 11.16 [3] - WSP patches from 09.16 [4] - tevent-glib-glue patches which the WSP-daemon is based on [5] - Microsoft `[MS-WSP]: Windows Search Protocol` documentation [6] [0]: https://wiki.samba.org/index.php/Spotlight#Setup. [1]: https://github.com/noelpower/samba/pull/1 [2]: https://lists.samba.org/archive/samba-technical/2017-January/117990.html [3]: https://lists.samba.org/archive/samba-technical/2016-November/117312.html [4]: https://lists.samba.org/archive/samba-technical/2016-September/116248.html [5]: https://lists.samba.org/archive/samba-technical/2016-February/112172.html [6]: https://msdn.microsoft.com/en-us/library/cc251767.aspx
Additional security comment: The WSP-daemon checks every file in the result-set, if it is accessible by the querying user.
> A little patch was necessary to get the daemon to work with Windows (see [1]). Ok, we should upstream that to Noel. > There are some quirks: > > - The share to be searched must be selected. Just selecting a remote host is > not enough. Ok, that would be the next level. I guess we should continue with a proof of concept and make it installable & configurable on a UCS system before diving into that. > - The search yields slightly different results than the native Windows search. > This is due to different semantics and capabilities to search e.g. in files > (Windows searches PDF content, the WSP-daemon does not. The WSP-daemon > searches MP3-metadata, the Windows search does not.). Sure, that's ok. > Open questions: > > - How to package the WSP-daemon? For the PoC we should create a scope, e.g "samba-wsp" cherrypick the samba package into it and apply the samba-specific patches to it. > - How to package auxiliary configuration (tracker/systemd)? Usually we create an integration package, e.g. univention-indexer-tracker, which depends on the tracker debian package and ships the UCR-variable definitions and UCR-templates for the required config files. > - How to automatically add the samba-shares to the tracker-configuration? As discussed, that should be done via listener module. Also, we might need to adjust fs.inotify.max_user_watches or at least mention it in the documentation at some point.
The packaging is realized in the new scope `samba-wsp`. Two packages are patched, one is new. Add the unpublished scope to your sources and install the following packages: - samba 2:4.6.0~rc2-1A~4.2.0.201703021526 - univention-samba4 6.0.9-7A~4.2.0.201703061427 - univention-samba-wsp 0.0.1-2A~4.2.0.201703061406 The `samba` package contains the patches from [1] (including the necessary fixes) and provides the basic WSP daemon. `univention-samba4` was patched to introduce a new UCR variable `samba4/service/wsp/enabled` to control the WSP daemon. If set to `yes`, the daemon is started together with the `samba` service. This package was also patched to enable searches via the WSP daemon for all configured shares. `univention-samba-wsp` is the integration package containing the glue between UCS, the WSP daemon and the tracker indexing and searching service. Upon installation the WSP daemon is enabled via `samba4/service/wsp/enabled` and `samba` restarted. A listener in `univention-samba-wsp` reacts to share creation/deletion and updates the tracker configuration to enable/disable indexing. It also reacts to user creation/deletion to update the tracker configuration to support the samba `homes` shares. Some limitations: - The Samba service is not automatically restarted upon changing `samba4/service/wsp/enabled`. This must be done manually. - The tracker services are always running, disregarding `samba4/service/wsp/enabled`. - The `samba-wsp-shares` listener enables/disables tracker indexing of shares regardless of `samba4/service/wsp/enabled`. - WSP is enabled for shares if `samba4/service/wsp/enabled` was set to `yes` on creation time (as the listener created the configuration). A workarround would be to run `univention-directory-listener-ctrl resync samba-shares`. - The user can not enable/disable WSP on a per-share basis. - The Debian Jessy version of tracker is old. A newer version would be nice, but requires a new version of `glib`. See also comment #1. [1]: https://github.com/noelpower/samba/tree/WSP-WIP [2]: https://github.com/noelpower/samba/pull/1
After the UCS 4.2 release and the merge of the WSP patches into Noels repo [1], the packages in the `samba-wsp` got updated. For amd64 the following versions are current: - samba 4.6.1-1A~4.2.0.201704131754 - univention-samba4 6.0.9-10A~4.2.0.201704131757 - univention-samba-wsp 0.0.1-2A~4.2.0.201703061406 At this point the following command is necessary if the share was existend prior to installing the packages, to enable the WSP indexing on a per-share level. univention-directory-listener-ctrl resync samba-shares && service samba restart [1]: https://github.com/noelpower/samba/tree/WSP-WIP
Git repo for Issue 6903 has a few additional (untested) patches.