Bug 45326 - Samba 4.7.5
Samba 4.7.5
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Samba4
UCS 4.3
Other Linux
: P5 enhancement (vote)
: UCS 4.3
Assigned To: Arvid Requate
Stefan Gohmann
https://wiki.samba.org/index.php/Mana...
: interim-2
: 45993 (view as bug list)
Depends on: 37286 40662 42045
Blocks: 46338 47367
  Show dependency treegraph
 
Reported: 2017-09-05 19:49 CEST by Arvid Requate
Modified: 2018-07-23 11:24 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2017-09-05 19:49:10 CEST
UCS 4.2-3 should probably be shipped with Samba 4.7.
Comment 1 Stefan Gohmann univentionstaff 2017-09-25 09:17:01 CEST
Samba 4.7 has been released. From the release announcement:

----------------------------------------------------------------------------
Release Announcements
---------------------

This is the first stable release of Samba 4.7.
Please read the release notes carefully before upgrading.

UPGRADING
=========

'smbclient' changes
------------------

'smbclient' no longer prints a 'Domain=[...] OS=[Windows 6.1] Server=[...]'
banner when connecting to the first server. With SMB2 and Kerberos,
there's no way to print this information reliably. Now we avoid it at all
consistently. In interactive sessions the following banner is now presented
to the user: 'Try "help" do get a list of possible commands.'.

The default for "client max protocol" has changed to "SMB3_11",
which means that 'smbclient' (and related commands) will work against
servers without SMB1 support.

It's possible to use the '-m/--max-protocol' option to overwrite
the "client max protocol" option temporarily.

Note that the '-e/--encrypt' option also works with most SMB3 servers
(e.g. Windows >= 2012 and Samba >= 4.0.0), so the SMB1 unix extensions
are not required for encryption.

The change to SMB3_11 as default also means 'smbclient' no longer
negotiates SMB1 unix extensions by default, when talking to a Samba server with
"unix extensions = yes".  As a result, some commands are not available, e.g.
'posix_encrypt', 'posix_open', 'posix_mkdir', 'posix_rmdir', 'posix_unlink',
'posix_whoami', 'getfacl' and 'symlink'. Using "-mNT1" reenables them, if the
server supports SMB1.

Note the default ("CORE") for "client min protocol" hasn't changed,
so it's still possible to connect to SMB1-only servers by default.

'smbclient' learned a new command 'deltree' that is able to do
a recursive deletion of a directory tree.


NEW FEATURES/CHANGES
====================

Whole DB read locks: Improved LDAP and replication consistency
--------------------------------------------------------------

Prior to Samba 4.7 and ldb 1.2.0, the LDB database layer used by Samba
erroneously did not take whole-DB read locks to protect search
and DRS replication operations.

While each object returned remained subject to a record-level lock (so
would remain consistent to itself), under a race condition with a
rename or delete, it and any links (like the member attribute) to it
would not be returned.

The symptoms of this issue include:

Replication failures with this error showing in the client side logs:
 error during DRS repl ADD: No objectClass found in replPropertyMetaData for
 Failed to commit objects:
 WERR_GEN_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE

A crash of the server, in particular the rpc_server process with
 INTERNAL ERROR: Signal 11

LDAP read inconsistency
 A DN subject to a search at the same time as it is being renamed
 may not appear under either the old or new name, but will re-appear
 for a subsequent search.

See https://bugzilla.samba.org/show_bug.cgi?id=12858 for more details
and updated advise on database recovery for affected installations.

Samba AD with MIT Kerberos
--------------------------

After four years of development, Samba finally supports compiling and
running Samba AD with MIT Kerberos. You can enable it with:

    ./configure --with-system-mitkrb5

Samba requires version 1.15.1 of MIT Kerberos to build with AD DC support.
The krb5-devel and krb5-server packages are required.
The feature set is not on par with the Heimdal build but the most important
things, like forest and external trusts, are working. Samba uses the KDC binary
provided by MIT Kerberos.

Missing features, compared to Heimdal, are:
  * PKINIT support
  * S4U2SELF/S4U2PROXY support
  * RODC support (not fully working with Heimdal either)

The Samba AD process will take care of starting the MIT KDC and it will load a
KDB (Kerberos Database) driver to access the Samba AD database.  When
provisioning an AD DC using 'samba-tool' it will take care of creating a correct
kdc.conf file for the MIT KDC.

For further details, see:
https://wiki.samba.org/index.php/Running_a_Samba_AD_DC_with_MIT_Kerberos_KDC

Dynamic RPC port range
----------------------

The dynamic port range for RPC services has been changed from the old default
value "1024-1300" to "49152-65535". This port range is not only used by a
Samba AD DC, but also applies to all other server roles including NT4-style
domain controllers. The new value has been defined by Microsoft in Windows
Server 2008 and newer versions. To make it easier for Administrators to control
those port ranges we use the same default and make it configurable with the
option: "rpc server dynamic port range".

The "rpc server port" option sets the first available port from the new
"rpc server dynamic port range" option. The option "rpc server port" only
applies to Samba provisioned as an AD DC.

Authentication and Authorization audit support
----------------------------------------------

Detailed authentication and authorization audit information is now
logged to Samba's debug logs under the "auth_audit" debug class,
including in particular the client IP address triggering the audit
line.  Additionally, if Samba is compiled against the jansson JSON
library, a JSON representation is logged under the "auth_json_audit"
debug class.

Audit support is comprehensive for all authentication and
authorisation of user accounts in the Samba Active Directory Domain
Controller, as well as the implicit authentication in password
changes.  In the file server and classic/NT4 domain controller, NTLM
authentication, SMB and RPC authorization is covered, however password
changes are not at this stage, and this support is not currently
backed by a testsuite.

For further details, see:
https://wiki.samba.org/index.php/Setting_up_Audit_Logging

Multi-process LDAP Server
-------------------------

The LDAP server in the AD DC now honours the process model used for
the rest of the 'samba' process, rather than being forced into a single
process.  This aids in Samba's ability to scale to larger numbers of AD
clients and the AD DC's overall resiliency, but will mean that there is a
fork()ed child for every LDAP client, which may be more resource
intensive in some situations.  If you run Samba in a
resource-constrained VM, consider allocating more RAM and swap space.

Improved Read-Only Domain Controller (RODC) Support
---------------------------------------------------

Support for RODCs in Samba AD until now has been experimental. With this latest
version, many of the critical bugs have been fixed and the RODC can be used in
DC environments requiring no writable behaviour. RODCs now correctly support
bad password lockouts and password disclosure auditing through the
msDS-RevealedUsers attribute.

The fixes made to the RWDC will also allow Windows RODC to function more
correctly and to avoid strange data omissions such as failures to replicate
groups or updated passwords. Password changes are currently rejected at the
RODC, although referrals should be given over LDAP. While any bad passwords can
trigger domain-wide lockout, good passwords which have not been replicated yet
for a password change can only be used via NTLM on the RODC (and not Kerberos).

The reliability of RODCs locating a writable partner still requires some
improvements and so the 'password server' configuration option is generally
recommended on the RODC.

Samba 4.7 is the first Samba release to be secure as an RODC or when
hosting an RODC.  If you have been using earlier Samba versions to
host or be an RODC, please upgrade.

In particular see https://bugzilla.samba.org/show_bug.cgi?id=12977 for
details on the security implications for password disclosure to an
RODC using earlier versions.

Additional password hashes stored in supplementalCredentials
------------------------------------------------------------

A new config option 'password hash userPassword schemes' has been added to
enable generation of SHA-256 and SHA-512 hashes (without storing the plaintext
password with reversible encryption). This builds upon previous work to improve
password sync for the AD DC (originally using GPG).

The user command of 'samba-tool' has been updated in order to be able to
extract these additional hashes, as well as extracting the (HTTP) WDigest
hashes that we had also been storing in supplementalCredentials.

Improvements to DNS during Active Directory domain join
-------------------------------------------------------

The 'samba-tool' domain join command will now add the A and GUID DNS records
(on both the local and remote servers) during a join if possible via RPC. This
should allow replication to proceed more smoothly post-join.

The mname element of the SOA record will now also be dynamically generated to
point to the local read-write server. 'samba_dnsupdate' should now be more
reliable as it will now find the appropriate name server even when resolv.conf
points to a forwarder.

Significant AD performance and replication improvements
-------------------------------------------------------

Previously, replication of group memberships was been an incredibly expensive
process for the AD DC. This was mostly due to unnecessary CPU time being spent
parsing member linked attributes. The database now stores these linked
attributes in sorted form to perform efficient searches for existing members.
In domains with a large number of group memberships, a join can now be
completed in half the time compared with Samba 4.6.

LDAP search performance has also improved, particularly in the unindexed search
case. Parsing and processing of security descriptors should now be more
efficient, improving replication but also overall performance.

Query record for open file or directory
---------------------------------------

The record attached to an open file or directory in Samba can be
queried through the 'net tdb locking' command. In clustered Samba this
can be useful to determine the file or directory triggering
corresponding "hot" record warnings in ctdb.

Removal of lpcfg_register_defaults_hook()
-----------------------------------------

The undocumented and unsupported function lpcfg_register_defaults_hook()
that was used by external projects to call into Samba and modify
smb.conf default parameter settings has been removed. If your project
was using this call please raise the issue on
samba-technical@lists.samba.org in order to design a supported
way of obtaining the same functionality.

Change of loadable module interface
-----------------------------------

The _init function of all loadable modules in Samba has changed
from:

NTSTATUS _init(void);

to:

NTSTATUS _init(TALLOC_CTX *);

This allows a program loading a module to pass in a long-lived
talloc context (which must be guaranteed to be alive for the
lifetime of the module). This allows modules to avoid use of
the talloc_autofree_context() (which is inherently thread-unsafe)
and still be valgrind-clean on exit. Modules that don't need to
free long-lived data on exit should use the NULL talloc context.

SHA256 LDAPS Certificates
-------------------------

The self-signed certificate generated for use on LDAPS will now be
generated with a SHA256 self-signature, not a SHA1 self-signature.

Replacing this certificate with a certificate signed by a trusted
CA is still highly recommended.

CTDB changes
------------

* CTDB no longer allows mixed minor versions in a cluster

  See the AllowMixedVersions tunable option in ctdb-tunables(7) and also
  https://wiki.samba.org/index.php/Upgrading_a_CTDB_cluster#Policy

* CTDB now ignores hints from Samba about TDB flags when attaching to databases

  CTDB will use the correct flags depending on the type of database.
  For clustered databases, the smb.conf setting
  dbwrap_tdb_mutexes:*=true will be ignored. Instead, CTDB continues
  to use the TDBMutexEnabled tunable.

* New configuration variable CTDB_NFS_CHECKS_DIR

  See ctdbd.conf(5) for more details.

* The CTDB_SERVICE_AUTOSTARTSTOP configuration variable has been
  removed

  To continue to manage/unmanage services while CTDB is running:

  - Start service by hand and then flag it as managed

  - Mark service as unmanaged and shut it down by hand

  - In some cases CTDB does something fancy - e.g. start Samba under
    "nice", so care is needed. One technique is to disable the
    eventscript, mark as managed, run the startup event by hand and then
    re-enable the eventscript.

* The CTDB_SCRIPT_DEBUGLEVEL configuration variable has been removed

* The example NFS Ganesha call-out has been improved

* A new "replicated" database type is available

  Replicated databases are intended for CTDB's internal use to
  replicate state data across the cluster, but may find other
  uses. The data in replicated databases is valid for the lifetime of
  CTDB and cleared on first attach.

Using x86_64 Accelerated AES Crypto Instructions
------------------------------------------------

Samba on x86_64 can now be configured to use the Intel accelerated AES
instruction set, which has the potential to make SMB3 signing and
encryption much faster on client and server. To enable this, configure
Samba using the new option --accel-aes=intelaesni.

This is a temporary solution that is being included to allow users
to enjoy the benefits of Intel accelerated AES on the x86_64 platform,
but the longer-term solution will be to move Samba to a fully supported
external crypto library.

The third_party/aesni-intel code will be removed from Samba as soon as
external crypto library performance reaches parity.

The default is to build without setting --accel-aes, which uses the
existing Samba software AES implementation.

Parameter changes
-----------------

The "strict sync" global parameter has been changed from
a default of "no" to "yes". This means smbd will by default
obey client requests to synchronize unwritten data in operating
system buffers safely onto disk. This is a safer default setting
for modern SMB1/2/3 clients.

The 'ntlm auth' option default is renamed to 'ntlmv2-only', reflecting
the previous behaviour.  Two new values have been provided,
'mschapv2-and-ntlmv2-only' (allowing MSCHAPv2 while denying NTLMv1)
and 'disabled', totally disabling NTLM authentication and password
changes.

smb.conf changes
================

  Parameter Name                     Description             Default
  --------------                     -----------             -------
  allow unsafe cluster upgrade       New parameter           no
  auth event notification            New parameter           no
  auth methods                       Deprecated
  client max protocol                Effective               SMB3_11
                                     default changed
  map untrusted to domain            New value/              auto
                                     Default changed/
                                     Deprecated
  mit kdc command                    New parameter
  profile acls                       Deprecated
  rpc server dynamic port range      New parameter           49152-65535
  strict sync                        Default changed         yes
  password hash userPassword schemes New parameter
  ntlm auth                          New values              ntlmv2-only
----------------------------------------------------------------------------
Comment 2 Florian Best univentionstaff 2017-09-25 10:24:04 CEST
(In reply to Stefan Gohmann from comment #1)
> 'smbclient' changes
> ------------------
> 
> 'smbclient' no longer prints a 'Domain=[...] OS=[Windows 6.1] Server=[...]'
> banner when connecting to the first server. With SMB2 and Kerberos,
> there's no way to print this information reliably. Now we avoid it at all
> consistently. In interactive sessions the following banner is now presented
> to the user: 'Try "help" do get a list of possible commands.'.

We should check if our smbclient parser in UCS@school still works.
Comment 3 Arvid Requate univentionstaff 2017-12-13 20:11:48 CET
samba 4.7.3 has been imported and built.


The Debug-Packages are not yet propper and need some work:

root@master10:~# apt-Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 samba-dbg : Depends: samba (= 2:4.6.1-1A~4.2.0.201711291933) but 2:4.7.3+dfsg-1A~4.3.0.201712121935 is to be installed
E: Unable to correct problems, you have held broken packages.
Comment 4 Arvid Requate univentionstaff 2017-12-21 20:17:25 CET
I've added the /etc/init.d/samba script to uinvention-samba4 and univention-samba.

But some things are not ok yet with the init scripts provided by the Debian samba source package itself: 

1. systemctl stop nmbd   takes about a minute, aparently doing nothing before finally stopping the service. Adding "# pidfile: /var/run/samba/nmbd.pid" to the init script didn't seem to help (after systemctl daemon-reload).

2. The smbd processes don't run as child of the main samba process any more.
Comment 5 Stefan Gohmann univentionstaff 2017-12-27 10:14:11 CET
The samba 4 join aborts with the following output:

[master091] 2017-12-27T09:58:23.424626	Multifile: /etc/samba/smb.conf
[master091]	/usr/lib/univention-install/96univention-samba4.inst: line 781: /etc/init.d/samba: No such file or directory
[master091] 2017-12-27T09:58:25.849999	Create kerberos/kdc
[master091] 2017-12-27T09:58:25.849999	Setting kerberos/kpasswdserver
[master091] 2017-12-27T09:58:25.849999	File: /etc/krb5.conf
[master091] 2017-12-27T09:58:29.094508	Create samba4/function/level
[master091] 2017-12-27T09:58:29.094508	Multifile: /etc/samba/smb.conf
[master091] 2017-12-27T09:58:31.510597	Looking up IPv4 addresses
[master091] 2017-12-27T09:58:31.519026	Looking up IPv6 addresses
[master091] 2017-12-27T09:58:31.519026	No IPv6 address will be assigned
[master091] 2017-12-27T09:58:31.520396	ldb: unable to stat module /usr/lib/x86_64-linux-gnu/samba/ldb : No such file or directory
[master091] 2017-12-27T09:58:32.014975	Setting up share.ldb
[master091] 2017-12-27T09:58:32.014975	ldb: unable to stat module /usr/lib/x86_64-linux-gnu/samba/ldb : No such file or directory
[master091] 2017-12-27T09:58:32.054930	Setting up secrets.ldb
[master091] 2017-12-27T09:58:32.054930	ldb: unable to stat module /usr/lib/x86_64-linux-gnu/samba/ldb : No such file or directory
[master091] 2017-12-27T09:58:32.084994	ldb: unable to stat module /usr/lib/x86_64-linux-gnu/samba/ldb : No such file or directory
[master091] 2017-12-27T09:58:32.084994	WARNING: Module [samba_secrets] not found - do you need to set LDB_MODULES_PATH?
[master091] 2017-12-27T09:58:32.084994	Unable to load modules for /var/lib/samba/private/secrets.ldb: (null)
[master091] 2017-12-27T09:58:32.085395	ERROR(ldb): uncaught exception - None
[master091] 2017-12-27T09:58:32.085989	File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py", line 176, in _run
[master091] 2017-12-27T09:58:32.085989	return self.run(*args, **kwargs)
[master091] 2017-12-27T09:58:32.085989	File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py", line 474, in run
[master091] 2017-12-27T09:58:32.094386	nosync=ldap_backend_nosync, ldap_dryrun_mode=ldap_dryrun_mode)
[master091] 2017-12-27T09:58:32.094386	File "/usr/lib/python2.7/dist-packages/samba/provision/__init__.py", line 2188, in provision
[master091] 2017-12-27T09:58:32.094878	backend_credentials=provision_backend.credentials, lp=lp)
[master091] 2017-12-27T09:58:32.123019	File "/usr/lib/python2.7/dist-packages/samba/provision/__init__.py", line 961, in setup_secretsdb
[master091] 2017-12-27T09:58:32.123019	secrets_ldb = Ldb(path, session_info=session_info, lp=lp)
[master091] 2017-12-27T09:58:32.123019	File "/usr/lib/python2.7/dist-packages/samba/__init__.py", line 114, in __init__
[master091] 2017-12-27T09:58:32.123019	self.connect(url, flags, options)
[master091] 2017-12-27T09:58:32.123019	ERROR: Samba4 provision failed, exiting /usr/share/univention-samba4/scripts/setup-s4.sh
Comment 6 Stefan Gohmann univentionstaff 2017-12-29 15:41:41 CET
Some tests failed because the Samba 4 join was not successful. It looks much better after open TCP port:

https://git.knut.univention.de/univention/ucs/commit/c466bc3941c9ee9f29f7bfbba6d8146418c8ec56

See also:
https://wiki.samba.org/index.php?title=Samba_AD_DC_Port_Usage&type=revision&diff=13625&oldid=13157

Please check and adjust. A changelog entry is missing.
Comment 7 Arvid Requate univentionstaff 2018-01-02 21:28:58 CET
More info regarding the issue of Comment 4:
=========================================================================
root@master10:~# ps fax | grep nmbd
19658 ?        Ss     0:00 /usr/sbin/nmbd
19659 ?        S      0:00  \_ /usr/sbin/nmbd

root@master10:~# systemctl stop nmbd

## takes over a minute, in the mean time the processes lokk like this:
19658 ?        Ss     0:00 /usr/sbin/nmbd
19666 ?        S      0:00  \_ /usr/sbin/nmbd
## And systemd somehow lost control over 19658:

root@master10:~# journalctl -u nmbd
Aug 24 20:58:45 master10 systemd[1]: Starting Samba NMB Daemon...
Aug 24 20:58:45 master10 systemd[1]: nmbd.service: Supervising process 19658 which is not our child. We'll most likely not notice when it exits.
Aug 24 20:58:45 master10 systemd[1]: Started Samba NMB Daemon.
Aug 24 20:58:49 master10 systemd[1]: Stopping Samba NMB Daemon...
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: State 'stop-sigterm' timed out. Killing.
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: Killing process 19658 (nmbd) with signal SIGKILL.
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: Killing process 19666 (nmbd) with signal SIGKILL.
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: Main process exited, code=killed, status=9/KILL
Aug 24 21:00:19 master10 systemd[1]: Stopped Samba NMB Daemon.
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: Unit entered failed state.
Aug 24 21:00:19 master10 systemd[1]: nmbd.service: Failed with result 'timeout'.
=========================================================================

I observed the same with samba-ad-dc.

Additionally we would have to follow https://wiki.samba.org/index.php/Managing_the_Samba_AD_DC_Service_Using_Systemd and mask the "smbd" and "winbind" services on Samba/AD DCs (and "samba-ad-dc" instead on non-DCs).

For these reasons I've rebuilt the Debian package now without the *.service files.
Comment 8 Arvid Requate univentionstaff 2018-01-09 15:24:19 CET
*** Bug 45993 has been marked as a duplicate of this bug. ***
Comment 9 Arvid Requate univentionstaff 2018-02-12 19:54:19 CET
I've imported and build Samba 4.7.5, changelog-4.3.xml adjusted.

The /etc/init.d/samba convenience script is now shipped as UCR template by univention-samba and univention-samba4.
Comment 10 Stefan Gohmann univentionstaff 2018-02-20 14:06:31 CET
Changelog: OK, I've added a note to the release notes bug: Bug #46338

Basic tests: OK (Windows Join, Samba Join, Memberserver, DRS replication, DNS)

Upgrade: OK, i was able to update Samba, mixed environment was successful as well.

Patches:  OK
Comment 11 Stefan Gohmann univentionstaff 2018-03-14 14:38:35 CET
UCS 4.3 has been released:
 https://docs.software-univention.de/release-notes-4.3-0-en.html
 https://docs.software-univention.de/release-notes-4.3-0-de.html

If this error occurs again, please use "Clone This Bug".