Bug 51535 - [4.4] After Server-Password-Change samba process is not started due to Failed to listen on ::1:135
[4.4] After Server-Password-Change samba process is not started due to Failed...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Password changes
UCS 4.4
Other Linux
: P5 normal with 5 votes (vote)
: UCS 4.4-8-errata
Assigned To: Iván.Delgado
Erik Damrose
https://git.knut.univention.de/univen...
:
Depends on:
Blocks: 53597 54356
  Show dependency treegraph
 
Reported: 2020-06-19 16:15 CEST by Christina Scheinig
Modified: 2022-01-19 13:44 CET (History)
10 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.343
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2020052521000591, 2020061821000521, 2020111921000439, 2021041321000112, 2021110121000194, 2021121621000601
Bug group (optional):
Max CVSS v3 score:
requate: Patch_Available+


Attachments
samba4 s-p-c Patch (487 bytes, patch)
2020-06-19 16:15 CEST, Christina Scheinig
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christina Scheinig univentionstaff 2020-06-19 16:15:53 CEST
Created attachment 10402 [details]
samba4 s-p-c Patch

A Partner reported, that two of his customers have the following messages in the samba log, after the server password change took place.
Samba is therefor not started in the morning.


stream_setup_socket: Failed to listen on ::1:135 - NT_STATUS_ADDRESS_ALREADY_ASSOCIATED

Troubleshooting the issue shows that on both customer environments some cifs connections were still established.

Like:

Samba version 4.10.1-Univention
PID     Username     Group        Machine                                   Protocol Version  Encryption           Signing              
----------------------------------------------------------------------------------------------------------------------------------------
32550   EXFTP$       Domain Computers 192.168.24.8 (ipv4:192.168.24.8:42972)    SMB3_11           -                    AES-128-CMAC         

Service      pid     Machine       Connected at                     Encryption   Signing     
---------------------------------------------------------------------------------------------
IPC$         32550   192.168.24.8  Tue Jun 16 17:41:08 2020 CEST    -            AES-128-CMAC
===============================================================================
or 

/bin/df -lhT
Filesystem                Type      Size  Used Avail Use% Mounted on
udev                      devtmpfs  2.0G     0  2.0G   0% /dev
tmpfs                     tmpfs     396M   27M  369M   7% /run
/dev/mapper/vg_ucs-rootfs ext4       22G   16G  4.7G  77% /
[...]
\\192.168.0.24\back    cifs      3.7T  1.2T  2.6T  31% /mnt/back
 
===============================================================================

So we patched the server-password-change script, to make sure the disconnection works.
==========================================================================
 diff -Nuar univention-samba4{.orig,}
--- univention-samba4.orig    2020-06-18 16:33:01.812634377 +0200
+++ univention-samba4    2020-06-18 16:32:10.169712325 +0200
@@ -83,6 +83,16 @@
         exit 1
     fi
 
-    test -x /etc/init.d/samba && /etc/init.d/samba restart
+    if test -x /etc/init.d/samba; then
+        /etc/init.d/samba stop
+        sleep 5
+        pids=$(pgrep smbd)
+        if [ -n "$pids" ]; then
+            pkill -9 smbd
+            /etc/init.d/samba stop
+        fi
+        /etc/init.d/samba start
+    fi
         
 fi
Comment 3 Julia Bremer univentionstaff 2021-01-12 10:24:30 CET
Since 07.01.2021, we've seen a similar problem in our testing environment. (Three times as of today)


Here, it does not happen due to a server-password-change, but while running the joinscripts during the setup of our test machines.
We see in the logs:

ERROR: dns-master091 account not found in local samba
**************************************************************
* ERROR: Failed to create DNS spn account.                   *
*        Please check the samba and the s4-connector logfile.*
**************************************************************

During 97univention-s4-connector.inst, samba is restarted, but fails due to

../../source4/smbd/service_stream.c:374(stream_setup_socket)
  stream_setup_socket: Failed to listen on 10.207.12.210:49154 - NT_STATUS_ADDRESS_ALREADY_ASSOCIATED
../../source4/rpc_server/dcerpc_server.c:3221(add_socket_rpc_tcp_iface)
  service_setup_stream_socket(address=10.207.12.210,port=49154) for samr rpcecho wkssvc mgmt failed - NT_STATUS_ADDRESS_ALREADY_ASSOCIATEDtask_server_terminate: task_server_terminate: [dcerpc: Failed to initialise end points]

Port 49154 is within the range for "Dynamic RPC Ports", while the other reported port belongs to "End Point Mapper (DCE/RPC Locator Service)"
https://wiki.samba.org/index.php/Samba_AD_DC_Port_Usage

Fixing the server-password-change script is not sufficient if this is regularly observed in our test environment.
Comment 9 Christina Scheinig univentionstaff 2021-06-03 12:54:49 CEST
Addition:
If the workaround is added, but a savecopy is made in the same directory with timestamp e.g. univention-samba4_20210413, then the univention-samba4 script is executed twice, caused by the simple regex ^univention-samba4 in /usr/lib/univention-server/server_password_change.
Now the secret is set twice and therefor the KVNO is incremented twice too, so the replication does not work anymore after serverpassword change.

The customer had share access problems
Comment 11 office 2021-07-30 12:25:21 CEST
similar Error happens here, causing intermittent restart for network-share service after running nightly "/usr/lib/univention-server/server_password_change". 

It's not happening on every password-change, but logs showing the similar messages

[2021/07/30 01:11:13.154095,  0, pid=17120] ../../lib/util/become_daemon.c:136(daemon_ready)
  daemon_ready: daemon 'samba' finished starting up and ready to serve connections
[2021/07/30 01:11:13.256619,  0, pid=17145] ../../source4/smbd/service_stream.c:374(stream_setup_socket)
  stream_setup_socket: Failed to listen on ::1:135 - NT_STATUS_ADDRESS_ALREADY_ASSOCIATED
[2021/07/30 01:11:13.256682,  0, pid=17145] ../../source4/rpc_server/dcerpc_server.c:3221(add_socket_rpc_tcp_iface)
  service_setup_stream_socket(address=::1,port=135) for epmapper mgmt failed - NT_STATUS_ADDRESS_ALREADY_ASSOCIATEDtask_server_terminate: task_server_terminate: [dcerpc: Failed to initialise end points]
[2021/07/30 01:11:13.258072,  0, pid=17120] ../../source4/smbd/server.c:371(samba_terminate)
  samba_terminate: samba_terminate of samba 17120: dcerpc: Failed to initialise end points
[2021/07/30 11:11:27.542700,  0, pid=6663] ../../source4/smbd/server.c:587(binary_smbd_main)
  samba version 4.10.18-Univention started.
Comment 13 Arvid Requate univentionstaff 2021-12-01 15:05:09 CET
FYI:
* Please be aware that the samba services are started/restarted/stopped
  by a classic sysv-init script /etc/init.d/samba instead of a systemd unit
* /etc/init.d/samba restart does this sequence:  stop, sleep, start
* /etc/init.d/samba start/stop actually delegates to /etc/init.d/samba-ad-dc /etc/init.d/smbd etc.
* /etc/init.d/smbd should be a no-op on UCS Samba/AD DCs (like primary, backup, replica nodes)
* On UCS Samba/AD DCs the smbd processes (which occupies port 135 amongst others) are automatically started
  as children of the main "samba" process instead (On UCS Managed Nodes/Memberservers the smbd is started directly instead).
* /etc/init.d/samba-ad-dc stop takes quite a bit of care to actually make sure that the "samba" processes actually are dead when the script exists,
  (We patch that script via svn/patches/samba/4.4-0-0-ucs/2:4.10.18-1-errata4.4-8/15_samba4_stop.patch )
  probably originally introduced via Bug 37343
Comment 14 Iván.Delgado univentionstaff 2022-01-18 14:19:46 CET
The patch suggested in the description was applied.
Now the samba process is restarted after server password change.

cdfe193dcb Bug #51535: Update advisory
a6050f0ca6 Bug #51535: After Server-Password-Change samba process is not started


univention-samba4: 8.0.0-37A~4.4.0.202201181413
Comment 15 Erik Damrose univentionstaff 2022-01-19 11:11:27 CET
OK: Adapted u-samba4 server password change script
OK: univention-samba4: 8.0.0-37A~4.4.0.202201181413
OK: yaml, i made some small changes to the text
Verified