Bug 39940 - Repository sync does not work via HTTPS - wget killed by kernel OOM
Repository sync does not work via HTTPS - wget killed by kernel OOM
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Update - Repository administration
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-0-errata
Assigned To: Philipp Hahn
Daniel Tröder
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-13 15:11 CET by Stefan Gohmann
Modified: 2016-01-20 13:22 CET (History)
1 user (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2015-11-13 15:11:44 CET
I've installed a UCS 4,1 master and created a repository via univention-repository-create.

I started a "univention-repository-update net" but it didn't download the packages. The /var/log/univention/repository.log output:
-------------------------------------------------------------------------------
Downloading 724 index files using 10 threads...
Begin time: Fri Nov 13 15:01:26 2015
[10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]...
End time: Fri Nov 13 15:05:24 2015

Proceed indexes: [PPPPPPPPPPPPsh: 1: cannot open 3.0/maintained///errata2/amd64//Packages.gz: No such file
apt-mirror: can't open index in proceed_index_gz at /usr/bin/apt-mirror line 448.
***** Starting univention-repository-update at Fri Nov 13 15:00:38 2015

File: /etc/apt/mirror.list
-------------------------------------------------------------------------------

After switching from https to http, everything works as expected:
2015-11-13 15:07:21: set repository/mirror/server=http://updates.software-univention.de/ old:https://updates.software-univention.de/

Workaround:
 ucr set repository/mirror/server='http://updates.software-univention.de/'
Comment 1 Philipp Hahn univentionstaff 2015-12-11 15:50:17 CET
apt-mirror forks URCV(repository/mirror/threads)=10 processes by default to download the files using "wget". There seems to be a memory leak, since after some time the memory get completely filled and the kernel starts killing the wget processes: Those that get killed release enough memory for the other wget processes to finish.

> $ strace -ff apt-mirror
> 24770 fstat(7, 24770 +++ killed by SIGKILL +++

> $ dmesg
> [81754.430395] Out of memory: Kill process 24774 (wget) score 203 or sacrifice child
> [81754.430398] Killed process 24774 (wget) total-vm:682688kB, anon-rss:187712kB, file-rss:620kB

> # wget --version
> GNU Wget 1.13.4 übersetzt unter linux-gnu.

Bug <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=642563>

repo_admin.py --cherrypick -r 4.0 -s ucs4.0-1 --releasedest 4.1 --dest errata4.1-0 -p wget

Package: wget
Version: 1.13.4-3.27.201512111540
Branch: ucs_4.1-0
Scope: errata4.1-0

r66279 | Bug #39940 wget: Fix memory hog YAML
 wget.yaml
Comment 2 Daniel Tröder univentionstaff 2016-01-20 10:33:00 CET
OK: updated wget was imported to 4.1
OK: manual test "univention-repository-update net" with HTTPS sources
Comment 3 Janek Walkenhorst univentionstaff 2016-01-20 13:22:32 CET
<http://errata.software-univention.de/ucs/4.1/59.html>