Univention Bugzilla – Bug 51313
univention-directory-listener segfault when importing python apt
Last modified: 2023-03-07 07:30:50 CET
Listener segfault This isn't directly a listner problem but a module problem. At least with "app_attributes.py" It can be replicated with: python -c "from univention.listener.handler import ListenerModuleHandler; from univention.appcenter.app_cache import AllApps" The problem might just be that I had the 4.x repositories activated to install univention packages.
You then get a mix of Debian-{Stretch,Buster}-Binary-packages. I also re-did your experiment with both 4.4-4 and 5.0-0 being included and no crash.
The App Center uses python-apt with its C bindings to find out which packages are installed. But I don't think we actively use this in the listener module...
... and ldap (via UDM)
Works with the latest DVD
Don't know if this is related, but during the manual "univention-upgrade" phase just before we can run the "Configure system" step in the latest DVD, I got this: /var/lib/dpkg/info/python-univention-appcenter.postinst: line 42: 6686 Segmentation fault univention-config-registry register python-univention-appcenter So this may be the C bindings of UCR?
I was wrong :( This is still a problem. For some reason I had one installation there the listener did work. But on another it fails. I was able to isolate the problem a bit further: ``` python -c "import _ldap; _ldap.get_option(_ldap.OPT_API_INFO); import apt" ``` (This seg faults on both systems, the order is important) A temporary workaround is making sure that "apt" is always imported before "ldap" ``` echo "import apt" >> /etc/univention/templates/files/etc/python2.7/sitecustomize.py.d/00header.py ucr commit /etc/python2.7/sitecustomize.py ```
Note: I installed from the latest ucs 5 dvd and had the ucs 4 sources disabled. I can NOT reproduce this on a normal debian buster system.
Happens also with Python 3: gdb --args python3 -c "import _ldap; _ldap.get_option(_ldap.OPT_API_INFO); import apt" $ run $ bt #0 0x00007ffff667a499 in GlobalError::PushToStack() () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0 #1 0x00007ffff66f5ed2 in pkgInitConfig(Configuration&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0 #2 0x00007ffff6c0d608 in InitConfig (Self=<optimized out>, Args=<optimized out>) at python/apt_pkgmodule.cc:484 #3 0x00000000005d6fb2 in _PyMethodDef_RawFastCallKeywords (method=<optimized out>, self=0x7ffff74809a8, args=0x7ffff7591378, nargs=<optimized out>, kwnames=<optimized out>) at ../Objects/call.c:694 …
Created attachment 10384 [details] The patch fixes the segfault but probably breaks thread safety The bug seems to be with the thread_local initialization of the the GlobalError object in apt-pkg. If that really is the case I guess this is just a side effect of an other bug and not a bug in apt-pkg? In gdb it looks like the "GlobalError" object is not initialized on first use. Instead a already initialized list with a pointer to some (random?) memory address is returned. Which results in the segfault.
The use of "thread_local" is a refactoring commit. This can be reverted and pthread is instead used again. This fixes the segmentation fault. I have to admit I don't know why this is a problem on ucs. The commits in the apt repo are: "Use C++11 threading support instead of pthread" b28636e54f682dc65aef169f201a34a8e2893369 "Fix memory leak in C++-thread-local _error implementation" 414b7d57eda362328568484957fbd6c06b1b0d1f The revert has been added to our patches repo: r19032: Bug #51313: apt is a debian internal package -> use .patch r19031: Bug #51313: Use pthread in apt-pkg instead of thread_local
FYI: apt/5.0-0-0-ucs/1.8.2/11-silence-warning.patch can be dropped - we generate correct Release files for a long time: > grep ^Codename */Release > all/Release:Codename: ucs_5.0-0/all > amd64/Release:Codename: ucs_5.0-0/amd64 > source/Release:Codename: ucs_5.0-0/source TODO: I find 20-Bug-51313-revert-thread_local-usage.patch very disturbing as APT is quiet important for UCS and Debian. A bug in that component I consider critical. I'm wondering by Debian is not affected by this issue? - Do wo have a problem with out tool- and build-chain? - On the other hand there are more bug reports when searching for "GlobalError libapt-pkg.so.5.0", so that issue should be tracked and reported to Debian. For now I accept your solution but will track that myself. OK: # SELECT * FROM binpkg WHERE major=5 AND site='apt' AND srcpkg='apt'; binpkg | binver | arch | srcpkg | srcver | major | minor | patch | scope | site | maintained | id -----------------------+---------------------------+-------+--------+---------------------------+-------+-------+-------+-------+------+------------+---------- apt-doc | 1.8.2A~5.0.0.202006111148 | all | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18316090 apt-transport-https | 1.8.2A~5.0.0.202006111148 | all | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18316091 libapt-pkg-doc | 1.8.2A~5.0.0.202006111148 | all | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18316092 apt-dbgsym | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344734 apt-utils-dbgsym | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344735 apt-utils | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344736 apt | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344737 libapt-inst2.0-dbgsym | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344738 libapt-inst2.0 | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344739 libapt-pkg-dev | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344740 libapt-pkg5.0-dbgsym | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344741 libapt-pkg5.0 | 1.8.2A~5.0.0.202006111148 | amd64 | apt | 1.8.2A~5.0.0.202006111148 | 5 | 0 | 0 | | apt | | 18344742 (12 Zeilen) OK: python -m univention.repong.dchdiff -l 1 amd64/apt_1.8.2A~5.0.0.202006111148_amd64.deb
The lasso patch has fixed this: lasso/5.0-0-0-ucs/2.6.0-2/10_expose_lasso_provider_verify_saml_signature.quilt The apt patch could be dropped again: apt/5.0-0-0-ucs/1.8.2.1/20-Bug-51313-revert-thread_local-usage.patch I'm still wondering if this was just a symptom or a bug in the error handling. Note: lasso is used by crudesaml, which is used for sasl by python-ldap, which was imported before apt.
UCS 5.0 has been released: https://docs.software-univention.de/release-notes-5.0-0-en.html https://docs.software-univention.de/release-notes-5.0-0-de.html If this error occurs again, please use "Clone This Bug".