Bug 51053 - apps seem to not be installed anymore, when missing in appcenter cache
apps seem to not be installed anymore, when missing in appcenter cache
Status: NEW
Product: UCS
Classification: Unclassified
Component: App Center
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: App Center maintainers
App Center maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-04-01 09:33 CEST by Daniel Tröder
Modified: 2022-06-02 16:01 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 4: Will affect most installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.137
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2021122221000554
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2020-04-01 09:33:00 CEST
Apps seem to not be installed anymore, when switching the appcenter cache and the installed app does not exist in the new active cache.

"univention-app info" does not show the installed app anymore.

Reproducible with apps that exist in the test appcenter, but not the productive appcenter.

This is confusing and leads to errors.

univention-app info should display all installed apps, regardless of origin.

It could display a modified to indicate it, lets say "[test]" for test appcenter. In the following example the app "oxseforucs" is from the test appcenter:

ucsschool=4.4v5 oxseforucs=7.10.3-ucs2[test] mailserver=12
Comment 1 Christina Scheinig univentionstaff 2022-02-07 13:55:11 CET
We have a customer, which does not expose his server to the internet. So the installed apps are not shown in univention-app info anymore.

I guess the cache is outdated?
Comment 2 Frank Greif univentionstaff 2022-02-11 12:19:17 CET
Chiming in on a similar (or the same?) problem, this time without switching between 'test' and 'prod' appcenter servers.

Without any interaction, univention-app info randomly forgets some previously installed apps, randomly shows them again.

* Not clicked in UMC's Appcenter module
* Not called univention-upgrade

The only univention-app call in use is univention-app update every 6th hour, just to get up-to-date information about updates. Is this 'the right way' to refresh the cache, or is it prone to corrupt caches? (and if so: which is the right way then?)

Shall I open a separate issue, or is this deemed really the same problem?
Comment 3 Erik Damrose univentionstaff 2022-02-14 13:39:13 CET
The initial report here mentioned as an easy reproducer when switching from the test app center to the production app center, when a not yet released app version is used. 

All following comments are probably another issue with an unreliable app center cache rebuild on individual UCS systems, a regression that is fixed at bug 54452
Comment 4 Frank Greif univentionstaff 2022-02-14 13:58:05 CET
Thanks for the hint to Bug 54452 -- but that bug is opened as a regression in UCS 5, while this bug is about a behavior in 4.4. But maybe the info there can help me tracing down the problem. I'll be back when I've spotted something.
Comment 5 Frank Greif univentionstaff 2022-03-21 12:15:38 CET
Back from #54452 -- it seems it was a different problem indeed. So the problem persists, here in 4.4.

I see the errorneous behavior "apps seem to not be installed anymore" even in the situation that I never switched between test and prod appcenter. Currently I don't see any pattern, just randomly some apps disappear.

The only remarkable pattern is that `univention app list` always shows the truth (apps being installed where they really are) but `univention app info` does not show them. And when some apps are missing, even `univention app update` does not get them back (always reproducible).

So I would think it does not relate to a switch between software sources; there must be something wrong in the way the cache is built or rebuilt.
Comment 6 Erik Damrose univentionstaff 2022-03-21 14:24:49 CET
My previous comment was not that elaborate, because bug 54452 was an issue that required immediate action at the time - sorry about that.

The initial report about the caching issues (which i still suspect are reported here) happened at bug 51986, which was only fixed for UCS 5. If a backport is required, please clone bug 51986.
Comment 7 Frank Greif univentionstaff 2022-06-02 15:42:42 CEST
While trying write some kind of consistency check, I came across an even weirder behaviour: on one of the affected systems the appcenter (or whatever) even deleted the UDM/LDAP entry of the installed app. Interestingly, the app container object

`cn=appname,cn=apps,cn=univention,$ldap_base`

is still there, but the 'installed app' beneath it is missing.

In the appcenter.log, I find

`
22-06-01 16:49:50 [    INFO]: onlyoffice-ds
22-06-01 16:49:50 [    INFO]:   Name: ONLYOFFICE Docs
22-06-01 16:49:50 [    INFO]:   Latest version: 6.3.1.33
22-06-01 16:49:50 [    INFO]:   Installations: my.hostna.me
`
then
`
22-06-01 16:49:51 [   DEBUG]: Calling info
22-06-01 16:49:51 [   DEBUG]: 0
22-06-01 16:49:51 [    INFO]: UCS: 4.4-9 errata1233
22-06-01 16:49:51 [   DEBUG]: Cache outdated. Need to rebuild
22-06-01 16:49:51 [   DEBUG]: Loaded 449 apps from cache
22-06-01 16:49:51 [   DEBUG]: Cache outdated. Need to rebuild
22-06-01 16:49:51 [   DEBUG]: Loaded 255 apps from cache
22-06-01 16:49:51 [   DEBUG]: Cache outdated. Need to rebuild
22-06-01 16:49:52 [   DEBUG]: Loaded 220 apps from cache
22-06-01 16:49:52 [   DEBUG]: Cache outdated. Need to rebuild
22-06-01 16:49:52 [   DEBUG]: Loaded 399 apps from cache
22-06-01 16:49:52 [    INFO]: Installed: (onlyoffice-ds is missing here!)
`
the next mention of onlyoffice-ds is:
`
22-06-01 16:56:01 [    INFO]: No hostdn for onlyoffice-ds found. Nothing to remove
`
(and the pseudo-hostname for the app is really missing)
and suddenly the next univention-app list shows:
`
22-06-01 18:58:50 [    INFO]: onlyoffice-ds
22-06-01 18:58:50 [    INFO]:   Name: ONLYOFFICE Docs
22-06-01 18:58:50 [    INFO]:   Latest version: 6.3.1.33
22-06-01 18:58:50 [    INFO]:   Installations: 
`

Yes I know of the 'pragmatic' way of removing the cache altogether, but please somebody tell me how I can check that the cache or whatever is damaged -- currently it was just a random encounter: use univention-app info, know for sure that onlyoffice-ds IS installed and find it missing.
Comment 8 Frank Greif univentionstaff 2022-06-02 16:01:53 CEST
Found some more traces in the listener.log.
`
01.06.22 14:37:41.674  LISTENER    ( PROCESS ) : updating 'cn=onlyo-96641501,cn=memberserver,cn=computers,$ldap_base' command d
Revoke certificates: onlyo-96641501.my.doma.in
Using configuration from /etc/univention/ssl/openssl.cnf
Revoking Certificate 0A.
Data Base Updated
unable to write 'random state'
Using configuration from /etc/univention/ssl/openssl.cnf
unable to write 'random state'
Revoke certificates: *.onlyo-96641501.my.doma.in
awk: warning: escape sequence `\*' treated as plain `*'
no certificate for *.onlyo-96641501.my.doma.in registered
01.06.22 14:37:42.834  LISTENER    ( PROCESS ) : updating 'cn=Computers,cn=groups,$ldap_base' command m
01.06.22 14:37:49.586  LISTENER    ( PROCESS ) : updating 'cn=dudle,cn=portal,cn=univention,$ldap_base' command m
01.06.22 14:37:49.616  LISTENER    ( PROCESS ) : updating 'cn=kopano-webapp,cn=portal,cn=univention,$ldap_base' command m
01.06.22 16:56:06.994  LDAP        ( PROCESS ) : connecting to ldap://my.hostna.me:7389
01.06.22 16:56:07.019  LISTENER    ( PROCESS ) : updating 'univentionAppID=onlyoffice-ds_6.3.1.33,cn=onlyoffice-ds,cn=apps,cn=univention,$ldap_base' command m
01.06.22 16:56:07.048  LISTENER    ( PROCESS ) : app_attributes: Gathering AppAttributes...
01.06.22 16:56:08.449  LISTENER    ( PROCESS ) : app_attributes: Finished
01.06.22 16:56:08.517  LISTENER    ( PROCESS ) : updating 'univentionAppID=onlyoffice-ds_6.3.1.33,cn=onlyoffice-ds,cn=apps,cn=univention,$ldap_base' command d
01.06.22 16:56:08.518  LISTENER    ( PROCESS ) : app_attributes: Gathering AppAttributes...
01.06.22 16:56:08.760  LISTENER    ( PROCESS ) : app_attributes: Finished
`
Maybe somebody can spot WHO did that or even WHY...

(for the sake of completeness: the UDM appcenter module did not log anything but the entries already found in the appcenter.log)