Comment 1 Christian Völker 2020-11-03 14:03:00 CET

Interesting:

When running univention-app update shortly after the failure the request just downloads fine!

Comment 2 Christian Völker 2020-11-03 14:04:47 CET

could reproduce.

Remote the cache as above, trying to update will end in failure, running update again will succeed!

Comment 3 Ingo Steuwer 2020-11-06 11:56:44 CET

Am I right that this fails in the first attempt but works later, right? And it happens only if the App Center cache data is deleted, which normally should'n happen, right?

I change the "How will those affected feel about the bug" and "What type of bug is this?" based on these assumptions.

Comment 4 Christina Scheinig 2020-11-06 15:59:41 CET

For the customer, switching off the proxy for the hosts of Univention and allow the UCS hosts (possibly only temporarily) direct Internet access, is no option.

They would like to deactivate the update test, till this bug is fixed.

Is there a possibility to deactivate this univention-app update test, which seems to be the problem, because the cache is never up to date, with the proxy activated.

Comment 5 Ingo Steuwer 2020-11-06 16:13:04 CET

(In reply to Ingo Steuwer from comment #3)
> Am I right that this fails in the first attempt but works later, right? And
> it happens only if the App Center cache data is deleted, which normally
> should'n happen, right?
> 
> I change the "How will those affected feel about the bug" and "What type of
> bug is this?" based on these assumptions.

I set "Needmoreinfo" as I don't understand the consequences for the customer.

Comment 6 Christina Scheinig 2020-11-06 17:05:56 CET

So the customer described the issue as follows:

"The UCS systems try to pull an update. The proxy and the FW allow this. But the UCS systems ALWAYS request the same file without any progress. This puts so much load on the proxy that it can hardly perform any other actions, while at the same time a lot of logs are written by the proxy. ( > 2 Mio in a few minutes )"

Some update check routine makes a zsync. For example:

zsync http://appcenter.software-univention.de/meta-inf/4.4/all.tar.zsync -q -o /var/cache/univention-appcenter/appcenter.software-univention.de/4.4/.all.tar

This is executed but never stopped or not finished for days, making the UCS millions of accesses to the proxy.

Does this description help?
So for the customer, I suppose, it blocks further progress of the daily work.

Comment 7 Ingo Steuwer 2020-11-09 09:53:21 CET

(In reply to Christina Scheinig from comment #6)
> So the customer described the issue as follows:
> 
> "The UCS systems try to pull an update. The proxy and the FW allow this. But
> the UCS systems ALWAYS request the same file without any progress. This puts
> so much load on the proxy that it can hardly perform any other actions,
> while at the same time a lot of logs are written by the proxy. ( > 2 Mio in
> a few minutes )"
> 
> Some update check routine makes a zsync. For example:
> 
> zsync http://appcenter.software-univention.de/meta-inf/4.4/all.tar.zsync -q
> -o
> /var/cache/univention-appcenter/appcenter.software-univention.de/4.4/.all.tar
> 
> This is executed but never stopped or not finished for days, making the UCS
> millions of accesses to the proxy.
> 
> Does this description help?
> So for the customer, I suppose, it blocks further progress of the daily work.

mhm, sounds more like a problem in the proxy or its configuration?

Comment 9 Christina Scheinig 2020-11-09 11:52:56 CET

(In reply to Ingo Steuwer from comment #7)
> (In reply to Christina Scheinig from comment #6)
> > So the customer described the issue as follows:
> > 
> > "The UCS systems try to pull an update. The proxy and the FW allow this. But
> > the UCS systems ALWAYS request the same file without any progress. This puts
> > so much load on the proxy that it can hardly perform any other actions,
> > while at the same time a lot of logs are written by the proxy. ( > 2 Mio in
> > a few minutes )"
> > 
> > Some update check routine makes a zsync. For example:
> > 
> > zsync http://appcenter.software-univention.de/meta-inf/4.4/all.tar.zsync -q
> > -o
> > /var/cache/univention-appcenter/appcenter.software-univention.de/4.4/.all.tar
> > 
> > This is executed but never stopped or not finished for days, making the UCS
> > millions of accesses to the proxy.
> > 
> > Does this description help?
> > So for the customer, I suppose, it blocks further progress of the daily work.
> 
> mhm, sounds more like a problem in the proxy or its configuration?

I don't understand. afais Christian reproduced the issue, in his testenvironment, so why it is now a misconfiguration of the customers proxy?

Comment 10 Ingo Steuwer 2020-11-09 16:15:35 CET

(In reply to Christina Scheinig from comment #9)
> > 
> > mhm, sounds more like a problem in the proxy or its configuration?
> 
> I don't understand. afais Christian reproduced the issue, in his
> testenvironment, so why it is now a misconfiguration of the customers proxy?

Seems like we need to remove misunderstandings. My point was:

1. there are two tickets linked, based on a discussion with Dirk the ticket  2020102921000109 which I had a look at might not related to this bug. In this ticket it is written that the proxy configuration of the customer is suspected to cause the issues, not a bug in UCS.

2. the documented way to reproduce this issue is "Remove the cache as above, trying to update will end in failure, running update again will succeed!" which I understand as (deliberately exaggerated):
- I intentionally break the UCS System
- at first I have trouble in the component
- but in the end everything is fine and fixes it by itself

So, the main question for me is: How is the behaviour of the UCS App Center in case it is running behind a full functional proxy?

Comment 11 Christian Völker 2020-11-30 12:32:42 CET

(In reply to Ingo Steuwer from comment #10)


At first, cleaning a cache is NOT "breaking a system". Any system using a cache should work flawlessly when the cache is empty or not existing!
How about systems being installed and thus not having such a cache? They will fail...

Second, you are right, it fixes itself when manually restarting the process. Which does not happen when the update is triggered by UMC, udm, cron or whatever other way. Any following attempt still fails then. It fixes itself only when doing it again (ie for testing purpose) manually on shell.

Third, as already written the behavior is seen in an ucs-only environment with no special configurations of the Squid proxy. 

If this is a proxy issue it is a product issue, too.

Comment 12 Ingo Steuwer 2020-11-30 20:41:02 CET

(In reply to Christian Völker from comment #11)

thanks a lot for the clarification:

> (In reply to Ingo Steuwer from comment #10)
> 
> 
> At first, cleaning a cache is NOT "breaking a system". Any system using a
> cache should work flawlessly when the cache is empty or not existing!
> How about systems being installed and thus not having such a cache? They
> will fail...

yes, caches might be deleted and systems should handle that. I was refering to the warning/error messages seen - it is OK for me that a system gives some error message in case expected cache files are missing.

> 
> Second, you are right, it fixes itself when manually restarting the process.
> Which does not happen when the update is triggered by UMC, udm, cron or
> whatever other way. Any following attempt still fails then. It fixes itself
> only when doing it again (ie for testing purpose) manually on shell.

OK, so to summarize the situation:
- everything is fine while using appcenter command line tools
- cache updates and maybe further steps fail for web interface interaction with the App Center -> this is a bug
 
> Third, as already written the behavior is seen in an ucs-only environment
> with no special configurations of the Squid proxy. 
> 
> If this is a proxy issue it is a product issue, too.

I was referring to the support ticket, where the customer reports that the proxy gets unusable / unresponsive by requests send by the App Center; and in that environment the App Center reports invalid / no responses from the proxy. For me that is a bug in the proxy configuration / implementation and not in UCS.

Comment 13 Christian Völker 2020-11-30 22:45:45 CET

(In reply to Ingo Steuwer from comment #12)

> OK, so to summarize the situation:
> - everything is fine while using appcenter command line tools
I am unsure about this- it prints loads of error messaages but I do not know if it does what it should do!

> - cache updates and maybe further steps fail for web interface interaction
> with the App Center -> this is a bug
Which might be related to the above error messages.

> I was referring to the support ticket, where the customer reports that the
> proxy gets unusable / unresponsive by requests send by the App Center; and
> in that environment the App Center reports invalid / no responses from the
> proxy. For me that is a bug in the proxy configuration / implementation and
> not in UCS.
Well, if the proxy does not handle these requests properly it is indeed not an issue of UCS. But the bug here is univention-app does not work properly with a default proxy (the one configured in UCS@school by default).

So the bug is:
univention-app does not deal properly when using a proxy (configured in default way on UCS). Symptoms are:
-from UMC hanging and/or failing processes
-from command line error messages related to zsync

Comment 15 Mirac Erdemiroglu 2025-08-21 12:30:36 CEST

1. Summary
When executing the Univention App Center update check, the process fails and not working through a proxy and runs into a timeout. The root cause is traced back to the zsync utility, which does not support HTTPS connections and therefore cannot retrieve required control files.

2. Environment

    Product: Univention Corporate Server (UCS)

    Affected Component: Univention App Center update-check mechanism (univention-app update or univention-app update-check --version X.X)

    Network: Uses proxy with enforced HTTPS

3. Steps to Reproduce

    Execute the update check using univention-app update-check --ucs-version 5.1 through proxy.

    Observe that the update process starts and attempts to download various metadata files.

    The process eventually stalls during the download of http://appcenter.software-univention.de/meta-inf/5.1/all.tar.zsync, resulting in a timeout after approximately 15 minutes.

4. Observed Behavior
The appcenter.log show multiple download attempts for metadata files, followed by a zsync invocation that fails with warnings such as "bad status code 504" and "failed to retrieve from http://appcenter.software-univention.de/meta-inf/5.1/all.tar.gz

". The process falls back to a direct download but ultimately ends in failure with a message indicating that the update to UCS 5.1 is currently not possible due to missing app compatibility. The total execution time was approximately 15 minutes.

5. Expected Behavior
The update check should download all necessary metadata and app archives successfully, using a secure protocol (HTTPS) where applicable, without unnecessary delays or timeouts.

6. Analysis

    The logs indicate that zsync is invoked to download the app archive (all.tar.zsync).

    The tool zsync does not support HTTPS due to its outdated codebase and lack of integration with SSL/TLS libraries such as OpenSSL or GnuTLS.

    As a result, it fails to retrieve the control file over HTTPS and causes a prolonged timeout.


 21409 actions.update-check             25-08-19 12:22:33 [   DEBUG]: Calling update-check
 21409 actions.update-check.progress    25-08-19 12:22:33 [   DEBUG]: 0
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]: Calling update
 21409 actions.update.progress          25-08-19 12:22:33 [   DEBUG]: 0
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/app-categories.ini"...
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/rating.ini"...
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/license_types.ini"...
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/ucs.ini"...
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/suggestions.json"...
 21409 update-check                     25-08-19 12:22:33 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:33 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/5.0/all.tar.gpg"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/4.4/all.tar.gpg"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/4.3/all.tar.gpg"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 actions.update.progress          25-08-19 12:22:34 [   DEBUG]: 100
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]: Calling update
 21409 actions.update.progress          25-08-19 12:22:34 [   DEBUG]: 0
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/app-categories.ini"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/rating.ini"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/license_types.ini"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/ucs.ini"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/suggestions.json"...
 21409 update-check                     25-08-19 12:22:34 [   DEBUG]:   ... Not Modified
 21409 update-check                     25-08-19 12:22:34 [    INFO]: Downloading "https://appcenter.software-univention.de/meta-inf/5.1/all.tar.gpg"...
 21409 update-check                     25-08-19 12:22:35 [    INFO]: Downloading "http://appcenter.software-univention.de/meta-inf/5.1/all.tar.zsync"...
 21409 update-check                     25-08-19 12:22:35 [   DEBUG]: Calling in /tmp/tmpcz9nnbv1:
 21409 update-check                     25-08-19 12:22:35 [   DEBUG]: Calling zsync http://appcenter.software-univention.de/meta-inf/5.1/all.tar.zsync -q -o /tmp/tmpcz9nnbv1/.tmp.tar -i /tmp/tmpcz9nnbv1/.all.tar
 21409 update-check                     25-08-19 12:22:36 [ WARNING]: open: No such file or directory
 21409 update-check                     25-08-19 12:22:36 [ WARNING]: not using seed file /tmp/tmpcz9nnbv1/.all.tar
 21409 update-check                     25-08-19 12:37:37 [ WARNING]: bad status code 504
 21409 update-check                     25-08-19 12:37:37 [ WARNING]: failed to retrieve from http://appcenter.software-univention.de/meta-inf/5.1/all.tar.gz
 21409 update-check                     25-08-19 12:37:37 [ WARNING]: Aborting, download available in /tmp/tmpcz9nnbv1/.tmp.tar.part
 21409 update-check                     25-08-19 12:37:37 [ WARNING]: Downloading the App archive via zsync failed. Falling back to download it directly.
 21409 update-check                     25-08-19 12:37:37 [ WARNING]: For better performance, try to make zsync work for "http://appcenter.software-univention.de/meta-inf/5.1/all.tar.zsync". The error may be caused by a proxy altering HTTP requests


7. Proposed Solution

    Replace zsync with zsync-curl, which supports HTTPS, in the update-check mechanism.
    
    The original zsync tool was developed many years ago and has received only a few updates since then. Its underlying code does not include integration with libraries such as OpenSSL or GnuTLS, which are required to handle encrypted HTTPS connections. As a result, it cannot process encrypted URLs (https://) and terminates with an error stating that the control file (*.zsync) could not be read.

    Alternatively, ensure that the App Center provides HTTP fallback URLs or skips zsync for HTTPS-only resources.

8. Impact

    Prolonged update checks (15+ minutes).

    Inability to upgrade to UCS 5.1 when apps are blocked due to missing metadata retrieval.

    Reduced system availability during the update process.

9. Additional Information
Directly invoking zsync on the same URL (https://appcenter.software-univention.de/meta-inf/5.1/all.tar.zsync) results in the error: "could not read control file from URL".
This confirms that the issue is directly tied to zsync’s lack of HTTPS support.

Comment 16 Fabian Schneider 2025-09-18 09:05:43 CEST

Bump the priority:

What type of bug is this?:	
6: Setup Problem: Issue for the setup process

-> it's not possible to setup applications if the cache is invalid and cannot be fetched


How will those affected feel about the bug?:	
4: A User would return the product

-> the client just communicated that's nothing he can accept on the long run

Comment 20 Marius Meschter 2025-11-12 15:01:47 CET

univention-appcenter.yaml
afb00a00c1bf | fix(appcenter): add a configurable timeout to the zsync call

univention-appcenter (11.3.3)
afb00a00c1bf | fix(appcenter): add a configurable timeout to the zsync call

univention-appcenter (11.3.2)
4dbfa2d1b19e | feat(appcenter): add UCR variable to skip zsync for proxy compatibility

ucs-test (12.3.34)
c62c34bb7eca | test(appcenter): add test for appcenter update app download mechanism

Added the UCRV appcenter/update/skip-zsync to skip downloading app metadata via zsync and directly use direct HTTPS download.

Added appcenter/update/zsync-timeout to stop zsync after the specified timeout expires.

Bug for backport to 5.0-10 #58798

Comment 21 Christian Castens 2025-11-19 15:12:45 CET

<https://errata.software-univention.de/#/?erratum=5.2x285>