Bug 38697 - univention-upgrade should suggest App Center updates
univention-upgrade should suggest App Center updates
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
UCS 4.0
Other Linux
: P5 enhancement (vote)
: UCS 4.0-2-errata
Assigned To: Dirk Wiesenthal
Felix Botner
:
Depends on:
Blocks: 30417 33144 39284
  Show dependency treegraph
 
Reported: 2015-06-12 12:52 CEST by Philipp Hahn
Modified: 2015-10-05 07:21 CEST (History)
9 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Fix Thread(kwargs) (1.33 KB, patch)
2015-07-20 09:52 CEST, Philipp Hahn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2015-06-12 12:52:48 CEST
The App Center needs functions the updater can use to query and trigger for App updates.

+++ This bug was initially created as a clone of Bug #30417 +++

Using "univention-upgrade" or the Online Update UMC Module tells the user the system is up to date even if there are updates in the App Center available. It would be easier for the user to be informed about available updates or at least to be informed about the other tool/UMC-module to check for updates.
Comment 1 Dirk Wiesenthal univentionstaff 2015-07-13 14:08:30 CEST
Added python-univention-appcenter in
  univention-management-console-module-appcenter 4.1.20-36.378.201507131337
Comment 2 Florian Best univentionstaff 2015-07-15 11:04:39 CEST
FYI: Part of the commit has been accidentally merged to UCS 4.1 in svn r62115.
Comment 3 Philipp Hahn univentionstaff 2015-07-20 09:50:35 CEST
# univention-add-app -a xrdp_20150720
...
20.07.15 09:46:56.041 MODULE      ( PROCESS ) : Running join scripts
Traceback (most recent call last):
  File "/usr/sbin/univention-add-app", line 140, in <module>
    success = requested_app.install(package_manager, component_manager, add_component=False, send_as=function, only_master_packages=only_master_packages) and success
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter/app_center.py", line 1622, in install
    self._finished(send_as, status, component_manager.ucr, package_manager, username, password)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter/app_center.py", line 1627, in _finished
    self._run_joinscripts(package_manager, username, password, ucr)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter/app_center.py", line 1643, in _run_joinscripts
    run_join_scripts = Thread(target=subprocess.call, args=[args], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
TypeError: __init__() got an unexpected keyword argument 'stderr'

r62070
Comment 4 Philipp Hahn univentionstaff 2015-07-20 09:52:59 CEST
Created attachment 7031 [details]
Fix Thread(kwargs)
Comment 5 Dirk Wiesenthal univentionstaff 2015-08-03 00:34:15 CEST
Thanks.

univention-management-console-module-appcenter 4.1.20-44.389.201508021808
Comment 6 Felix Botner univentionstaff 2015-08-04 11:56:35 CEST
I applied the univention-upgrade patch from Bug #30417

* FAIL - changes are not merged to 4.1, why?

* FAIL - YAML

* FAIL - app update - to much output (File: ...), the actual update works

  Do you want to update Horde Groupware Webmail Edition [Y|n]? y
  Username [Administrator]: 
  Password for Administrator: 
  > File: /etc/apt/sources.list.d/20_ucs-online-component.list
  > File: /var/www/ucs-overview/entries.json
  > File: /etc/apt/mirror.list
  > File: /etc/apt/sources.list.d/15_ucs-online-version.list
  > File: /usr/share/univention-management-console/modules/apps.xml
  > File: /usr/share/univention-management-console/i18n/de/apps.mo
  > File: /etc/apt/apt.conf.d/55user_agent
  done
  Checking for release updates:                           none
  Checking for package updates:                           none
  Checking for app updates:                               none
  Setting update/available

* FAIL - Username and Password parameters
  * There is no explanation for these parameters (i guess for the join)
    Does the target group for univention-upgrade know what's going on?
  * What is the worst thing that can happen if username or password is wrong?
  * Normally, univention-upgrade does not execute join scripts, why now
    for apps? Why not just rely on univention-run-join-scripts/UMC-Join module

* FAIL - noninteractive upgrade
  -> univention-upgrade --noninteractive
  Starting univention-upgrade. Current UCS version is 4.0-2 errata264
  Checking for local repository:                          none
  Checking for release updates:                           none
  Checking for package updates:                           none
  Checking for app updates:                               found
  The following apps can be updated:
  EGroupware: Version 14.2.20150717 can be updated to 14.2.20150729
  Starting app update                  An error occurred -stopping here.
  -> updater.log:
  Starting app update                                    
  Traceback in univention-upgrade:
  Traceback (most recent call last):
    File "/usr/sbin/univention-upgrade", line 416, in main
      performUpdate(options.updateto, ignoressh=options.ignoressh, 
        ignoreterm=options.ignoreterm, interactive=not options.noninteractive, 
        silent=False)
    File "/usr/sbin/univention-upgrade", line 323, in performUpdate
      if run_app:
  UnboundLocalError: local variable 'run_app' referenced before assignment

* FAIL - utils.app_is_running
  import univention.appcenter.utils
  print univention.appcenter.utils.app_is_running('tine20')
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
      File "/usr/lib/pymodules/python2.7/univention/appcenter/utils.py", 
       line 54, in app_is_running
      from univention.appcenter.app import AppManager
    ImportError: No module named app

* OK -> update found, but aborted
  -> univention-upgrade 
  Starting univention-upgrade. Current UCS version is 4.0-2 errata264
  Checking for local repository:                          none
  Checking for release updates:                           none
  Checking for package updates:                           none
  Checking for app updates:                               found
  The following apps can be updated:
  EGroupware: Version 14.2.20150717 can be updated to 14.2.20150729
  Starting app update                                    
  Do you want to update EGroupware [Y|n]? n
  done
  Setting update/available
  -> ucr get update/available 
  yes

* OK - app update
  from univention.appcenter.actions import get_action
  from univention.management.console.modules.appcenter.app_center import
    Application
  app = 'egroupware'
  app = Application.find(app)
  app_upgrade = get_action('upgrade')
  app_upgrade = app_upgrade()
  app_upgrade.call(app=app)

* OK - install
  from univention.appcenter.actions import get_action
  from univention.management.console.modules.appcenter.app_center import   
    Application
  app = Application.find('tine20')
  app_install = get_action('install')
  app_install = app_install()
  app_install.call(app=app)

* OK - univention-add-app -a xrdp_20150720


ps.
Just a personal note. The "API" seems a bit confusing to me.
Why this "from univention.appcenter.actions import get_action" and "get_action('upgrade')" stuff?
Why is there no appcenter and app classes in python-univention-appcenter?
Why not (app) class methods for these actions?

appcenter = univention.appcenter()
for app in appcenter.installed_apps()
  if app.update_available():
    app.upgrade()
app = appcenter.find(app='xyz')
app.install()
...
Comment 7 Dirk Wiesenthal univentionstaff 2015-08-05 00:55:27 CEST
(In reply to Felix Botner from comment #6)
> I applied the univention-upgrade patch from Bug #30417
> 
> * FAIL - changes are not merged to 4.1, why?
> 

Will be part of the docker patch. I opened Bug#39082 specifically for this one.

> * FAIL - YAML
> 

Added

> * FAIL - app update - to much output (File: ...), the actual update works

Yes, I know. Not too happy about that. I have suppressed all other writings to stdout by redirecting a logger instance. But these "File:" statements come from UCR itself:
  config_registry/handler.py
    print 'File: %s' % self.to_file
Not so much I can do except suppressing stdout itself which I do not want to do.

I think univention-upgrade does a similar thing by doing the actual work in a
  apt-get distupgrade > /var/log/univention/updater.log

Are these lines a blocker so that I shall do it somehow like that?

> * FAIL - Username and Password parameters
>   * There is no explanation for these parameters (i guess for the join)
>     Does the target group for univention-upgrade know what's going on?

Yes, it is for the join (always). And sometimes for DefaultPackagesMaster, too. Good point.
I think the credentials (and why they should be entered) should be asked by univention-upgrade.

>   * What is the worst thing that can happen if username or password is wrong?

Only the join should fail. But this should not abort anything.
If the app has DefaultPackagesMaster and the system is a DC Slave, though, the installation should fail.

>   * Normally, univention-upgrade does not execute join scripts, why now
>     for apps? Why not just rely on univention-run-join-scripts/UMC-Join
> module

Because the App Center does it like this.

> 
> * FAIL - noninteractive upgrade

Sorry the patch in Bug#30417 is broken. This is not App Center code that fails.

> 
> * FAIL - utils.app_is_running

I removed this funtion. It is not needed for the univention-upgrade functionality. I just copied the complete utils.py from the docker patch... Good catch! You should have been able to see this traceback by using univention-upgrade, have you? This can only be called by using a python interpreter and calling it directly.

> 
> * OK -> update found, but aborted
> * OK - app update
> * OK - install
> * OK - univention-add-app -a xrdp_20150720
> 
> 
> ps.
> Just a personal note. The "API" seems a bit confusing to me.
> Why this "from univention.appcenter.actions import get_action" and
> "get_action('upgrade')" stuff?
> Why is there no appcenter and app classes in python-univention-appcenter?
> Why not (app) class methods for these actions?
> 
> appcenter = univention.appcenter()
> for app in appcenter.installed_apps()
>   if app.update_available():
>     app.upgrade()
> app = appcenter.find(app='xyz')
> app.install()
> ...

This file was not necessary for the univention-upgrade functionality. The docker patch will allow:

from univention.appcenter import AppManager

for app in AppManager.get_all_locally_installed_apps():
  ...

The get_action('upgrade') is used to be able to overwrite the default upgrade action by a new package:
  python-univention-appcenter will bring a basic upgrade
  univention-appcenter-docker will bring its own upgrade with docker specific code that overwrites the basic upgrade and will be returned by get_action(). This is not needed right now but will be in UCS 4.1

Just for the app_is_running:
  univention-management-console-module-appcenter 4.1.20-45.390.201508050052
Comment 8 Dirk Wiesenthal univentionstaff 2015-08-05 01:20:46 CEST
(In reply to Dirk Wiesenthal from comment #7)
> (In reply to Felix Botner from comment #6)
> >   * What is the worst thing that can happen if username or password is wrong?
> 
> Only the join should fail. But this should not abort anything.
> If the app has DefaultPackagesMaster and the system is a DC Slave, though,
> the installation should fail.
> 

Clarification: The code should continue and fail if the credentials are valid, but the user in unprivileged. If the user is privileged (Addministrator), the password is asked three times. After that, upgrade is aborted.
Comment 9 Felix Botner univentionstaff 2015-08-05 13:09:03 CEST
(In reply to Dirk Wiesenthal from comment #7)
> (In reply to Felix Botner from comment #6)
> > I applied the univention-upgrade patch from Bug #30417
> > 
> > * FAIL - changes are not merged to 4.1, why?
> > 
> 
> Will be part of the docker patch. I opened Bug#39082 specifically for this
> one.

OK
 
> > * FAIL - YAML
> > 
> 
> Added

OK

> > * FAIL - app update - to much output (File: ...), the actual update works
> 
> Yes, I know. Not too happy about that. I have suppressed all other writings
> to stdout by redirecting a logger instance. But these "File:" statements
> come from UCR itself:
>   config_registry/handler.py
>     print 'File: %s' % self.to_file
> Not so much I can do except suppressing stdout itself which I do not want to
> do.
> 
> I think univention-upgrade does a similar thing by doing the actual work in a
>   apt-get distupgrade > /var/log/univention/updater.log
> 
> Are these lines a blocker so that I shall do it somehow like that?

OK, that's fine with me

> 
> > * FAIL - Username and Password parameters
> >   * There is no explanation for these parameters (i guess for the join)
> >     Does the target group for univention-upgrade know what's going on?
> 
> Yes, it is for the join (always). And sometimes for DefaultPackagesMaster,
> too. Good point.
> I think the credentials (and why they should be entered) should be asked by
> univention-upgrade.
> 
> >   * What is the worst thing that can happen if username or password...
> 
> Only the join should fail. But this should not abort anything.
> If the app has DefaultPackagesMaster and the system is a DC Slave, though,
> the installation should fail.
> 
> >   * Normally, univention-upgrade does not execute join scripts, why now
> >     for apps? Why not just rely on univention-run-join-scripts/UMC-Join
> > module
> 
> Because the App Center does it like this.

OK - if that is what we want (univention-upgrade runs app join-skripts) than we
 have to ask for the credentials, if they are not correct, the join fails but 
 not the update

> > * FAIL - noninteractive upgrade
> 
> Sorry the patch in Bug#30417 is broken. This is not App Center code that
> fails.

OK
  
> > * FAIL - utils.app_is_running
> 
> I removed this funtion. It is not needed for the univention-upgrade
> functionality. I just copied the complete utils.py from the docker patch...
> Good catch! You should have been able to see this traceback by using
> univention-upgrade, have you? This can only be called by using a python
> interpreter and calling it directly.

OK - (called app_is_running from within my python test script, no univention-
 upgrade involved )
 
> > * OK -> update found, but aborted
> > * OK - app update
> > * OK - install
> > * OK - univention-add-app -a xrdp_20150720
> > 
> > 
> > ps.
> > Just a personal note. The "API" seems a bit confusing to me.
> > Why this "from univention.appcenter.actions import get_action" and
> > "get_action('upgrade')" stuff?
> > Why is there no appcenter and app classes in python-univention-appcenter?
> > Why not (app) class methods for these actions?
> > 
> > appcenter = univention.appcenter()
> > for app in appcenter.installed_apps()
> >   if app.update_available():
> >     app.upgrade()
> > app = appcenter.find(app='xyz')
> > app.install()
> > ...
> 
> This file was not necessary for the univention-upgrade functionality. The
> docker patch will allow:
> 
> from univention.appcenter import AppManager
> 
> for app in AppManager.get_all_locally_installed_apps():
>   ...
> 
> The get_action('upgrade') is used to be able to overwrite the default
> upgrade action by a new package:
>   python-univention-appcenter will bring a basic upgrade
>   univention-appcenter-docker will bring its own upgrade with docker
> specific code that overwrites the basic upgrade and will be returned by
> get_action(). This is not needed right now but will be in UCS 4.1
> 
> Just for the app_is_running:
>   univention-management-console-module-appcenter 4.1.20-45.390.201508050052

OK
Comment 10 Janek Walkenhorst univentionstaff 2015-08-06 17:53:45 CEST
<http://errata.univention.de/ucs/4.0/267.html>