Bug 42175 - add user interface to add links to the UCS overview/portal site
add user interface to add links to the UCS overview/portal site
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Portal
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2
Assigned To: Dirk Wiesenthal
Stefan Gohmann
: interim-2
: 40626 (view as bug list)
Depends on:
Blocks: 42233 43670 43671 43680 44093
  Show dependency treegraph
 
Reported: 2016-08-29 20:13 CEST by Florian Best
Modified: 2017-04-04 18:30 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Icon for portal (7.90 KB, image/svg+xml)
2017-03-15 14:44 CET, Alexander Kläser
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-08-29 20:13:49 CEST
There should be a graphical user interface to add links to the new portal site.
Comment 1 Florian Best univentionstaff 2016-08-29 20:17:20 CEST
The entries should be stored in LDAP, e.g. by a new handler settings/... w.r.t. the hostname where the links/entries should be displayed.

There should be a migration script for the current ucs/web/overview/entries/ UCR variables.
Comment 2 Florian Best univentionstaff 2016-08-29 20:20:02 CEST
It should be configurable who can see which links.

My idea for this approach is to use the current UMC operation sets and UMC policies for this.
Are there other opinions? Does anything speak against this?

Alex opinion was that the name "UMC operation set" is irritating. Maybe we can rename this? Otherwise we need to create a new object class + schema for this.
Comment 3 Florian Best univentionstaff 2016-08-29 20:20:59 CEST
*** Bug 40626 has been marked as a duplicate of this bug. ***
Comment 4 Dirk Wiesenthal univentionstaff 2017-02-28 12:25:55 CET
New LDAP schema in
  univention-ldap (13.0.3-7)

New UDM handlers in
  univention-directory-manager-modules (12.0.14-5)
(and)
  univention-directory-manager-modules (12.0.14-6)

Listener scripts write the UDM objects into json:
  univention-portal (1.0.6-6)

settings/portal and settings/portal_entry are new UDM modules which are dumped into
  /usr/share/univention-portal/portal.json

Should implement important parts of https://mail.univention.de/appsuite/#!!&app=io.ox/office/text&folder=1206&id=1206/1698
Comment 5 Alexander Kläser univentionstaff 2017-03-02 10:06:25 CET
As discussed, after updating my system, the portal is empty. I can even not add a new portal entry since the list of UDM objects does not contain "Portal: Entry" when navigating with the LDAP module to cn=univention/cn=portal when clicking "Add".
Comment 6 Alexander Kläser univentionstaff 2017-03-02 10:14:58 CET
When adding "superordinate = 'settings/cn'" to portal.py and portal_entry.py, I can add new entries via the LDAP navigation.

When adding a new portal entry, the field "category" is a text box. IMHO, this should better be a combo box.
Comment 7 Alexander Kläser univentionstaff 2017-03-02 11:28:54 CET
Some other things that I noticed:

* I do not understand what "Language code (e.g. en_US) - Text" means. This 
  happens to exist 2 times ATM. I would add a title pane "Display name" and
  "Description" in which each widget will be placed. It is also not clear that
  en_US is a mandatory entry. Therefor it might even be better to have a text
  box "Display name" and than below separately a multi input for the 
  translations. In code, I would vote for the following layout:

layout = [
    Tab(_('General'), _('Entry options'), layout=[
        Group(_('General'), layout=[
            ["name", "category"],
            ["portal"],
            ["icon"],
        ]),
        Group(_('Display name'), layout=[
            ["displayName],
            ["displayNameL10n"],
        ]),
        Group(_('Description'), layout=[
            ["description"],
            ["descriptionL10n"],
        ]),
        Group(_('Link'), layout=[
            ["link"],
        ]),
        Group(_('Advanced'), layout=[
            ["activated"],
            ["authRestriction"],
            ["favorite"],
        ]),
    ]),
]

* The same idea should be applied to setting/portal:

layout = [
    Tab(_('General'), _('Portal options'), layout=[
        Group(_('Name'), layout=[
            ["name"],
        ]),
        Group(_('Display name'), layout=[
            ["displayName"],
            ["displayNameL10n"],
        ]),
        Group(_('Appearance'), layout=[
            ["background"],
        ]),
        Group(_('General Content'), layout=[
            ["showMenu"],
            ["showSearch"],
            ["showLogin"],
            ["showApps"],
            ["showServers"],
        ]),
    ]),
]

* "Upload certificate" → this might not be the correct syntax. This upload
  button is quite confusing. If it does not exist yet, we should add a syntax
  with a proper label for uploading a SVG icon.

* UMC should consist of two portal entries. One with the label "System and 
  domain settings" pointing to "/univention/management" and being visible on 
  master + backup. The other one with the label "System settings" which is 
  visible on all other roles. This means that the JavaScript code needs to be
  able to handle relative links. The UCR handler needs to ignore UMC entries.

* The UCR handler should ignore the entries for root certificate as well as 
  certificate revocation list (see Bug 43695).

* I added the function umc/tools::isIPAddress() which uses the same reg exps
  as you added to main.js. This check will be used more often in the future,
  I guess ;) .
Comment 8 Florian Best univentionstaff 2017-03-02 15:09:48 CET
I fixed the LDAP ACL's:
univention-ldap (13.0.4-1):
r77247 | Bug #42175: ACLs for Portal Entries
r77246 | Bug #42175: ACLs for Portal Entries
Comment 9 Florian Best univentionstaff 2017-03-02 18:45:06 CET
When opening the module there is a pop up:

Die folgenden leeren Eigenschaften wurdem im Formular auf Vorgabewerte gesetzt. Die Werte werden beim Speichern angewendet.
Allgemein - Erweitert - Authorisation: Visible for Admins only

Can you set a default value? Or add an empty value if possible.
Comment 10 Florian Best univentionstaff 2017-03-02 18:51:18 CET
After removing a portal the portal entries still have a reference in univentionPortalEntryPortal to the not anymore existing settings/portal object:

univention-ldapsearch univentionObjectType=settings/portal_entry univentionPortalEntryPortal
→ The references should be removed.
→ If it's the last reference for a link the link should also be removed.

I personally would like the superordinate approach more, so that there exists the following LDAP structure:

cn=portal,cn=univention,dc=base
* cn=global,cn=portal,cn=univention,dc=base
* * cn=umc,cn=global,cn=portal,cn=univention,dc=base
* * cn=app1,cn=global,cn=portal,cn=univention,dc=base
* cn=DC-Slave1,cn=portal,cn=univention,dc=base
* * cn=umc,cn=global,cn=portal,cn=univention,dc=base
* * cn=app2,cn=global,cn=portal,cn=univention,dc=base

This would fix also the reference problem when removing/renaming/moving the portal.
Comment 11 Dirk Wiesenthal univentionstaff 2017-03-03 12:34:13 CET
Fixed description and widgets. I added default values where appropriate.

UMC is blacklisted in the UCR module. A dedicated UMC link is generated in the joinscript - one for DC Master + Backup, one for the other roles.

All entries still lie in cn=portal. It would be convenient for moving and deleting, but I am not sure whether this is a real world issue. Maybe portals are never removed.

It would also require additional work in LDAP, UDM, Portal und App Center.

Last but not least, changing an entry (which I assume is more common than changing a portal) would require a change in each of the copied entries unless one adds additional logic.
Comment 12 Erik Damrose univentionstaff 2017-03-08 17:51:53 CET
Looks good on a first look. Some quick first impressions what should remains to be done:

* The portal.json may not be cached by the browser - i reinstalled a domain and got the cached portal.json from the old domain
* missing small icon udm-settings-portal.png in LDAP view
* uploaded backgrounds for portals are not written correctly for the json: background = '/portal/icons/backgrounds/%s.png'. missing /univention/. In a quick test with a modified portal.json the background image was not loaded by the browser (no request)
* Maybe more of a umc issue: An uploaded SVG is not displayed in the UMC, but correctly handled by the implementation.
Comment 13 Erik Damrose univentionstaff 2017-03-09 10:29:15 CET
* new bug?: The Portal name is not evaluated and displayed by the JS code, it is always 'Univention Portal'
* portal entry: path to image is not correct in json, same as for 'portal' image links
* The favorite attribute should probably not be available for entries, see bug #42234
* There seems to be no way of sorting the entries, as it was previously possible with UCRVs overview/../priority?
Comment 14 Alexander Kläser univentionstaff 2017-03-09 14:09:13 CET
As discussed:
* The show* attributes are currently not evaluated in JavaScript. For now it will
  be sufficient to evaluate showLogin
* The portal "local" should only show locally installed Apps
* The portal "local" needs an management entry which points to the portal 
  "domain"
* The portal "local" should use as title "Startsite for {hostname}"
* In the portal title, it should be possible to write {hostname} or {domainname} 
  which are expanded to the corresponding values.
Comment 15 Dirk Wiesenthal univentionstaff 2017-03-10 03:22:55 CET
(In reply to Erik Damrose from comment #12)
> Looks good on a first look. Some quick first impressions what should remains
> to be done:
> 
> * The portal.json may not be cached by the browser - i reinstalled a domain
> and got the cached portal.json from the old domain

Needs to be examined. Could not reproduce, but yes, there is no no-cache mechanism.

> * missing small icon udm-settings-portal.png in LDAP view

Done

> * uploaded backgrounds for portals are not written correctly for the json:
> background = '/portal/icons/backgrounds/%s.png'. missing /univention/. In a
> quick test with a modified portal.json the background image was not loaded
> by the browser (no request)

Fixed path. Image loading: Bug#43670 and Bug#42235

> * Maybe more of a umc issue: An uploaded SVG is not displayed in the UMC,
> but correctly handled by the implementation.

Yes, I will open a Bug.

(In reply to Erik Damrose from comment #13)
> * new bug?: The Portal name is not evaluated and displayed by the JS code,
> it is always 'Univention Portal'

Done.

> * portal entry: path to image is not correct in json, same as for 'portal'
> image links

Done.

> * The favorite attribute should probably not be available for entries, see
> bug #42234

Yes, removed.

> * There seems to be no way of sorting the entries, as it was previously
> possible with UCRVs overview/../priority?

Not for now. And maybe never.

(In reply to Alexander Kläser from comment #14)
> As discussed:
> * The show* attributes are currently not evaluated in JavaScript. For now it
> will
>   be sufficient to evaluate showLogin

Can you give me a hint how to achieve this? Could not figure out how to access the menu.

> * The portal "local" should only show locally installed Apps
> * The portal "local" needs an management entry which points to the portal 
>   "domain"
> * The portal "local" should use as title "Startsite for {hostname}"
> * In the portal title, it should be possible to write {hostname} or
> {domainname} 
>   which are expanded to the corresponding values.

Done all of that.


Still REOPENED for the open issues.
Comment 16 Richard Ulmer univentionstaff 2017-03-10 11:36:31 CET
Updating from 4.1-4 to 4.2-0, the portal join script failed to executed. I observed the following error in the join.log:

> RUNNING 35univention-portal.inst
> 2017-03-10 11:31:04.593529457+01:00 (in joinscript_init)
> Object exists: cn=portal,cn=univention,dc=univention,dc=intranet
> Object exists: cn=portal,cn=custom attributes,cn=univention,dc=univention,dc=intranet
> Object exists: cn=domain,cn=portal,cn=univention,dc=univention,dc=intranet
> /usr/lib/univention-install/35univention-portal.inst: 138: /usr/lib/univention-install/35univention-portal.inst: Syntax error: Unterminated quoted string

Running the script via bash -x , everything was fine, though.
Comment 17 Richard Ulmer univentionstaff 2017-03-10 11:39:12 CET
... OK, running dash -x fails... the following line is not working with dash:

echo $'"fr_FR" "page d\'accueil pour {hostname}"'
Comment 18 Alexander Kläser univentionstaff 2017-03-10 12:28:46 CET
(In reply to Dirk Wiesenthal from comment #15)
> (In reply to Erik Damrose from comment #12)
> > Looks good on a first look. Some quick first impressions what should remains
> > to be done:
> > 
> > * The portal.json may not be cached by the browser - i reinstalled a domain
> > and got the cached portal.json from the old domain
> 
> Needs to be examined. Could not reproduce, but yes, there is no no-cache
> mechanism.

We had similar issues in 4.1/4.0 with the ucs-overview site which needed to read the JSON file with all entries to be rendered. Have a look at the apache config in:

ucs-4.1-4/services/univention-apache/conffiles/etc/apache2/conf.d/ucs.conf

Essentially you need to adjust the caching information which is sent to the browser:

  <FilesMatch "(entries|languages).json">
      Header set Cache-Control "max-age=0, must-revalidate, no-cache, no-store"
  </FilesMatch>
Comment 19 Erik Damrose univentionstaff 2017-03-10 14:09:29 CET
I fixed the joinscript quoting issue (comment 16+17), because a rejoin failed.

r77575 univention-portal 1.0.8-3A~4.2.0.201703101407
Comment 20 Dirk Wiesenthal univentionstaff 2017-03-13 17:57:14 CET
Caching has been implemented in
  univention-portal (1.0.8-4)
(via /etc/apache2/ucs-sites.conf.d/portal.conf)

As discussed, all show* attributes in settings/portal are now dormant. I have removed them from the list of UDM attributes (at least from layout).
  univention-directory-manager-modules 12.0.14-11A~4.2.0.201703131751
Comment 21 Alexander Kläser univentionstaff 2017-03-14 20:50:00 CET
I spotted the following line in main.js l. 114:
> if (! onlyOneKind) {
Should that be not rather:
> (!onlyOneKind.length) {
... as ![] === false in JS

(In reply to Dirk Wiesenthal from comment #20)
> [...]
> As discussed, all show* attributes in settings/portal are now dormant. I
> have removed them from the list of UDM attributes (at least from layout).
>   univention-directory-manager-modules 12.0.14-11A~4.2.0.201703131751

AFAIS, the domain portal should not show locally installed apps by default, but only Apps which are installed in the whole domain.
Comment 22 Dirk Wiesenthal univentionstaff 2017-03-14 22:46:18 CET
(In reply to Alexander Kläser from comment #21)
> I spotted the following line in main.js l. 114:
> > if (! onlyOneKind) {
> Should that be not rather:
> > (!onlyOneKind.length) {
> ... as ![] === false in JS
> 

var onlyOneKind = array.every() -> boolean

> (In reply to Dirk Wiesenthal from comment #20)
> > [...]
> > As discussed, all show* attributes in settings/portal are now dormant. I
> > have removed them from the list of UDM attributes (at least from layout).
> >   univention-directory-manager-modules 12.0.14-11A~4.2.0.201703131751
> 
> AFAIS, the domain portal should not show locally installed apps by default,
> but only Apps which are installed in the whole domain.

The domain portal does in fact not show installed Apps.


35univention-portal.inst:
[...]
univention-directory-manager settings/portal create "$@" --ignore_exists \
	[...]
	--set showApps=FALSE \

This is implemented. The comment on this bug was just wrong.
Comment 23 Alexander Kläser univentionstaff 2017-03-15 14:44:56 CET
Created attachment 8543 [details]
Icon for portal
Comment 24 Erik Damrose univentionstaff 2017-03-17 16:32:00 CET
I created bug #43928 for updating entries upon moving / deleting

OK: missing small icon udm-settings-portal.png in LDAP view

>> * Maybe more of a umc issue: An uploaded SVG is not displayed in the UMC,
>> but correctly handled by the implementation.
> Yes, I will open a Bug.

Did you? I could not find one, and the issue persists. I created bug #43930

OK: The Portal name is not evaluated and displayed by the JS code, it is always 'Univention Portal'
OK: portal entry: path to image is not correct in json, same as for 'portal' image links
OK: The favorite attribute should probably not be available for entries, see bug #42234 
OK~ There seems to be no way of sorting the entries, as it was previously possible with UCRVs overview/../priority?
I created bug #43933 to separate the feature discussion

OK: The portal "local" should only show locally installed Apps
OK: The portal "local" should use as title "Startsite for {hostname}"
OK: In the portal title, it should be possible to write {hostname} or {domainname} which are expanded to the corresponding values.
OK: JSON caching has been implemented 
OK: The domain portal does in fact not show installed Apps.
OK: Icon for portal is shown

Reopen: The portal "local" needs an management entry which points to the portal "domain"
Currently, this is configured by configuring old ucs-overview variables, why not create a udm object directly? The way it is now, re-executing the joinscript if the ucs-local-to-domain object is missing fails, because on execution the object will not be recreated: all ucr set calls are conditional (ucr set k?value).

Reopen: backgrounds are accepted and written correctly to json and filesystem, but are not loaded.
I just saw that there is another bug for portal backgrounds, please coordinate with assignee of bug #42235

Reopen: The show* attributes are currently not evaluated in JavaScript. For now it will be sufficient to evaluate showLogin
This is currently not clear to me: What should happen if 'showLogin' is set?
* Currently it can not be set for a portal
* I can configure 'Visible for authenticated users' for portal entries, but such entries are always visible (e.g. server overview)

Reopen: Icons for 'installed services' are shown, but not for installed apps that are shown in the 'Administration' category, e.g. Nagios
Comment 25 Daniel Tröder univentionstaff 2017-03-22 14:45:23 CET
The join script 35univention-portal.inst overwrites an existing setting of "computers/$role"→portal, effectively removing a custom portal, if the join script was ever rerun.
Comment 26 Daniel Tröder univentionstaff 2017-03-22 15:29:52 CET
if [ "$server_role" = "domaincontroller_master" -o "$role" = "domaincontroller_backup" ]; then
	UCS_PORTAL="cn=domain,cn=portal,cn=univention,$ldap_base"
else
	UCS_PORTAL="cn=local,cn=portal,cn=univention,$ldap_base"
fi

CURRENT_PORTAL=$(univention-ldapsearch -LLL -b "cn=computers,$ldap_base" "cn=$hostname" univentionComputerPortal | ldapsearch-wrapper | egrep -v ^dn: | cut -f 2 -d ' ')
if [ "$CURRENT_PORTAL" = "$UCS_PORTAL" ]; then
	# computer portal unchanged from original
	udm "computers/$server_role" modify "$@" --dn="$ldap_hostdn" \
		--set portal="$UCS_PORTAL" || die
fi
Comment 27 Florian Best univentionstaff 2017-03-22 15:34:55 CET
(In reply to Daniel Tröder from comment #26)
> CURRENT_PORTAL=$(univention-ldapsearch -LLL -b "cn=computers,$ldap_base"
This won't work with UCS@school because the computers aren't in cn=computers,$ldap_base.
Comment 28 Daniel Tröder univentionstaff 2017-03-23 08:04:21 CET
It's just incomplete and untested example code - not ready for production.
It must also test for unset CURRENT_PORTAL [ -z ] etc.
Comment 29 Florian Best univentionstaff 2017-03-23 12:35:14 CET
As discussed, instead of the search filter -b "$ldap_hostdn" can be used directly.
Comment 30 Dirk Wiesenthal univentionstaff 2017-03-27 17:05:39 CEST
As discussed, icons, background have had separate bugs.

showLogin will not be supported. It has been removed from the UDM form. (Not from LDAP, though)

I have fixed the joinscript to not overwrite existing portal DNs in
  univention-portal 1.0.10-10A~4.2.0.201703271700

Using ucr set k?v is the same as using udm ... create ignore_exists. It is better to not overwrite the values of an object. A script to "sync" UCRVs with settings/portal_entry objects would be nice, though. I will open a bug for that.
Comment 31 Stefan Gohmann univentionstaff 2017-03-27 21:25:37 CEST
(In reply to Dirk Wiesenthal from comment #30)
> As discussed, icons, background have had separate bugs.

Yes.

> showLogin will not be supported. It has been removed from the UDM form. (Not
> from LDAP, though)

Yes.
 
> I have fixed the joinscript to not overwrite existing portal DNs in
>   univention-portal 1.0.10-10A~4.2.0.201703271700

Two issues:

1)
That doesn't work. I've added set -x:
--------------------------------------------------------------------------------
++ sed -n -e 's/^  portal: \(.*\)$/\1/p'
++ univention-directory-manager computers/domaincontroller_master list --filter cn=master421
+ set_portal=None
+ '[' -z None ']'
--------------------------------------------------------------------------------

So, if the entry is not set, UDM shows None.

2)
If I open a backup, slave or member after updating to UCS 4.2, a portal is automatically added by UMC. 
---------------------------------------------------------------------------
The following empty properties were set to default values in the form. These values will be applied when saving.

    [Advanced settings] - Portal - Portal: domain
---------------------------------------------------------------------------
So, after updating the DC master to UCS 4.2, I get this message for every time a open one UCS server object which haven't been updated. I think, you have to add an empty value. If you are using settings/syntax this might help:
root@master421:~# udm settings/syntax | grep -i empty
                addEmptyValue                            Add an empty value to choice list
root@master421:~# 

 
> Using ucr set k?v is the same as using udm ... create ignore_exists. It is
> better to not overwrite the values of an object. A script to "sync" UCRVs
> with settings/portal_entry objects would be nice, though. I will open a bug
> for that.

Yes
Comment 32 Dirk Wiesenthal univentionstaff 2017-03-28 10:48:58 CEST
Fixed in
  univention-portal 1.0.10-13A~4.2.0.201703281009
and
  univention-directory-manager-modules 12.0.16-3A~4.2.0.201703281005

Already published.
Comment 33 Stefan Gohmann univentionstaff 2017-03-28 17:04:37 CEST
OK, works now.
Comment 34 Stefan Gohmann univentionstaff 2017-04-04 18:30:02 CEST
UCS 4.2 has been released:
 https://docs.software-univention.de/release-notes-4.2-0-en.html
 https://docs.software-univention.de/release-notes-4.2-0-de.html

If this error occurs again, please use "Clone This Bug".