Bug 50250 - Allow complex (exclusive !, or |) filters?
Allow complex (exclusive !, or |) filters?
Status: NEW
Product: UCS
Classification: Unclassified
Component: UDM - REST API
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: UMC maintainers
UMC maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-09-23 17:49 CEST by Florian Best
Modified: 2019-09-24 11:30 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2019-09-23 17:49:03 CEST
There are currently two ways to filter in a search in the UDM REST API:
1. With a raw ldap/udm filter: ?filter=(objectClass=*)
→ This allows full flexibility.

2. Via ?query[username]=foo&query[description]=bar.
→ This is basically for clients, which aren't aware of LDAP filter syntax (e.g. UMC).

With the latter variant, all property filters are conjuncted with "&".

Maybe we need to allow "exclusive" (!) and "or" (|) filters?
Comment 1 Daniel Tröder univentionstaff 2019-09-24 08:31:23 CEST
The following syntax could be
* easy to write for both humans and machines and
* reliable to parse for the UDM REST API,
because it directly maps the classes we have in univention.admin.filter:

?query[objectType]=users/user&OR(query[username]=foo,query[username]=bar*)&NOT(query[username]=barista)

Would be decomposed to:

* query[objectType]=users/user                  => expression()
* OR(query[username]=foo,query[username]=bar*)  => conjunction(|, [])
* NOT(query[username]=barista)                  => conjunction(!, [])

Would be translated to:

conjunction(&, [
    expression(objectType, users/user),
    conjunction(|, [
        expression(username, foo),
        expression(username, bar*)
    ],
    conjunction(!, [
        expression(username, barista)
    ]
])

Would be str() to:

(&
    (univentionObjectType=users/user)
    (|(uid=foo)(uid=bar*))
    (!(uid=barista))
)

Mapping OR to conjunction(|, []) and NOT to conjunction(!, []) will also require to handle '&' inside the NOT.
Unfortunately that means no greedy parsing anymore, but IMHO not a real problem.
I suggest to use ',' here instead of '&', because it will not change the meaning of HTTP parameter interpretation (to decompose parameters at '&' boundaries):

?NOT(OR(query[a]=b,query[c]=d),query[e]=f)

conjunction(!, [
    conjunction(&, [
        conjunction(|, [
            expression(a, b),
            expression(c, d)
        ],
        expression(e, f)
    ]
])

(!
    (&
        (|(a=b)(c=d))
        (e=f)
    )
)
Comment 2 Florian Best univentionstaff 2019-09-24 09:49:51 CEST
(In reply to Daniel Tröder from comment #1)
> I suggest to use ',' here instead of '&', because it will not change the
> meaning of HTTP parameter interpretation (to decompose parameters at '&'
> boundaries):
No, we need a simple format, which can be generated by most clients e.g. pythons urllib or regular browsers easily, i.e. application/x-www-form-urlencoded.
If it's too complex to build the client can just use the raw ldap filter.

The format must be describable by OpenAPI schema, i.e. the "deepObject" query string format.
The keys are simply properties, the values the search values.
I suggest to either
* append a ! or | to the property-name/key
* or create a nested structure, e.g.:

{"query": [{'!': {"username": "foo"}, "|": {"lastname": "foo"}, "&": {"firstname": "foo"}}]}
Comment 3 Daniel Tröder univentionstaff 2019-09-24 11:03:05 CEST
(In reply to Florian Best from comment #2)
> (In reply to Daniel Tröder from comment #1)
> > I suggest to use ',' here instead of '&', because it will not change the
> > meaning of HTTP parameter interpretation (to decompose parameters at '&'
> > boundaries):
> No, we need a simple format, which can be generated by most clients e.g.
> pythons urllib or regular browsers easily, i.e.
> application/x-www-form-urlencoded.
> If it's too complex to build the client can just use the raw ldap filter.
> 
> The format must be describable by OpenAPI schema, i.e. the "deepObject"
> query string format.
Ah ok - I'm not familiar with the requirements for the OpenAPI schema.

> The keys are simply properties, the values the search values.
> I suggest to either
> * append a ! or | to the property-name/key
> * or create a nested structure, e.g.:
> 
> {"query": [{'!': {"username": "foo"}, "|": {"lastname": "foo"}, "&":
> {"firstname": "foo"}}]}
I don't understand this example. To what LDAP filter would that translate?
Comment 4 Florian Best univentionstaff 2019-09-24 11:11:29 CEST
(In reply to Daniel Tröder from comment #3)
> > {"query": [{'!': {"username": "foo"}, "|": {"lastname": "foo"}, "&":
> > {"firstname": "foo"}}]}
> I don't understand this example. To what LDAP filter would that translate?

(&( (!(username=foo)) (|(lastname=foo)) (&(firstname=foo)) ))

Of course with one entry in the dict this doesn't makes sense. But you can add an arbitrary amount, which can again contain | & and ! keys.
Comment 5 Daniel Tröder univentionstaff 2019-09-24 11:30:05 CEST
So this:

(!
    (&
        (|(a=b)(c=d))
        (e=f)
    )
)

Would have to be written like this?:

{"query": [{'!': {"&": {"|": {"a": "b", "c: "d"}, "e": "f"}}}]}

That's certainly not easy to read or write...

Is the enclosing list really required? It seems kind of useless.