Sorting

images/download/attachments/167860729/image-2024-2-2_16-21-37-version-1-modificationdate-1706887298064-api-v2.png

All Search types can optionally specify a list of search criteria for a hierarchical Sorting of the results.

Configured search criteria are usually mapped directly to the ORDER BY clause of generated SELECT statements.

NOTESorting is always applied to the search results list before the 'First result' and 'Maximum results' parameters take effect. Sorting can therefore be used to control which subset of matches is considered a search result.

Configuration

The data object of the search offers a list field Order by (orders), which expects data objects of the 'Search order' (SearchOrder) type as list values:

  • By default (see screenshot above), the Order by list is empty.

  • A sorting criterion (SearchOrder instance) can be added to the list by clicking on the images/s/-95e2zf/9012/8yg2g7/_/images/icons/emoticons/add.svg icon.

  • An existing sorting criterion can be removed by clicking on the garbage can symbol.

  • If the context provides a tree structure for displaying the search, the sequence line item of the sorting criteria and therefore their rank within the hierarchy can be adjusted using drag & drop.

Each SearchOrder data object has exactly two properties (see also the placeholder structure in the screenshot above):

Labeling

Data field

Data type

Meaning

Ascending

asc

Boolean 
  • With the value true (default), the sorting criterion uses an ascending order.

  • With the value false, the sorting criterion uses a descending order.

Projection

projection

Projection
(entity)

Technically, all types of Projections can be inserted via the projection configuration.


IMPORTANT◄ For an Aggregatein the context of Sorting, restrictions may need to be observed:

  • Aggregation functions (see Aggregate) are only applicable if all output columns are either aggregated or grouped.

  • Calculation functions (see Aggregate) that only convert values and do not aggregate them can be used for Sorting without restriction.

Examples

Simple use case: Single-stage sorting

A CSV search for Users should be sorted in ascending order according to the 'username' (username) String property.

Configuration:

The screenshot on the right shows the configuration for Sorting with a search criterion:

  • The ASC (ascending) option has been retained as checked by default.

  • A Property projection for the 'username' (username) property is used as a Projection.

On the database side, this configuration generates the sort expression
ORDER BY username ASC.

images/download/attachments/167860729/image-2024-2-2_16-21-8-version-1-modificationdate-1706887268514-api-v2.png

Runtime example: ... with exactly one output column that corresponds to the Property projection from the Order by ...

username
========
admin
test
tEST
TEST
test0
test1
test2
TEST2

The example data on the left suggests that the sorting distinguishes between upper and lower case letters, but that the difference between upper and lower case is comparatively low:

"test" < "TEST" < "test0"

NOTE◄Essentially, an ORDER BY clause for a SELECT statement is generated from the sorting criteria. Lobster_pro therefore only defines the projection and sorting direction for each search criterion. The effective sorting logic therefore depends exclusively on the database used and any specific settings for the implementation (locale/territory, distinction between upper/lower case, etc.).

More complex use case: Multi-stage sorting

The previous example should be extended as follows:

  • In addition to the 'username' (username), an additional output column specifies the internal 'ID' (id), i.e. the unique Long value that is used as the primary key for entities in the database.

  • The sorting for the 'username' (username) property should be case-insensitive.

  • However, the internal 'ID' (id) should be taken into account as an additional (subordinate) sorting criterion.

Configuration:

The screenshot on the right shows the two Projections:

images/download/attachments/167860729/image-2024-2-2_16-18-18-version-1-modificationdate-1706887098906-api-v2.png

Order by is adjusted as shown on the right:

  • The first sorting criterion still refers to a Property projection for the username property. However, this is now converted within an Aggregate of the 'Lower' Type, so that the username is converted to lowercase before it is included in the sorting. This conversion makes the sorting 'blind' or indifferent to differences in upper/lower case.

  • The second sorting criterion is only effective if the first sorting criterion categorises several values as equivalent. The Property projection then refers to the id property, which by definition provides a unique Long value.

images/download/attachments/167860729/image-2024-2-2_16-20-29-version-1-modificationdate-1706887230078-api-v2.png

Runtime example:

username,id
===========
admin,
TEST,6851
test,7552
tEST,7553
test0,7183
test1,7184
test2,7302
TEST2,7551

The order of the users 'TEST', 'test' and 'tEST' is no longer determined by the alphanumeric sorting used in the previous runtime example, but by the Long value from the id property.

The sorting of 'test2' vs. 'TEST2' is now also based on the id values, even if it has not effectively changed as a result.

NOTE◄ Without the Aggregate for the first sorting criterion, the sequence would have been the same as in the previous runtime example. The id property would then effectively have no influence because the username strings are all case-sensitive. This is absolutely necessary, not least for use as an identifier when logging in.

More complex use case: Multi-stage sorting with specific treatment for NULL values

Configuration: (limited solution)

The screenshot on the right shows the configuration for a hierarchical Order by with two sort criteria:

  • The first sorting criterion uses a Typed attribute projection for the Attribute owner path address, which returns the flag value of the flag attribute 'Vip' (VIP). The ASC (ascending) option has been unchecked so that the VIP Users appear first.

  • The second sorting criterion accesses the 'username' (username) property directly via a Property projection. Sorting should take place in ascending (ASC) order.


images/s/-95e2zf/9012/8yg2g7/_/images/icons/emoticons/warning.svg CAUTIONimages/s/-95e2zf/9012/8yg2g7/_/images/icons/emoticons/warning.svg This configuration is only effective if the VIP attribute is explicitly created for all relevant addresses so that the 'Flag value' can be read as either true or false.

If Users exist whose address does not contain the VIP flag attribute, the Typed attribute projection returns the value $null. The sorting distinguishes between true, false and $null and therefore forms three instead of two groups with alphabetical sorting according to the username (see runtime example).

images/download/attachments/167860729/image-2024-2-2_16-17-5-version-1-modificationdate-1706887025955-api-v2.png

Runtime example: (Groups regarding VIP identification were subsequently artificially separated by a blank line)

VIP_flag,username
========================
false,tEST
false,TEST
false,test0
false,test1
false,test2
false,TEST2

true,bcollins
true,mpolo
true,TEST_VIP
false,fbeutel
false,ftuscania
false,test
  • For the first six Users in the list on the left, the VIP indicator attribute does not exist in the address, so the sorting takes 'No value' ($null) into account instead of the indicator value.

    NOTEBoolean output columns are subject to an automatic type conversion in a CSV search or a Tuple search, in which $null is replaced by false. As a result, false also appears in the VIP_flag column for the first group.

  • For the next three Users, the VIP flag attribute exists in the address and returns the flag value true.

  • The VIP flag attribute exists in the address for the last three Users in the list on the left, but returns the flag value false.

Variant:

In our use case, the conversion from $null to false should not only affect the output column, but also the sorting.

Configuration:

The Projection for the first sorting criterion explicitly assigns the value false as the return value via a case differentiation per Case projection and a Literal projection if the typed attribute projection returns 'No value' ($null) instead of the flag value (flagValue).

If the Typed attribute projection returns a flag value (flagValue), this is used as the return value for the sorting.

images/download/attachments/167860729/image-2024-2-2_16-16-17-version-1-modificationdate-1706886977718-api-v2.png

Runtime example: (Groups regarding VIP identification were subsequently artificially separated by a blank line).

VIP_flag,username
========================
true,bcollins
true,mpolo
true,TEST_VIP
false,fbeutel
false,ftuscania
false,test
false,tEST
false,TEST
false,test0
false,test1
false,test2
false,TEST2

The results list now only contains two groups:

  • The first three Users are explicitly marked as VIP.

  • All other Users are not marked as VIP. No distinction is made as to whether the flag attribute is missing or false.

More complex use case: Sorting a CSV search with grouping by aggregation result

A CSV search for Users should specify the 'username' (username) and the number of assigned 'roles' (roles) for each user account and sort the list in several levels as follows:

  • First, the number of roles per user should be sorted in descending order.

  • Users with the same number of roles should be sorted in ascending order by 'username'.

Configuration:

The output columns for the Tuple search are configured as shown on the right:

  • The first output column (count_roles) applies an Aggregate of the Type 'Count' to a Property projection for the 'roles' (roles) property of the user account.

  • The second output column accesses the 'username' property via Property projection.

NOTE◄ These Projections must subsequently also be used for 'Group by's' and 'Order by'. For the sake of transparency, we arbitrarily choose the same Name for the Aggregate as for the output column (count_roles), even if this is not functionally necessary.

images/download/attachments/167860729/image-2024-2-5_18-3-2-version-1-modificationdate-1707152582847-api-v2.png

In order for the 'Number' aggregation function to be applicable within the Aggregate, Group by's must be ordered that names the 'username' property as the only grouping criterion via Property projection, as shown on the right.

images/download/attachments/167860729/image-2024-2-5_18-3-23-version-1-modificationdate-1707152604046-api-v2.png

The desired Sorting uses two sorting criteria:

  • The first sorting criterion reproduces the projection for the first output column (count_roles). The Aggregate provide the number of roles per user according to which sorting is to be performed in descending order (unchecked in ASC ascending sorting).

  • The second sorting criterion reproduces the second output column and accesses the 'username' (username) property via Property projection in order to sort users with the same number of roles in ASC (ascending) order.

NOTE◄ The order of the output columns does not necessarily have to correspond to the hierarchy of the sorting criteria. From a technical point of view, it is not even necessary for the Projections used in the Sorting to be created as output columns at all. However, it does make it easier to interpret the results.

images/download/attachments/167860729/image-2024-2-5_18-3-51-version-1-modificationdate-1707152631170-api-v2.png

Runtime example:

count_roles,username
====================
5,admin
3,TEST_VIP
2,bcollins
2,fbeutel
2,mpolo
1,ftuscania
1,test
...

The example data (left) illustrates that users with the same number of roles are first sorted in descending order according to the result in the count_roles column, while the username column as a subordinate sorting criterion ensures that users with the same number of roles are sorted alphabetically in ascending order according to the user name.

More complex use case: Sorting complete entities by aggregation result

As a variation on the previous example, a Search for Users returns a list of complete user accounts, the order of which is defined by the same two-level sorting:

  • First, the number of roles per user should be sorted in descending order.

  • Users with the same number of roles should be sorted in ascending order by 'username'.

Configuration:

Since a Search (unlike the CSV search in the previous example) does not support 'Group by's', Sorting cannot use a 'number' Aggregate directly as a projection.

The following trick solves this problem, albeit at the expense of query performance:

How does the Sorting define two sorting criteria:

  • The first sorting criterion defines a Subsearch projection that does not refer to the 'Role' Entity, but to the 'User' Entity.

    • The Property to join and Property of entity to join with parameters each refer to the 'ID' (id) property of a user account, whereby it is required that these IDs match for the assignment of data to the user in the higher-level search. At first glance, this suggests little added value, as the same user account is 'assigned' to each user account once again.

    • However, this 1:1 assignment enables an Aggregate within the Projection for column of subselect: Using the 'Count' Type, we can initiate the counting of 'roles' (Property projection roles) per user ID, which is not possible directly in the parent Search due to the lack of 'Group by's'. The subselect is processed for each ID in the parent Search. The aggregation function 'Count' in the subselect can add up all the role values found without having to define a grouping. This number can be used as a sorting criterion (count_roles).

    • The option ASC (ascending) is unchecked for this criterion so that the user accounts appear with a descending number of roles.

  • As in the previous example, a Property projection for the 'username' (username) property serves as the second sorting criterion. Users with the same number of roles are sorted alphabetically in ascending order by username.

images/download/attachments/167860729/image-2024-2-5_18-0-29-version-1-modificationdate-1707152429713-api-v2.png