Definition

Basically, every field in Lobster_data (in the source and destination structure) has a value (i.e. at least an empty string "").

In addition, each value has a so-called Empty Flag. The Empty Flag in Lobster_data is a property that tells whether a value is to be considered as empty (in the sense of non-existent) regardless of whether the value represents an empty string (so there are empty strings with and without Empty Flag and also concrete values with Empty Flag, as we will see later).

Note: Strictly speaking, a value has an Empty Flag and a field has a value. For the sake of simplicity, however, we speak of a field with an Empty Flag. If the Empty Flag of the value of a field is true, then we say the field has an Empty Flag and if the Empty Flag of the value of a field is false, we say the field has no Empty Flag.

Origin

Source Structure

If for example, the input format XML is used and the input data is missing a tag (which corresponds to a source field), the source field receives an empty string and the Empty Flag. The same applies to a CSV source structure with x fields if the input file only contains x-1 fields. The last field of the source structure is then given an empty string and the Empty Flag.
In this context, option Treat empty value as 'not existing' (Phase 2, Extended settings) plays a role. If the option is set and the input data for a source structure field (data type CSV) contains only an empty string, then an empty string and the Empty Flag is set for the source field. If the option is not set, only the empty string is set, but not the Empty Flag.
A further special case can be found in the section 'Hidden Values'.

Destination Structure

No source structure field is assigned to a destination structure field (i.e. it is not mapped). The destination structure field also contains no functions and no fixed value. In this case, the destination structure field receives an empty string and the Empty Flag.
A source structure field is assigned to a destination structure field (i.e. it is mapped). The destination structure field also contains no functions and no fixed value. In this case, the destination structure field receives the value and the Empty Flag of the source structure field (if it has one).
Explicit setting of the Empty Flag for a destination structure field with function set empty(). The function sets an empty string and the Empty Flag.
There are a number of other functions that can set an Empty Flag. Please refer to the respective function documentation.

Check for Empty Flag

You can use the function is empty(a) to check whether the Empty Flag is set for a (source or destination structure) field.

Hidden Values

'Hidden' here means that fields with these values are displayed with no value in the mapping test, but internally there is a value that is only not visible because of the set Empty Flag. Hidden values can occur in the following ways.

A field of the source structure has the type Date and the parser could not successfully parse the value of the input data for this field into a date using the field's template. In this case, the source field receives the current date and the Empty Flag. In the mapping test, you will get no value displayed (the log of the test, however, will show the assignment of the current date). As mentioned above, a mapped destination structure field will also get this hidden value and the Empty Flag.
If a field of the destination structure uses one of the functions create date strict(value a, template b, default c), create date strict(value a, adjustable template b, default c, [locale d]) or date-from-excel(text a, default b), the current date and the Empty Flag are also set if no valid date can be generated with the value and template specified in the function.
As result value of some functions if they get a hidden value as an input value. Please refer to the respective function documentation.
Explicitly using the set empty() function.

Effect of Empty Flag and Hidden Values

In a generated output file, usually, no value is output for a destination field with an Empty Flag. However, there are also exceptions where the output of these values can be forced (see option With empty fields in the XMLNoTemplateUnit for example). This can be relevant for hidden date values. Please refer to the documentation for the respective output type/components.
In a mapping test, a field with a set Empty Flag is displayed with no value (in the source structure and the destination structure).
The function is empty(a) returns true (for source and target structure fields). This would not happen if the field had the value "" (i.e. an empty string, meaning a string of length 0) and no Empty Flag.
There are also functions that allow hidden values as input parameters. As an example, the functions concat( a, b ) and concat(a,b,c,d,e,f,g,h) shall be mentioned here. Please also refer to the respective function documentation.
In SQL statements in phase 4, values with the Empty Flag set can be skipped.