Main Settings (ETL/ELT)

(1) Files to be read in child tasks that are smaller than the specified size are kept in memory (and not on disk). Note: If you have enough memory, this will be faster. If your memory is not large enough, you risk a memory overflow.

(2) Defines which information is logged.

(3) The processing of the ETL/ELT entry can take place locally or on a remote server (to conserve resources).

Important note:

Files to be processed by the remote server must also be located on the remote server and the paths in the ETL/ELT entry must correspond to those of the remote server. For example, if the file is located remotely at /opt/input/file.csv and locally at /opt/Lobster/Lobster_data/webapps/root/upload/file.csv, then the path /opt/input/file.csv must be specified as the input source in the ETL/ELT entry.

Likewise, target files are saved on the remote server.

I. e. one must either mount the folders of the remote server on the _data server, or vice versa, so that the files can be processed directly. Alternatively, the distribution of the files can also be organised with profiles (e.g. in workflows).

If you use databases in the ETL/ELT entry, you must also have access to them on the remote server. I.e. you have to adapt the configuration file ./etc/database.xml of the used Lobster Bee.

(4) If set, multiple instances of this ETL/ELT entry cannot run at the same time. So if there is already a job running for this entry, you cannot start another one.

(5) When selecting this checkbox, read-in NULL values will not be replaced by an empty string in function chains.

(6) ETL/ELT fields can be kept in a clipboard. If this checkbox is set, the clipboard is kept in a file and not in memory. Note: If you have enough memory, keeping the clipboard in memory will be faster. If your memory is not large enough, you risk a memory overflow.

(7) In this field any URL that refers to a documentation of this ETL/ELT entry can be specified. It be called up via the arrow. Additionally, system constants can be inserted into the URL.

(8) Here you can display the search index of the ETL/ELT entry. These values can be queried in the expert search.