File folder (source ETL/ELT)
(1) Files are searched for in this directory (not recursively). You can also specify a URL. Example: https://example.com/path
(2) Selection of ETL/ELT variables.
(3) Here you can filter the files based on their name . Example: *.txt returns all files with the file extension .txt . With the prefix regex: you can use regular expressions for filtering. Leave the field empty or use the expression *.* to return all files.
(4) Sorting of the found files by file name or modification date. Ascending or descending.
(5) Test query that returns a list of files.
(6) These ETL/ELT fields can be used to read out the individual file names in the destination data or in subtasks. More about this in the following section.
Procedure
A list of files is created from the specified file folder (depending on the settings). Each file name corresponds to a data row.
You can either process these data rows in the destination data as usual. Use the ETL/ELT fields defined in (6) for access.
An alternative possibility is to process the files in a subtask. In the first task all files are read out, in the subtask you then process one file after the other. Use one of the ETL/ELT fields from (6) in the subtask to specify the file name.
Practical Example
A practical example of a bulk insert from a file into a database can be found here.