Settings

(1) Communication protocol: You can choose between the protocols "FTP", "SFTP" and "FTPS". See also (3).

(2) Via DMZ: Send data via DMZ server. If no DMZ is configured, this checkbox will not be displayed. The DMZ server is contacted, it executes the FTP commands, receives the files and stores them locally. Then these files are retrieved from the DMZ server via an internal FTP communication partner and processed. This option is only available if a channel is configured in (3). Attention: The internal communication partner DMZ Internal is created automatically, but must not be changed.

(3) Channel selection: Here you can select a channel of type "FTP" (for protocols "FTP" and "FTPS") or " SSH" (for protocol " SFTP" ) . Note: If possible, always use channels to centrally manage connection parameters. In addition to the increased clarity that also facilitates maintenance. If the connection parameters change, you only have to adjust the data in one place and you do not forget a profile.

(4) FTP server, Port: DNS name or IP address (and port) of the FTP server. If value "0" is specified for the port, then either the default port (./etc/ftp.xml) is used (if no channel is selected) or the port specified in the channel is used (if a channel is selected). If a value > 0 is specified, then this value is used as port number. Example: "ftp.example.com".

(5) GXS account: User and password for the target system. When connecting to a Value Added Network (VAN), it may be necessary to specify the "GXS" account in addition to the user. It must be ensured that the entered user also has access rights for the specified directory (7).

(6) Mode of transmission: In passive mode, the client establishes the control and the data connection. Normally (in the active case) the server establishes the data connection, but it may be the case that the necessary port is blocked by the local firewall. In the ASCII transfer mode, platform-dependent text formats (special characters) are converted.

(7) Directory: Directory on the FTP server side. You can also use the placeholders for dates and MSG_CALL_ variables (other variables are initialised in phase 3) in this field. Example: "./ftp/@MSG_CALL_VAR_DATA@".

(8) Use fast list mode: The FTP command "NLST" is used instead of the command "LIST" to create the file list. The file list creation is faster this way, but only the names of the files and no other information (e.g. if it is a directory) are provided. Since the response of the "LIST" command is not standardised, there may be problems parsing the received data. The use of the fast list mode will eliminate errors of this kind.

(9) File pattern: File name pattern. The files can be sorted for processing by filename in descending or ascending order or as supplied by the operating system. Example: ".txt|.asc".

(10) File handling: The read file can be handled in the following ways.

• Do not delete. Causes Lobster Integration not to delete the read file. An existing control file will also not be deleted.

• Delete. Causes Lobster Integration to delete the read file and any existing control file.

• Rename. Causes Lobster Integration to rename the read file. The new file name can be specified in the associated input field. Any existing control file is neither deleted nor renamed. The following placeholders can be used to refer to the name of the processed file.

  • <file> is the full name of the processed file.

  • <file-prefix> is the name of the processed file without the file extension.

(11) Use control file: Causes Lobster Integration to read a file only if there is an associated control file. See also (10). See also the example in (12). See also system variable VAR_SYS_CTRL_FILE.

(12) Master control file: Suffix or prefix of the control file. Example: If the suffix .ctrl is entered in the field, the control file name1.txt.ctrl will be searched for the file name1.txt. Please note that the dot in the suffix .ctrl must be explicitly specified.

(13) Process master control file: If this option is set, you only need a control file for the directory and not a separate control file for each file. The job will be started if this file exists in directory (7).

(14) Activate parallel processing: If this checkbox is set, several instances of this profile can work in parallel. The checkbox Profile may only run in one instance must not be set for this profile

(15) Delete empty files: Empty files are always deleted on the remote system, regardless of the setting in (10).

(16) Do not skip empty files: If this option is activated, jobs for this profile will even be created for empty files. Note: In a DMZ environment, the DMZ server can be started with the following startup option in the configuration file ./etc/startup.xml to control the behaviour.

• skip: (Default) Empty file remains on the DMZ server. Log entry in DmzEvents.

• delete: Empty file is deleted on the DMZ server.

• transfer: Empty file is transferred to the profile and option (8) will have an effect.

(17) Delete or rename AS400 main file: Specifies whether the AS400 main file should be deleted after the files have been fetched. Attention: If there are files within the main file that do not correspond to the file pattern (9), the main file will still be deleted.

(18) Additional FTP commands: Additional FTP SITE commands and standard FTP commands can be specified here (according to RFC 959). Which SITE commands can be specified here depends on which commands the respective FTP server supports (with SITE HELP the FTP server sometimes provides a list of its specific SITE commands). If you specify the additional FTP command somecommand, the FTP command SITE somecommand will be sent internally. If you want to send standard FTP commands, please use the prefix rfc:. So if you enter the additional FTP command rfc:CDUP, the FTP command CDUP will be sent internally. If there is a prefix before: before the command, it will be sent before the file transfer, otherwise afterwards. So to send the command CDUP before the file transfer, before:rfc:CDUP must be used. Note: When sending commands, the return code is checked by the server and all values 2xx except 202 are accepted. For other values, the profile generates an error. Note: Variables in the form @MSG_CALL_VARNAME@ can also be used to specify the additional commands. Note: Additional FTP commands cannot be used with SFTP (1). Note: The successful execution of the SITE commands themselves is not checked, i.e. Lobster Integration does not generate an error if a SITE command fails (except for an IO exception).

Further notes

• Some FTP servers also provide subdirectories without making them recognisable. To prevent jobs from running on non-existent files, the long file list can be requested from these FTP servers.

• If value Do not delete is set for option File handling, it is necessary to delete/rename this file and a potential control file on the FTP server side after successfully reading this file, as
  the files would otherwise be processed again during the next time the profile is run.

• If deleting or renaming of a file that is to be processed fails, for example, because a file of the same name already exists, no job is started. This will lead to an error if checkbox "Missing data are not ignored" is set, even if a file, that is to be processed, exists.

• If using this Input Agent, a maximum size of the input file can be checked. See section Maximum Size of Input Files.

• Files that can not be assigned to any profile are stored in the Unresolved area. See also section Warning for non-assignable input data.

• FTP site commands can be sent just like in the Response "FTP".

Dealing with timeouts due to extremely large files

When transferring extremely large files via DMZ, there may be a timeout in the communication between Lobster Integration and DMZ. The following is a solution that we nevertheless advise against, as it can lead to a loss of performance.

The default timeout is 30 minutes. This is how long Lobster Integration waits for a response from the DMZ server. If the file not yet been completely transferred between the DMZ and the partner by this time, Lobster Integration abandons the transmission and produces an error message. To wait longer, you have to start the Integration Server with an additional system property.

It is also possible that the communication between Lobster Integration and the DMZ server is not possible because the FTP service of the DMZ server does not use the default port 21 or because a firewall does not allow the connection in passive mode. The following system properties can be used to adjust those settings after a restart of the Integration Server.