Lobster Integration itself runs as an application on the Integration Server. It also runs all services used for communication between Lobster Integration and the outside world and the service that connects Lobster Integration to its database.

The Integration Server is managed via the Admin Console .

This documentation uses "./" (a dot and a slash) to specify the installation directory of the Integration Server. Please note this when configuration files are described.

images/download/attachments/21304153/Diagramm_data_IS-version-13-modificationdate-1740042516570-api-v2.PNG

Directory structure of the Integration Server

Below the installation directory of the Integration Server there are a number of subdirectories.

./as2

Contains MDN files of sent and received AS2 messages.

./asm

Contains files for the Asynchronous Sending Module (ASM).

./backup

In this directory, the files handled in a version update or patch are backed up in advance in a .zip file.

./bin

This directory contains all the start programs for running the Integration Server and related framework applications, some of which have already been described in section About the Admin Console. There are two types of scripts.

Scripts for the operation under Windows. These can be recognised by the file extension .bat.
Scripts for the operation under Unix-like systems. These can be recognised by the file extension .sh.

To be able to execute these scripts, the standard shell "/bin/sh" has to be available. The subdirectory shared with all the contained files and the files "starter.jar", "wrapper.jar" and "wrapper.exe" are required for operation as a Windows service and should not be changed or removed without an explicit request from Lobster.

./conf

Configuration files that are not part of the standard scope of the Integration Server should be stored in this directory, so that no problems arise when there are name conflicts between project-specific files and standard configuration files.

Immediately after the installation of the Integration Server, there are already a number of files in the directory that can be adapted to your needs or used as an example for new configuration files.

./datawizard

This directory contains template files and other files used by Lobster Integration at runtime.

/backup	Directory for holding the input files of individual profiles. Each profile will have its own subdirectory created.
/moved	Contains outdated backup files if parameter "moveBackUpFiles" in configuration file ./etc/startup.xml is set to "true".
/settings	Contains created backups of profiles and exported source and target structures.
/swap	If a profile is configured such that data can be swapped out during mapping, the swapped data is stored in this directory.
/templates	Template files provided by Lobster, which are loaded via "Load template" in the mapping area of Lobster Integration for the source and target structure.

./dmz

Contains event files if a Lobster DMZ server is also installed.

./etc

Based on the etc directory of Unix systems, this directory contains everything necessary for configuration. Directly in the directory, these are mainly the XML-based configuration files of the individual services of the Integration Server, or additional files, such as the user definitions of the communication services. Usually there are four subdirectories of "etc".

admin	Contains configuration files for setting Lobster Integration in the subdirectory "datawizard". Subdirectory "webapps" : For new installations (not updates) from version 4.6.9 onwards, you can now also find here what was previously in the ./webapps directory.
dtd	Contains DTDs (XML descriptions), which are necessary for the operation of the Integration Server. See also section Format of XML configuration files.
realms	Contains configuration files for the HTTP server and for setting the password of the Admin Console.
scripts	This directory only exists for backward compatibility and can therefore be ignored.

./ext

This directory is intended for storing your own classes to be accessible for the Integration Server. This can be used, for example, when executing classes using the Admin Console tools.

The advantage of the ext directory is that new classes can be found even if the server has not been restarted. However, the replacement of once loaded classes still requires a reboot of the server so that the change finds its way into the virtual machine.

Attention: ".jar" files are ignored in this directory. They should be put in directory "./extlib".

./extlib

This directory is intended for storing your own libraries that should be accessible to the Integration Server.

Attention: The "extlib" directory has a higher priority than the default "lib" directory when determining classes. If you include libraries that are also used by the Integration Server itself, using other versions may cause problems that cannot be traced back to this fact immediately. Before importing your own libraries, you should therefore always check first whether a similar library that already exists can be used. Please note that you are responsible for maintaining this directory yourself!

Blacklists and whitelists for file extensions

See section Blacklist and whitelists for file extensions.

./index

If the FileSearchService is enabled, this directory is the base directory for storing index files. When the service is disabled or it has just been installed, the directory is usually empty.

./lib

This directory contains all the libraries required for the basic operation of the Integration Server. This directory should be left untouched by the user, i.e. own files should not be stored here.

./logs

One of the most important directories during ongoing operations. Although it is empty right after the installation, over time individual components of the server will generate subdirectories in which a number of log files are created.

For more information, please refer to the description of the LogService.

In addition to log data, other data from other services is also stored in this directory. Following a brief overview, a detailed description of the functionality can be found in the documentation of the respective service.

Subfolder	Service	Description
events	MessageService	When persistent messages are used, the "events" directory stores messages that cannot be forwarded immediately.
storage	StorageService	Data stored persistently via the StorageService is stored in this directory.

./patch

If Lobster delivers a hotfix, it is usually unpacked into this directory. The Integration Server empties this directory during a later update, which avoids version incompatibilities. Details are described on the corresponding download page for patches in the customer area.

Attention: If your own files are stored in this directory, they will also be deleted during an update, regardless of whether they are included in the update or not. This directory should therefore under no circumstances be used for a purpose other than described.

./tmpIO

Swap location for Java, for example for various client jetty files.

./training

This directory contains a demo profile with test data.

./transfer

Default root home of FTP, OFTP and SSH users.

./webapps

This directory is for the inclusion of web applications. Other web applications can be added here, such Axis.

Immediately after installation you will find the following subdirectories.

root	This is the base directory of the HTTP server.

Important note: For new installations (not updates) from version 4.6.9, see also ./etc/admin/webapps.

Directories to be backed up

For saving current working data of the Integration Server as part of a backup strategy, we recommend backing up the following directories daily.

./conf	To save your own configuration files.
./etc	To back up the default configuration files.
./datawizard	To save the data of Lobster Integration. To save space, you can omit the directory "templates".
./index	This directory can be omitted if the FileSearchService is not used.
./logs	Should not all log directories be backed up because the volume of data becomes too large (larger Lobster Integration installations generate up to 1 GB of log data per day), the backup can also be restricted to the directories ./logs/events and ./logs/storage.

System properties

Permanently in files

The general syntax for system properties when starting the Integration Server, and thus Lobster Integration, is the following.

-D<property name>=<property value>

System properties are set in the following configuration files.

./bin/hub.bat (Integration Server is started with a Windows batch file).
./etc/wrapper.conf (Integration Server runs as Windows service) (usually the case).
./bin/execute.sh (under Linux/Unix).

Temporarily in the Admin Console

Alternatively, system properties can also be set in the Admin Console.

Again, enter the name and value of the property. The "-D" at the beginning is omitted here!

Changes cab be made during runtime. Attention: The changes are lost after a restart!

Checks after configuration changes

After changes to the configuration files ./etc/*.xml, you have to check that no errors have occurred. The tests should preferably be done about 5 minutes after the Integration Server is fully started/restarted.

How are the tests carried out?

This depends on the operating system and the startup type of the Integration Server.

On Windows when starting in the foreground (not as service).	Observe the messages that appear in the command window input prompt.
On Windows when starting as a service.	Check file ./logs/wrapper.log.
On Unix/Linux when starting in the foreground (run).	Observe the messages in the console.
On Unix/Linux when starting as a service.	Check file ./hub.txt.

Note: Do not leave these files open in a blocking editor for a longer time. Use a non-blocking editor or copy the file and open the copy.

If there are error messages (exceptions) during or shortly after the normal start message, there is an urgent need for action. In this case, the Integration Server and therefore Lobster Integration may not be available with the entire functionality. Often it is not possible to log on to the client in the event of a faulty start. Some possible causes in the following.

Invalid license.
Database hub not ready or unreachable.
Error in a configuration file after modification.
Insufficient memory.
Other serious errors, for example, a class in the wrong version.
Error output from classes, which the user has (incorrectly) developed himself.

Errors output here always indicate a serious problem and should be analysed immediately. Since these messages are terminal outputs and do not occur in a log, they do not have a timestamp. During the normal, error-free operation, there should be no further messages after the following start message.

Lobster Integration Server (IS) started in 32453 ms , system is ready ...
--------------------------------------------------------------------------

If there is not enough memory available during operation, OutOfMemoryExceptions may appear. Most of the time, this error only leads to the termination of the profile job that has used too much memory. In rare cases, it is possible that other functions are impaired. Therefore, after an OutOfMemoryException, the entire Integration Server should be restarted. If such errors occur frequently, the memory should be increased.

Note: In order to detect OutOfMemoryExceptions, you should not only check after a restart but also afterwards, for example, once a day and obviously also if a profile job ends with an OutOfMemoryException.

Proxy server

The Lobster Integration Server (and thus also Lobster Integration) can establish outgoing network connections via proxy servers. The criterion is the direction of the TCP connection establishment, not the data direction. The Java Virtual Machine supports three proxy types.

HTTP proxy. Can mediate HTTP connections.
FTP proxy. Can mediate FTP connections.
SOCKS proxy. For mediating all TCP connections.

Once a SOCKS proxy is configured, all TCP connections go through it, including HTTP(S) and FTP connections.

If additional authentication is required at the proxy, the Integration Server (and thus Lobster Integration) has an authenticator that logs on to the proxy. The authenticator accepts passwords in plaintext or obfuscated.

Otherwise, the proxy must allow unauthenticated connections.

Since we were unable to verify all the features of the standard implementation as described by Oracle, a proxy selector was developed in addition to the authenticator to configure proxy exceptions for SOCKS. This special Lobster implementation can be deactivated via the following system property (see chapter above).

-Dhub.disableProxyHandling=true

Then the behaviour falls back to the standard implementation described in the following (and elsewhere).

http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
http://docs.oracle.com/javase/6/docs/technotes/guides/net/properties.html

The use of proxies is controlled at the startup of the Integration Server via system properties. Following is a list of properties for the three proxy types.

HTTP proxy

System property	Description
-Dhttp.proxyHost	IP address or DNS name of the HTTP proxy. Default: <none>.
-Dhttp.proxyPort	Port number of the HTTP proxy. Default: 80 if -Dhttp.proxyHost is set.
-Dhttp.nonProxyHosts	List of hostnames/addresses that should not be connected via HTTP proxy, but directly. Multiple entries are separated by "\|". Each entry can contain a wildcard "*". Default: <none>.
-Djava.net.http.username	User for the authentication at the HTTP proxy. Default: <none>.
-Djava.net.http.password	Password for the authentication at the HTTP proxy in plaintext or obfuscated. Default: <none>.

FTP proxy

System Property	Description
-Dftp.proxyHost	IP address or DNS name of the FTP proxy. Default: <none>.
-Dftp.proxyPort	Port number of the FTP proxy. Default: 21 if -Dftp.proxyHost is set.
-Dftp.nonProxyHosts	List of hostnames/addresses that should not be connected via FTP proxy, but directly. Multiple entries are separated by \|. Each entry can contain a wildcard *. Default: <none>.
-Djava.net.ftp.username	User for the authentication at the FTP proxy. Default: <none>.
-Djava.net.ftp.password	Password for the authentication at the FTP proxy in plaintext or obfuscated. Default: <none>.

SOCKS proxy

System property	Description
-DsocksProxyHost	IP address or DNS name of the SOCKS proxy. Default: <none>.
-DsocksProxyPort	Port number of the SOCKS proxy. Default: 1080 if -DsocksProxyHost is set.
-Djava.net.socks.username	User for the authentication at the SOCKS proxy. Default: <none>.
-Djava.net.socks.password	Password for the authentication at the SOCKS proxy in plaintext or obfuscated. Default: <none>.

Unfortunately, the standard implementation in the Java Virtual Machine does not allow exceptions for SOCKS proxies. We have, therefore, through the Lobster implementation of Authenticator and ProxySelector, created a way to define such exceptions for all three types of proxies in the configuration file ./etc/exclude_proxy.properties. Here, the IP address or DNS hostname is expected for computers to which the connection is to be made directly, that is, without a proxy. One exception is expected per line. The wildcard * is allowed.

If the entire file does not exist, the addresses 127.0.0.1 (loopback address) and 0.0.0.0 (all local addresses of this machine) and the name localhost (loopback name) are assumed by default.

HTTP servers of the Integration Server

The Integration Server provides a way to set up one or more HTTP servers via configuration files (and temporarily in the Admin Console). See section HTTP server of the Admin Console.