Introduction
This documentation describes the “Integration” functionality of the Lobster Data Platform.
There is an additional documentation for the “Orchestration” functionality of the Lobster Data Platform, which can also be accessed via the GUI (→ Context help via the question mark icon). These two documents will be merged in the near future→ Direct link to Orchestration documentation.
If the hierarchical structure of the documentation here does not correspond to the hierarchical structure of the Lobster Data Platform GUI, this will be explicitly pointed out in the text (see, for example, below for “Users,” “Clients,” and “Partners” or in the respective page title (in parentheses).
Users
Users can be managed under "Administration/Accounts/Users".
User permissions (roles): "Administration/Accounts/Roles".
Sessions: "Administration/Accounts/Sessions".
Partner
Communication partners can be managed under "Administration/Partners" (→ "Partners/Channels", "Certificates").
General control elements
General GUI control elements of the GUI are described here. Although the GUI is designed very intuitively, a short glance will certainly not do any harm.
Help and searching
Wherever you see a question mark icon in the GUI, online documentation is available.
Functions are an exception. Here, class-specific documentation is opened via the context menu of a function in the function list.
When searching for documentation, if possible, always go via the context. This automatically places you in the right area of the documentation. From there, usually every relevant topic is linked and you can also navigate in the menu on the left. If this does not lead to a result, you will also find a search window in the documentation at the top right.
For search options, please refer to section "Search and find". In addition, you will find an expert search in many overviews.
Glossary and concepts
Specific explanations of technical terms and concepts used in the documentation are listed here. But you will usually find direct links in the documentation when these terms/concepts are used. So you can get familiar with them if necessary.
Data processing in Lobster Integration
As already shown, the software consists of two components. A server component where data processing is performed, and a client component (browser GUI) that allows users to perform monitoring and troubleshooting tasks, administer communication parameters for business partners, and develop, test, and maintain profiles.
A "profile" represents a data processing program in graphical form, without the need to program. If data is received by the Integration Server and assigned to a profile, a job is created for this profile. Alternatively, a profile job can be started time-controlled or triggered by another profile. These jobs are the core units of data processing.
Additional data processing modules are also available, such as Content Inspection and Asynchronous Sending Module (ASM).
API (Java)
We provide a programming interface (API) that also allows you to develop your own components (preparsers, functions, etc.) in Java. However, other things are also possible, such as tapping changed profiles for external archiving (of the profile before the change). We offer in-depth training for this. If you are interested, please contact our Customer Success Team (customer.success@lobster.de).
Stable operation (system monitoring, error handling, configuration check)
The manual monitoring of the complete data processing takes place in the Control Center. You will find all the running, finished and planned jobs there. It is also possible to restart aborted jobs after, for example, incorrect input data has been corrected or previously unavailable data sources are now available. Also take a look at section Logging.
For automated monitoring you will find an overview of the possibilities in section concepts for ensuring stable operation.
You can request an automated check and optimisation of your installation via our support team. See section Lobster certified.
Backup concepts
See the following sections.
Password security in configuration files
Since you will probably access many different systems, it is unavoidable for passwords to be stored in configuration files. To handle these safely, we offer several mechanisms. See sections Obfuscation and Masked and Encrypted Passwords.
Performance
See sections OutOfMemoryError, Performance and Remote server for performance improvement.
In addition, profile jobs can be throttled if the CPU exceeds a certain load over a defined period of time. See the corresponding option in section Backup/Limits for details.
Docker/Kubernetes
Your software can also be run in a Docker/Kubernetes environment.
In this case, it is recommended to set the following system properties:
hub.datawizard.docker=true
hub.datawizard.enableDmzIpResolving=true
The character set can be customised in the following way:
file.encoding=UTF-8