HTTP (Input Agent)

Introduction: You can find a description of this phase in section Phase 1 (Introduction). See also section HTTP and Lobster_data.

images/download/thumbnails/131699110/image2020-3-17_15-13-48-version-1-modificationdate-1679643968648-api-v2.png

images/download/attachments/131699110/1872-version-3-modificationdate-1714623085231-api-v2.png

images/download/attachments/131699110/1874-version-1-modificationdate-1700643989867-api-v2.png


(1) You can obtain these templates (with presets) from the Update Center.

(2) The URL suffix. The URL for this profile is http(s)://<URL or IP of the Integration Server>/dw/Request/example or .../request/example. See section Request Procedure below. Note: You can see the leading part of the endpoint URL via the question mark icon on the left. This, together with the suffix, then results in the total URL.

(3) Only available in mode REST web service (7). Path parameters can be specified here. Exactly one path parameter can be defined between the slashes (see screenshot). Each specified path parameter can be read out with a system variable MSG_CALL_REST_API_PATH_<name of path parameter> (must be defined). Example and notes:

(4) The allowed HTTP methods. Note: If data is to be delivered with a DELETE, the checkbox (5) has to be used. If no payload is delivered, an error is generated in that case. Note: Body Data is supplied depending on the set document type.

XML

<body_data/>

JSON

{"body": "data"}

Everything else

body data

(5) See note in (4).

(6) If this checkbox is set, several instances of this profile can work in parallel. The checkbox Profile may only run in one instance must not be set then for this profile. Only works with a fixed response (13). The response then takes place immediately and not when the profile job is completed. Note: If this option is set, jobs of this profile are processed in a Thread Queue.

(7) The mode of the HTTP Input Agent.

HTTP interface

The standard mode. HTTP requests are received via the URL described in (2), in this case https://localhost/dw/Request/example. The options (3) and (9) are not available here.

REST web service

The Input Agent acts as REST interface. So if like here, the URL of the Input Agent is https://localhost/dw/Request/example, a request https://localhost/dw/Request/example/orders would be accepted as well. The additional request suffix /orders would then be stored in system variable MSG_CALL_HTTP_PATH_DATA. Otherwise, the Input Agent behaves in the same way as in mode HTTP interface, with the additional options (3) and (9).

SOAP web service

The Input Agent acts as a SOAP web service. See sections Profile as SOAP Web Service (static response) and Profile as SOAP Web Service (dynamic response).

(8) If this checkbox is set, the mapping is active and option REST web service (7) is set, a Swagger/OpenAPI interface description is provided in JSON or YAML format. Note: The profile must be saved first.

Example: https://localhost/dw/Request/example?api or https://localhost/dw/Request/example?api&yaml

You can also create your own swagger files via (19), which are then used instead of the automatically generated ones. These files have the name <profile name>. json or <profile name>.yaml and are located in the directory ./conf/openapi. The structure of these files is not described here, please refer to the OpenAPI documentation.

Additionally, a Swagger/OpenAPI interface description in YAML format will be provided to you in a graphical tool (see link on the right). There you can send requests to the interface. URL: http(s)://<URL or IP of the Integration Server>/api/?endpoint=<URL suffix>

Example: https://localhost/api/?endpoint=example

See generally section System Interaction via HTTP (REST API) (instructions there). The data for authentication for profile interfaces comes directly from (17). Important note: You can combine several such profiles into an overall REST API. See section Main Settings (Data Flow).

images/download/attachments/131699110/1345-version-1-modificationdate-1679643968632-api-v2.png

(9) Only available in mode REST web service (7). All stored test files are displayed here in a further dialogue. There you can specify for each file for which HTTP method (4) they are available in (8). These are then displayed (left) in (8) and can be selected (right) for a request (e.g. with POST).

(10) The MIME type of the response.

(11) Optional name for the multipart of the main data. Normally, Lobster_data will search for a part with name file. If the name is not file, you can enter the alternative part name here. Note: This only applies to multipart requests and if no part with the name file was found! See also section Handling HTTP Multipart Messages below.

(12) An empty HTTP body (e.g. with POST) is not considered an error.

(13) The response for the success case can be defined in two different ways. The response in the event of an error is always defined in (15).

(14) Fixed response value, see (13). Line breaks can be inserted with <br>. Note: The placeholders {--jobnr--} or @VAR_JOBNR@ and {--uuid--} or @VAR_SYS_MESSAGE_ID@ are available. Phase 3 variables can also be used with {--@MYVAR@--} or @MYVAR@. Phase 3 must be active for this.

(15) Static response that is sent in case of an error, for both cases in (13). Note: You can use the placeholder {--Exception--} to return the exception that occurred. If the exception message contains a substring §...§, then the placeholder is replaced by the substring text. Example: If you call the function throw exception("§Sorry!§")) in phase 3 of the profile, the placeholder {--Exception--} is replaced by the string Sorry!. Alternatively, the placeholder {--FullStackTraceException--} can be used to output the full stack trace.

(16) The HTTP status code returned in case of an error. The button can be used to enter further codes with the corresponding response on error (15). These entries can be used by function throw http exception(a,b) (see description there).

(17) Either no access control or access control either via one or more selected partner channels (Your Access) or with directly specified login data. If access control is used, the authentication method (BASIC, DIGEST, OAUTH2, SYSTEM) can be selected. See also system variable MSG_CALL_HTTP_AUTH_USER.

  • For SYSTEM, the setting from the configuration file ./etc/startup.xml (→ forceDigestAuthentication) is used. Is shown on page Start .

  • If OAUTH2 is used, you must specify an HTTP channel with OAUTH2 configuration.

(18) Specification of the mandatory request parameters (you can also select system constants and specify regular expressions with the prefix regex: or regex2: ) . See sections Request Procedure and Request Parameters below.

images/download/attachments/131699110/529-version-2-modificationdate-1700646219814-api-v2.png

http://<IP/URL Integration Server>/dw/Request/example?param1=value1&param2=value2&param3=value3&param4=something

(19) See (8).

Request Procedure


The HTTP servlet is started when Lobster_data is started and responds by default to both the HTTP context /dw/request and /dw/Request. Please also see section Defining URL Context System-wide.


http://<URL/IP Integration Server>/dw/request/


For an incoming request, the HTTP servlet first looks for the right profile. All active profiles that have an event-driven HTTP Input Agent and whose suffix (2) matches the suffix in the request are considered.


http://<URL/IP Integration Server>/dw/request/<suffix>


Additionally, the request parameters must match the mandatory request parameters (18). If, for example, you enter value example in (1) and two parameters (p1=value1, p2=value2) in the parameter list (18), the URL must look like this.


http://<URL/IP Integration Server>/dw/request/example?p1=value1&p2=value2


The first matching profile is started. If the profile requires authentication (17), however, the profile is only started after successful authentication. If other profiles meet the conditions, these profiles will not be considered.

Request Parameters


Request parameters can be read with the system variables MSG_CALL_<name of parameter in uppercase letters>. See also section HTTP Request Parameters and HTTP Headers.

Request Headers


Request headers can be read with the system variables MSG_CALL_HEADER_HTTP_<name of header in uppercase letters>. See also section HTTP Request Parameters and HTTP Headers. Note: The system variable MSG_CALL_HEADER_HTTP_AUTHORIZATION (header Authorization) is not set by default. However, you can force this variable to be set with the system property hub.datawizard.http.copy.allHeader=true.

Further system variables: MSG_CALL_HEADER_HTTP_METHOD, MSG_CALL_HEADER_HTTP_PROTOCOL, MSG_CALL_HEADER_HTTP_REMOTE_HOST, MSG_CALL_HEADER_HTTP_URI, MSG_CALL_HTTP_AUTH_USER, MSG_CALL_HEADER_HTTP_QUERY.

Handling HTTP Multipart Messages


The Input Agent always only reads the first multipart, see (11). But the other parts are not lost. They are stored in system variables.

The name of the second part would be saved in variable MSG_CALL_HTTP_MULTIPART1_NAME and its value in MSG_CALL_HTTP_MULTIPART1_VALUE. The third part in MSG_CALL_HTTP_MULTIPART2_NAME and MSG_CALL_HTTP_MULTIPART2_VALUE, etc. The total number of all parts, including the main part (11), is saved in system variable MSG_CALL_HTTP_MULTIPARTS.

Ideally, you know how many parts you will receive and create the variables accordingly. If this is not possible, you can use a little trick to avoid creating the variables. Simply use system variable MSG_CALL_HTTP_MULTIPARTS (of course you can create this variable beforehand) to find out how many parts were received. You then use this value to iterate over a node and read the respective variable values with function copy field by name(variable/result a). You can create the name of the variable with functions iteration-level and concat(a,b,c,d,e,f,g,h). Please do not forget that variable MSG_CALL_HTTP_MULTIPARTS contains the total number of all parts. But you can only read from the second part on from the variables.

Response Procedure


If no profile could be found, or the mandatory parameters do not fit, you will get a corresponding message back.

If a suitable profile was found, the response is sent when the profile was successfully terminated or an error occurred. However, if (6) is set, the response is immediate.

The content of the response is defined by the settings in (13).

Response Header


If (13) is set and system variables of the form VAR_RESPONSE_HTTP_HEADER_<name of response header> are defined, corresponding response headers are set with the values of the variables. Example: The system variable VAR_RESPONSE_HTTP_HEADER_Test with the initial value mytest creates the HTTP response header Test with value mytest. Note: The response header Content-Length is ignored, i.e. it cannot be overwritten.

Response Status


If the profile was executed successfully, the response status is 200. If (13) is set, the response status 200 can also be overwritten with the system variable VAR_RESPONSE_HTTP_HEADER_RESP_STATUS.

If an error occurred during profile execution, the response status is the value specified in (16) or as set by function throw http exception(a,b).

If (6) is set, the response status is 200 as soon as the profile has been successfully triggered. The function throw http exception(a,b) cannot change the response status in this case.