Settings

(1) Swagger import: You can obtain these templates from the Update Center. To the right, you can upload a swagger file (directly or via URL). Various settings will be made in the Input Agent (HTTP method, URL, content type) and, if necessary, a suitable source structure is created (and target structure via 1:1 mapping). If the path contains a URL variable, suitable MSG_CALL_ variables are created in the profile and inserted into the URL (→ trigger profile via HTTP).

Example: https://petstore.swagger.io/v2/swagger.json and path [GET] /pet/{petId}.

Example: https://petstore3.swagger.io/api/v3/openapi.json

(1.1) Endpoint: Enter the URL (or upload a file). Example: "https://petstore3.swagger.io/api/v3/openapi.json".

(1.2) Request: Click here.

(1.3) Path: Select the desired path.

(1.4) Apply: Click here.

Note: See also section Designer (Data Flow) (→ Swagger import).

(2) API protocol: The option Is a soap WS request is only required if a SOAP web service is called. If set, additional buttons will appear below the data field. See section Calling SOAP web service with profile in Input Agent.

(3) HTTP method: The HTTP method to be used (GET, GET (with content), POST, PUT, PATCH, DELETE, DELETE (with content), HEAD).

(4) URL: The URL (with port) of the remote HTTP(S) server. Note: There is no URL encoding. Note: The placeholder <serverurl> is replaced by the value of the channel field "Partner address" if a channel (14) has been selected. Note: You can use the placeholder {--0:s#<template>--} in the URL for the timestamp of the last run of the profile. Please note that the timestamp is only updated if you actually use the placeholder in the URL or in (13). In addition, the HTTP response status code must be >= 200 and < 300 . Example: http://example.com?time={--0:s#yyyyMMdd--} Note: See also section Paging/Pagination below. Note: You can use variables (syntax @MSG_CALL_MYVAR@), system constants (syntax %MYCONST%) and permanent profile values (syntax %perm:KEYNAME%) (→ selection menu).

(5) Via DMZ: If set, the DMZ server (if available) will be used for sending the request instead of the internal Integration Server. Note: This also affects the OAuth token refresh if a channel (14) with configured OAuth was specified.

(6) Encode URL automatically: If the checkbox is set, the value in (4) will automatically be URL-encoded.

(7) Timeout: Timeout for the HTTP connection.

(8) Adjust HTTP headers: Here you can add additional HTTP request headers. See section Request Headers below. Note: If a channel is used, headers can also be defined there in the additional IDs. Note: You can use variables for the header names and the values (syntax @MSG_CALL_MYVAR@). For the values you can also use system constants (syntax %MYCONST%) and permanent profile values (syntax %perm:KEYNAME%) (→ selection menu).

(9) Create request & structure: Executes a request and opens the file structure analysis dialogue for the response in order to generate a suitable source structure. Note: Can also be used if SOAP is set in (2) in the event that the server does not provide a WSDL.

(10) Multipart/form-data: For creating a multipart request. You can insert parts or file parts. See section "Multipart" below.

(11) Content type: The MIME-Type (Header Content-Type ). Note: By default, the encoding (from the "Main settings") is also specified in the Content-Type, e.g. application/octet-stream;charset=ISO-8859-1. If you want to prevent this, you can specify value application/octet-stream;raw in the field instead. In that case, only the value application/octet-stream is used for the Content-Type. If you remove the tick in (12), this will be entered automatically for you.

(12) Add encoding to content type: See note in (11).

(13) Data: The body data (not available for HTTP methods (3) HEAD, GET and DELETE or if (10) is used). You can use system constants in the form %MYCONSTANT% in (13). If the system constant has a date placeholder as its value, e.g. <yyyy>-<MM>-<dd>, the current date is output in this format. Note: You can use the placeholder {--0:s#<template>--} in the data for the timestamp of the last run of the profile. Please note that the timestamp is only updated if you actually use the placeholder in the data or in (4). In addition, the HTTP response status code must be >= 200 and < 300. Example: {--0:s#yyyyMMdd--} Note: You can use MSG_CALL_ variables.

(14) Channel selection: Here you can select a channel of type "HTTP". The authentication data from the channel is then used in (16) (fields " Own ID" and "Own password" ).

(15) Certificate selection: If (4) is an HTTPS URL, you can optionally assign a client certificate. To do this, you select one of the local certificates.

(16) User, Password: Optionally, authentication data can be specified here. MSG_CALL_ variables are allowed. Alternatively, the authentication data of a selected channel can be used, see (14). For Windows users with NTLMv1 or NTLMv2 authentication:

If the HTTP server to which you want to connect is a Microsoft IIS and expects an integrated Windows NTLM authentication of version 1 or 2, a domain name can be placed in front of the username.. The following syntax is expected in the user name field (please note the double backslash).

(17) Ignore HTTP status not matching 2xx: If this checkbox is set, then HTTP status codes not equal to "2xx" (success) do not cause the profile to terminate. This allows custom error handling. See section "Response and Response Headers" below. Note: Please note, however, that this does not work if the request does not work and does not return any data at all, for example, if the server cannot be reached. If the profile does not receive any data, no job is created.

(18) Activate parallel processing: If this checkbox is set, several instances of this profile can work in parallel. The checkbox Profile may only run in one instance must not be set then for this profile. Note: If this option is set, jobs of this profile are processed in a Thread Queue.

(19) Activate paging: See section "Paging/Pagination" below.

Multipart

Request headers

Additional request headers can be inserted via (8). If a channel (14) is used, request headers can also be set there, see section Additional IDs (in channel). If a header with the same name is set, the one set in the profile has priority.

Note: You can use header Set-Cookie to write cookies.

Response and response headers

The execution of the profile is continued for all response status codes 2xx, otherwise an error occurs. However, this behaviour can be changed with checkbox (17).

Response headers can be read with the system variables MSG_CALL_VAR_HTTP_<name of header in uppercase letters>. See also MSG_CALL_VAR_HTTP_STATUS_CODE and MSG_CALL_VAR_HTTP_STATUS_LINE.

Raw request/response

If you want to see the raw request and the raw response (and the respective headers) for test purposes when using HTTP method POST (especially with multipart), you will find these on page "General messages" in the Control Center as a Base64-encoded string if you activate the trace messages for phase 1 in the logging configuration. You can decode this string with the plugin "Encode/Decode". Alternatively, you can of course use external tools such as https://requestcatcher.com/.

Paging/Pagination

Paging (or pagination) refers to the process of having several partial data sets delivered when calling a REST API that delivers very large data sets.

You can implement this in the following way. You create a profile with this Input Agent and enter the first URL. This URL depends on the REST API used. Example: "https://example.com/users?limit=100&page=1".

The first call is made by a normal start of the time-driven profile.

To activate paging, the checkbox "Activate paging" (19) must be set.

In addition, the variable VAR_SYS_HTTP_PAGING_URL must be created.

You now fill the variable in the mapping with the URL for the next request (i.e. for the next partial data set). You either have to create this URL yourself or you receive it in a response header of your request (readable via variables, as you know it). The profile is then automatically started again with this URL (which creates another job). Note: A set checkbox (6) has no effect here.

This process is repeated until the variable contains the same value that it had in the last request or if the last request returned an empty body. If the checkbox Other pages can be accessed via the same URL when paging is also set, the process is also repeated if the value of the variable does not change. Attention: Please use this option with caution, as it can generate a large number of jobs! An automatic cancellation occurs after 1000 repetitions.