Importing addressbook entries
◄ Back to Weiteres Beispiel für Zuordnungskriterien
Procedures on this page:
This example assumes access to the Lobster_data installation on which the Lobster Data Platform / Orchestration is operated, as well as basic knowledge of how to use Lobster_data (creating profiles).
At the end of the tutorial the following task can be executed to learn about the interaction between lobster_data and Lobster Data Platform / Orchestration (see also Lobster_data integration).
Specifically, a list of addresses in CSV format is imported in Lobster Data Platform / Orchestration to add new Adressbucheinträge to an existing Address book or to update existing ones.
Preparations for the import
Create CSV file
The data to be imported should be in CSV format as a file. The semicolon (';') is expected as a separator between columns. The following columns should be filled per row:
ADNR → an identification number for the address, which is taken over into the field 'Account number' (accNumber) of the address during import.
►IMPORTANT◄ This code number is used to determine whether an address book entry already exists for an address, which the import should update. So it should be unique within the list!NAME1 → the text for the primary name field 'Name 1' (name1) of the address
STR → the text for the field 'Street 1' (street1) of the address
NR → the text for the field 'Street no.' (streetNo) of the address
CC → the text for the field 'Country code' (countryCode) of the address (see the dynamic enumeration Country)
ZIP → the text for the field 'Postal code' (zipcode) of the address
PLACE → the text for the 'City' field (city) of the address
The following sample data can be used to create a corresponding file. Of course, a file with the same structure can also be prepared with your own data.
ADNR;NAME1;STR;NR;CC;ZIP;PLACE1234;Duck, D.;Waltstr.;23;DE;26725;Emden-Hausen1252;Oliver, Roy;Burbank-Platz;71;DE;88316;Isny1860;Torpodofaktur KG;Blauweisswalder Str.;186;DE;81060;Giesing1968;SpringTimers sro;U Vesna;218;CZ;19068;Praha1999;Rogers, N.;Tafkap Drive;1958;US;55488;Minneapolis (MN)Prepare a CSV file for import from this data or according to this model, and save it in a local directory under the name VIP_ADDR.csv!
Create address book
An independent address book is set up as the destination for the address import.
Via the menu path Administration/Master data/Address books, you get to the overview for address books. Create a new Address book with the Name 'Consignees' and assign the Types shown in the screenshot (see Company type):
The newly created address book is assigned a unique internal ID (id) when it is saved. This integer ID is used in later steps to identify the address book.
►NOTE◄ The image for the 'ID' column has the value 105. Use the different ID value of your address book if this value is used in the following steps!
Create import profile
To Import the Adressbucheinträge, a new profile must be created in Lobster_data and then configured as described in the following section.
General data
Parameterize the General data tab of the new profile as follows:
Name → a descriptive name for the task of the profile e.g. 'ADDR_IMPORT_CNE' ('CNE' is the internal abbreviation for the company type 'Consignee', see Company type)
Document type → CSV
Phase 1: Input agent
Select the input agent Manual upload from the category 'Event controlled'.
Phase 2: Parser
For the parser the shown default settings can be used with one exception:
The value for Start at row should be increased from 1 to 2 because the CSV file contains column names in the first row, which of course should not be read as address data.
Phase 3: Mapping (Source structure)
The source structure for the mapping in Phase 3 can be created largely automatically using the Create structure from file analysis function by importing and evaluating the file 'VIP_ADDR.csv':
The function is called from the menu below the Source structure area. It allows the 'import' of the locally saved file via a file browser or by drag & drop.
The default settings can be retained for the parameters that can be selected in the Analyze file dialog during file analysis:
The value 1 for Row(s) has the effect here that in contrast to the parser configuration (see above), the first row is processed to interpret the column names as field names.
With the options Add as new test file and Load and show test data, the file is available for testing.
The file analysis with the sample data results in the following source structure (including the display of Test data and Data type):
The Row1 node was automatically inserted to merge the fields.
For this node, the record type identifier 'Is not: _____ nothing _____' is set as the record type identifier for this node. So all rows of the file are evaluated as a 'row' with the given fields.
Phase 3: Mapping (Target structure)
Import templates
The Target structure has to be set up appropriately for the planned Import. Lobster SCM templates are used in two steps:
Since a Batch import is to be executed, the core:BatchImport template must first be loaded into the Target structure.
Afterwards, the base:AddressBookEntry template with the Append structure as subnode option must be loaded to the batch node of the target structure then created:
Settings for 'BatchImport' node
The desired transaction control option (SINGLE or BATCH) must be selected in the trx_Control_attr field in the 'Header level' of the BatchImport structure:
With the BATCH option selected by fixed value in the field, the import of all rows is treated as a transaction. The import is then either successful for all 'rows' or it will fail completely.
With the alternative SINGLE option, each imported row would be treated as an independent transaction that can or cannot be successfully imported.
The batch node defines the import action repeated in the loop over all imported lines:
In the field action_attr, CREATE_OR_UPDATE is assigned as a fixed value, since only new addressbook entries should be created if no 'suitable' update is found.
Alternative options here would be CREATE (only add new ones), UPDATE (only update existing ones) and DELETE (delete existing ones).
The node eventStorage is not needed. It can be deleted or set to inactive (see above).
Further details are documented for the Single import, which corresponds to a single repetition of the batch node.
Define search for existing addressbook entries
For the Import with the CREATE_OR_UPDATE action, the search subnode is used to define a Search which checks for each source data line whether a unique target for an UPDATE can be identified or whether a new address book entry is to be created with CREATE.
In the concrete application case the following criteria will be examined:
The identifier read as ADNR should be exactly equal to the value of the address field 'account number' (address.accNumber) in the address book entry.
The address book entry should be contained in the address book created above. Then its internal ID (in the example: 105) appears in the addressBookId field of the addressbook entry.
The corresponding Search can be easily set up and tested with the Search builder, which is started in the Lobster Data Platform / Orchestration via the menu path Administration/System/Search builder.
|
The entity 'Address book entry' is selected as the Entity of the Search.
|
|
If the Search is structured as described and successfully tested by Execute (in the ribbon), its definition can be transferred to the Lobster_data profile:
►NOTE◄ Alternatively, it is also possible to 'build' the same structure by inserting Lobster SCM templates for the contained structures (core:SearchJunction, core:SimplePropertySearch) and parameterize them accordingly. However, at least for somewhat more complex searches, this method is certainly more time-consuming and more difficult to perform error-free than copy and paste via the clipboard. |
|
For insertion into the target structure of the Lobster_data profile, the search node should be selected there. Then the Lobster SCM templates function is selected from the target structure menu.
|
►IMPORTANT◄ In the dialog that opens, the option Append structure as subnode must be selected. The account search should already be selected in the target structure.
|
|
Now the link from the source field ADNR to the comparison value for the 'Account-No.' field within the second 'Simple property restriction' (PropertyProjection) must be created.
►NOTE◄ Alternatively, the link can also be created by drag & drop from the source field (1) to the target field (2).
►NOTE◄ Only when this has been removed and the Fix value property field has been exited does the 'lock' symbol for the field in question disappear and the value (1234) in the Fix value column (if displayed). |
|
Mapping for address data
|
Finally, the 'mapping', i.e. the assignment between source and target fields for the actual user data, must be set up. As in the previous step, this can be achieved by selecting and linking or by drag & drop.
|
|
Phase 5: Integration Unit
For the preparation of the target structure the option Use IntegrationUnit & post execution must be activated in Phase 5 so that an Integration Unit can be selected.
Since an XML structure is expected for the import in Lobster Data Platform / Orchestration, the XMLNoTemplateUnit must be selected here for creation.
The default values of the configuration can be kept here, only the Root node name must refer to the relevant 'root node' in the target structure. This is the 'BatchImport' node here.
Phase 6: Responses
The minimum configuration for Phase 6 is a response path of the type 'Custom class', for which the Class ImportResponder (_data-Responder) is selected.
Under Additional parameters a base LobsterDataLoginRequest with login data for Lobster Data Platform / Orchestration must be provided here (see ImportResponder (_data-Responder)).
The base LobsterDataLoginRequest for a logged in session can also be copied from the XML tab in the Search builder via the clipboard.
However, the xmlns:base="SCM.BASE" namespace attribute defined by the Search builder in the core:SearchTask parent element must be added.Copied, ...
<base:LobsterDataLoginRequest [...] userName="admin" selectedRole="101" selectedCompany="51" ... />
... Completed:
<base:LobsterDataLoginRequest xmlns:base="SCM.BASE" [...] userName="admin" selectedRole="101" selectedCompany="51" ... />
In the second Content settings tab, the option 'Output of IU' must be selected for Content so that the BatchImport XML from Phase 5 is passed to the responder.
Subsequently, the configuration for the reply path Apply must be accepted before the profile is saved as a whole.
Execute import profile
The profile can now be executed by 're-running' with the file 'VIP_ADDR.csv' to import the contained addresses.
If this procedure is repeated with a revised version of the file or an identically structured file with other data, 'known' addressbook entries should be updated and only 'new' entries should be added.