Repeating structures (Cargo-IMP)

Cargo-IMP message specifications may contain several kinds of repeating structures. The easiest cases are repeats of entire segments. Repeats of substructures of segments can be a bit tricky and the handling mainly depends on the position of the substructure in the segment and whether the repeat starts with a slant and contains no CRLF (slant repeat) or contains a CRLF (carriage return and line feed). We call lines following a CRLF and beginning with a slant 'continuous lines'.


Special repeating case and attribute "condition"


images/download/attachments/189463626/CargoIMP_6-version-1-modificationdate-1738737614870-api-v2.png



If your segment starts with an identifier and a following continuous line is basically a regular line without the identifier, you can use the 'special repeating case' to model this repeat. See repeat 5.2 - 5.4 in the screenshot above for the simplified and fictitious Cargo-IMP message AAA. Ignore the numbering for our example.

The special repeating case is triggered by the use of attribute condition="..." in the message specification of the XML configuration file for the Cargo-IMP parser. Following the XML specification of message AAA. Only focus on the condition line at this point.


<message id="AAA" version="2">
<segment id="ACS" condition="^/.*">
<token pos="2">2,0-12n,1,</token>
<token pos="3">2,0-12n,1,</token>
</segment>
</message>


The condition specifies a regular expression that describes lines that will be used for the repeat. So in our case any line that starts with a slant. The screenshot below shows a corresponding source structure (containing an additional segment Standard_Message_Identification).


images/download/attachments/189463626/890-version-1-modificationdate-1738737614864-api-v2.png

Assume the following input data.


AAA/2
ACS/WT10263.15CAD/WT10000.00CDL
/TX1542.55CAD/TX1503.00CDL
/TX1642.55CAD/TX1504.00CDL


If you parse that file, the log will show following entries.


...
Read line 'ACS/WT10263.15CAD/WT10000.00CDL'
...
Read continuous line '/TX1542.55CAD/TX1503.00CDL'
Adjusting line to 'ACS/TX1542.55CAD/TX1503.00CDL' for special repeating case
...


Note: Do not worry about the EDIfact node. This node is necessary, but can simply be copied from existing Cargo-IMP templates. Apart from that, you will not have to deal with this node.


Handling slant repeats and attribute "repeat"


images/download/attachments/189463626/891-version-1-modificationdate-1738737614862-api-v2.png


As you may have noticed, the sample data above does not only contain a continuous line repeat. It also contains a slant repeat.

Look at the data line ACS/WT10263.15CAD/WT10000.00CDL. The second token is WT10263.15CAD. If you check our XML message specification, you will see that there is a splitting definition for token 2: <token pos="2">2,0-12n,1,</token>. So we will actually get 4 subtokens as result: WT, 10263.15, C, and AD, which will fill fields Charge_Identifier, Charge_Amount, P_C_Ind and CASS_Indicator in our source structure.

But what happens to token WT10000.00CDL of data line ACS/WT10263.15CAD/WT10000.00CDL ?

There are basically two solutions for this problem.

1. The first solution is pretty much straightforward. You could simply copy your 4 fields and paste them into your structure as 4 additional fields. This will work perfectly fine, but of course, this will only cover one slant repeat (unless you copy those fields multiple times). So as convenient this solution is, it will only make sense for a limited and a low number of repeats. If the allowed number of repeats is infinite, you will not be able to use this method to correctly model your data structure.

2. The second solution is a bit more elegant. Especially if the node you want to repeat (Charge_Details) is the first node of your segment (AWB_Charge_Details with id ACS). As you might have seen (if you rebuilt our example), you did not have to do anything and the slant repeat worked anyway. The reason for that is the segment attribute repeat. The default value for that attribute is 1 and means that if there is any data of a line left, after all, the fields of a segment have been filled, the remaining data will be used to start filling the segment again, starting at node 1. If you do not want that mechanism to work, simply specify repeat="0" and the remaining data will not be used. If you want to start filling the remaining data into node number n, use repeat="n".

Looking at the log for our little test run, you will find something like the following:


...
[M:line=2,Charge_Details] Still data elements to process, using repeating node
[M:line=2,Charge_Identifier] Parsed, assigned value 'WT'
[M:line=2,Charge_Amount] Parsed, assigned value '10000.00'
[M:line=2,P_C_Ind] Parsed, assigned value 'C'
[M:line=2,CASS_Indicator-21] Parsed, assigned value 'DL'
...


Note: Take a look at the additional splitting definition <token pos="3">2,0-12n,1,</token>. This splitting definition will be used for the slant repeats (repeat="n" and hard-coded additional fields) and in the special repeating case (and only there!) also for the slant repeats in all the continuous lines.

Handling CRLF repeats and continuous lines in general


With the special repeating case, we have already seen one way to deal with continuous lines. But that case is only applicable if data for the whole segment is repeated. If you have a CRLF somewhere in the middle of your segment and following data, you need another way to handle it.

If you have no condition attribute defined and therefore the special repeating case does not apply, the normal mechanism for continuous lines is that the parser is looking for a sibling node. A sibling node is simply the next node on the same hierarchical level in your structure, relative to the previous node that has been entered. If such a node cannot be found, you will get a message like the following in your log.


Read continuous line '/TX1542.55CAD/TX1503.00CDL'
Skipping data '/TX1542.55CAD/TX1503.00CDL', no sibling node found


If on the other hand, there is a sibling node, you will get a message like this one (simply try to copy subnode Charge_Details and paste it in your structure, right after the original subnode).


Read continuous line '/TX1542.55CAD/TX1503.00CDL'
Enter sibling node for data '/TX1542.55CAD/TX1503.00CDL'


Unfortunately, a token splitting in continuous lines is not possible at this point, so you might have to merge some fields, as you have already seen. Especially in our case, where we are dealing with repeating data after a CRLF, it might be useful to design the two subnodes (the original subnode Charge_Details and its copy) uniformly by merging the fields in both subnodes. You could still split the tokens of the first subnode, but that might, in fact, be an obstacle if a developer wants to consolidate all the Charge_Details data in the target structure.

Note: Once a sibling node has been entered and there are additional repeats, this sibling node will be repeated for all repeated data, no matter if there are following nodes in the source structure.


Handling repeats of a single segment


If a single segment needs to be repeatable, you do not have to do anything. The repeat will simply work as it is, as long as the segment has a match code defined.


Handling repeats of entire substructures or groups of segments


Repeats of substructures

To handle repeats of entire substructures, you can use so-called repeat nodes. These are nodes whose name starts with repeat. Coming back to our previous example, you might simply rename your subnode Charge_Details to repeat_Charge_Details and the CRLF repeat will work as well. Note that you might also have to merge fields since token splitting is not possible (not even in the first occurrence of the data before the CRLF). In that case, you will find a message like the following in your log.


...
[M:line=3,repeat_Charge_Details] Read continuous line '/TX1542.55CAD/TX1503.00CDL'
[M:line=3,repeat_Charge_Details] Enter repeating node for data '/TX1542.55CAD/TX1503.00CDL'
...


Note: The slant repeat will not work in this case, even if it is specifically defined in the XML. You will have to work with the hard-coded solution by creating additional fields.

Repeats of entire groups of segments

If you want to repeat entire groups of segments, simply create a node repeat and move all your segments into that node. The repeat node will not need any match codes. The match codes of the segments will be enough.

The first repeat node (in the source tree) will be created if any of the subnodes (your segments) in the repeat node is visited.

A new repeat node (in the source tree) will only be created if the first subnode (segment) is revisited.

Note: Be aware that this could lead to multiple occurrences of the second or following subnodes (segments), so make sure to allow a maximum of 99999 to avoid runtime errors.