Project delays, cost explosions, quality issues: these are the three key management topics when it comes to combining business functions into one integrated business process. Whether you’re dealing with connections between different companies or between different business units within one company: with its versatile options for generating efficiency benefits, business integration is an important tool for the optimisation of value creation within a company.

However, linking IT systems is a complex undertaking and many mid-size and large companies report having issues during the implementation. The reasons for this are diverse but well known. The fact that programmers are in short supply and that the situation on the job market is only getting worse is nothing new. And almost everyone in IT circles knows that the number of interface projects is growing exponentially with the mega-trend of digitisation and Industry 4.0.

Most IT departments are thus suffering under a mix of staff shortages and excessive amounts of work. Those relying on external support also face long wait times and high costs. Overall, this results in significantly longer project times and higher costs due to additional external services as well as tedious project delays when integrating customers and suppliers, automating internal processes and connecting cloud systems, for example in the context of software as a service (SaaS).

In this case, the solution to the dilemma between cost drivers and time delays is also automation. Modern standard software for business integration makes it possible for departments to manage numerous business processes themselves. This does more than relieve the IT department. It creates a high degree of flexibility with customers and partners via agile project management.

Standard software can thus help to implement projects internally without additionally burdening IT specialists, provided it has a logical structure, ease of use with functions such as drag and drop, multi-device capability, automatic feedback in case of error and continuous documentation. The integration of data can then be initiated as automatism in just a few steps in a transparent, verifiable manner.

The important preliminary work primarily involves the collection of existing data, storage locations and available formats. It must then be clarified which data and file formats should be used in the target system. In addition, an intensive control of the data quality is essential. Incorrect entries lead to incorrect deliveries, incorrect customer addresses and process interruptions even after data integration. Proven analytical methods and tools allow you to quickly identify redundancies, revise incomplete and erroneous data, and uncover contradictions in data from multiple sources. From this point on, solid integration software helps to quickly increase the desired efficiency effects. The following seven steps help with the integration.

1. Define input data

With a pure data pump, the required data content is forwarded unchanged via one or more output paths. If mapping (see step 3) is necessary, an input agent should accept the data to be processed and transfer it to steps two and three so that the data can be brought into the corresponding structure. Modern integration software can actively retrieve the desired data once or recurrently.

Alternatively, the software waits for the delivery of the data to be processed by the desired partner in an event-driven or reactive manner. Similarly, times and intervals for data collection should be flexible. Ideally, the system should allow direct connections to as many systems as possible.

2. Clarify data structures

In step two, you generally clarify the structure of source data and document types such as CSV or Excel, XML, EDIFACT or SAP IDoc. This provides the software with the information on how to convert the data. Sometimes there is specific information that only relates to a specific format. This should also be configurable. This includes, for example, CSV quoting in CSV, the data sheets in Excel or schema files in XML. Furthermore, the integration software should have no problem with compression formats like zip or tgz.

3. Set up data mapping

Mapping involves generating the target or output data. This is why mapping is now generated. This refers to the desired target structure and the relationship between source and target structure. With a little technical understanding, this is not rocket science.

For advanced individuals: make sure you can also manipulate the assigned target data via function chains. In addition, check that your integration software includes the largest possible number of prepared functions. These aid you in creating even complex mapping, for example with the help of variables, lists and maps. As described in step 1, the simplest case is 1:1 mapping. Here, the source structure is mapped unchanged to the target structure. This means that the selected source fields are assigned to the corresponding target fields.

4. Describe databases

If you want to transfer your data to a database, you can determine whether, and if so, which target data is written to which database tables. Your software must naturally know the schema in which the corresponding table is created.

5. Design data integration

If the data integration software simply takes on the function of a data pump, the source file is output unaltered. However, data often needs to be transferred into other target formats through mapping. You may also need additional processing for the target format, for example if only part of the source data is to be used in the target format or if another structure is to be used in the target format.

Your integration software must be able to handle this. And now and then it gets even more complicated. For example, you may need several different revisions for the source data in different formats such as data from Excel or CSV lists, which are transferred to EDIFACT for the processing of orders, delivery notes and invoices.

6. Data transfer

The software retrieves the data from the source or waits for its input according to the preset value and transmits the data to the target system(s) on one or more response paths and in the desired form. If the integration software enables the setting of subsequent profiles, the transferred data can be automatically transferred to further systems and processed further in the same way.

7. Troubleshooting

The data source does not deliver or is not accessible, target systems are not available: a fast reaction is important here. The integration software should be able to automatically send a message, for example via e-mail or SMS, with the information about where a predefined task was not implemented. Modern software makes it possible to activate a notification function for each step. Sufficient security is provided when the system also uniquely and continuously documents all incidents and also allows source and target data backups.

The efficiency gains in terms of business integration arise from smooth but also quickly customisable and flexibly configurable automated processes. With an intuitive user guidance from the software, which has multi-device capability, and the individual expertise on the professional side, companies can bypass the business integration bottleneck, optimise on-time delivery of projects and keep costs under control even with increasing data volumes, rising numbers of integration issues and a high utilisation level of the in-house IT resources.