Retention periods of backup files, logs, crashed jobs
If an Input Agent receives files and assigns them to a profile, a job is generated for this profile and the input files are saved in the backup directory. These backed up input files can then later be accessed to restart a job, for example. Note: See also system variable VAR_SYS_BACKUP.
While a job is being processed, logs are being generated. In addition, there are Comm Logs especially for the communication via partner channels (see section below).
Default values for database-based logs and backups
The configuration for backups and also for database-based log files (except the Comm Logs) is done in the file ./etc/startup.xml, whereby individual parameters may not be used if they are overwritten directly in the profile. See table below. See also section Logging.
Note: On test systems, the database-based logs and backup files, regardless of the settings, are held for a maximum of 5 days.
...<Set name="backupDir">./datawizard/backup/</Set>...<Set name="minimumFreeDiskSpace">2048</Set><Set name="shutdownOnLowDiskSpace">false</Set>...<Set name="retainDays">30</Set><Set name="retainDaysLog">60</Set><Set name="retainDaysForStatistic">90</Set><Set name="retainDaysLogForNoBackupSettings">7</Set>...<Set name="generalLogDefault">5</Set>...<Set name="moveBackUpFiles">false</Set><Set name="multipleBackupSubDirs">true</Set>...<Set name="trackCrashedJobs">true</Set><Set name="startCrashedJobs">true</Set><Set name="restoreWaitTime">90</Set><Set name="ignoreOldCrashedJobs">3</Set>|
Parameter |
Description |
|
backupDir |
The backup directory is set here. |
|
minimumFreeDiskSpace |
The free space in the backup directory is checked. If the set value is undershot, an email is sent. If the free space is only 25 % of the minimum value, the job is aborted. Default: "2048" MB. Minimum: "1024" MB. |
|
shutdownOnLowDiskSpace |
If "true" is set here, a shutdown of the Integration Server is initiated by Force Stop, if "minimumFreeDiskSpace" is undershot. |
|
retainDays |
The value is used to specify the number of days that backup files will be kept. This value also applies to files on the "Unresolved" page. The value is used if the profile option "Retention time of data backups" is set to "-1". If another value is set, then the retention time for the respective profile changes. See examples below. Important note: A change does not have a retroactive effect because the deletion time is already set when a backup file is written. Example: You change the value from "30" to "60" and then restart the Integration Server. All backup files for jobs that were executed up to this restart are removed after 30 days. All backup files for jobs executed after this restart will be removed after 60 days. |
|
retainDaysLog |
The value is used to set the number of days that detail log entries are kept (detail logs in tables "dw_log" and "dw_log_fields", where "dw_log_fields" contains the values for fields that have the attribute "Log field value" set). The value is used if the profile option "Retention time of data backups" is set to "-1". If another value is set, then the retention time for the respective profile changes. See examples below. Note: This value is also used for the logs of the MFT server. Important note: A change does not have a retroactive effect because the deletion time is already set when an entry is written. Example: You change the value from "30" to "60" and then restart the Integration Server. All logs for jobs that were executed up to this restart are removed after 30 days. All logs for jobs executed after this restart will be removed after 60 days. |
|
retainDaysForStatistic |
The default retention period of the log entries in table " dw_log_sum" . See examples below. Important note: This value cannot be overridden in the profile but can be overridden for individual Log Pools. |
|
retainDaysLogForNoBackupSettings |
The number of days that log entries are stored is defined here if the "Retention time of data backups" has been set to "0" in the profile. See examples below. |
|
generalLogDefault |
The value determines how long general log entries are kept. |
|
moveBackUpFiles |
Specifies whether to delete or move outdated backup files. Value "true" means move. Default: "false". The base destination directory is "./datawizard/moved". Attention: The files are moved using the "rename" method, which is successful only if the target directory is on the same hard disk partition as the original backup directory. |
|
multipleBackupSubDirs |
The value determines whether the backup files of a profile are to be distributed over several subdirectories. If this value is set to " true" , all existing backup files of all profiles are distributed to several subdirectories after a restart. |
|
trackCrashedJobs |
Enables handling of crashed jobs - recommended. See section System crash and crashed jobs. |
|
startCrashedJobs |
If "false", crashed jobs are marked as faulty and are not restarted when the Integration Server is restarted. No error emails are sent for these jobs. Default: "true". |
|
restoreWaitTime |
Wait <x> seconds before restarting queued and crashed jobs after startup. |
|
ignoreOldCrashedJobs |
Do not start crashed jobs older than <x> days. "0" disables this option. |
Default values for file-based logs
File-based logs are deleted by default if they are more than 7 days old. This default setting can be changed via the following entry in the configuration file ./etc/log.xml. This entry should be placed in the file directly below "<Set name="defaultManager">File</Set>". Parameter "maxDays" specifies the retention time. To make the change effective, the Integration Server must be restarted. Note: The retention time for the respective LogManager can also be changed in the Admin Console in the LogService. This change will only be effective until the Integration Server is restarted.
<Call name="addManager"> <Arg>defaultdatawizardlog</Arg> <Arg> <New class="com.ebd.hub.services.log.file.FileLogManager"> <Set name="maxDays">42</Set> <Set name="rootDir">./logs/DataWizard/</Set> </New> </Arg></Call>Comm Log
The retention period of Comm Log entries (in table "comm_log") can be specified in days in the partner channels "Retain days". The default value, which is defined by parameter "retainDays" in configuration file ./etc/commlog.xml, is used if value "-1" is set in the GUI. The entries are held indefinitely for value "0" in the GUI. Entries whose retention period is exceeded are automatically deleted.
Best practice
Normally, you will probably want to search for jobs in the Control Center. The parameter "retainDaysForStatistic" must be set to the longest time period in which you want to search. If you want to retain the detail logs for all profiles for the same length of time, you must set "retainDaysLog" to the same value. If you only need the detail logs for such a long time for individual profiles, you should set "retainDaysLog" to the value you normally want and only overwrite the value for the individual profiles. Keeping the detail logs longer than the entries in table "dw_log_sum" is possible, but only makes sense in very few cases.
Examples
Assumes the values for parameters "retainDays", "retainDaysLog", "retainDaysForStatistic" and "retainDaysLogForNoBackupSettings" are set as in the XML file above.
|
Retention time of data backups in profile = "-1".
|
|
Retention time of data backups in profile = "0".
|
|
Retention time of data backups in profile = "42".
|