PI Archiving Conceptual Overview

Applies To: SAP NetWeaver Process Integration 3.0/ 7.0/ 7.10
Summary: This document provides both an introduction to the key concepts of SAP NetWeaver Process Integration XML message archive and delete and overview of the tasks, reports that comes into play in concept of Archiving and Deleting of XML messages. This document targeted for beginners wanting to get involved in the topic.
This document covers Archive and Delete of XML messages of 'Integration Engine' only
Archiving & Deleting PI XML messages Process
By and large Archiving applies to asynchronous (EO and EOIO) messages only on the ABAP and JAVA side. It is a general decision by SAP to not archive synchronous (Best Effort) messages and this is by design unless error occurs during processing.
SXMSPMAST is the master table that holds the key information about each of the messages going through the Integration engine. The data is persisted in this table based on the retention period defined for asynchronous and synchronous messages.
If a message enters the Integration Server the field ITFACTION of SXMSPMAST table is set to INIT. Then the interface is checked for the action to take, if it is listed as an interface for archiving, the field ITFACTION is changed to ARCH. Otherwise it changes to DEL
*Note: ITFACTION filed value in SXMBPMAST table that describes what action applies to the messages but does not mean that that the action is already took place. So it's only action status info not that action already done.
Deleting: All messages with ITFACTION=DEL will be deleted by report RSXMB_DELETE_MESSAGES as soon as message processing is complete (message status is final) and adapter status allows deletion. The Delete job deletes only Payload.
Message are deleted from PI persistency layer if they
are no longer in retention period
are flagged for deletion
have appropriate messages state
have appropriate adapter state
Archiving: All messages with ITFACTION=ARCH have to be archived by report RSXMB_ARCHIVE_MESSAGES first and will then be deleted by report RSXMB_DELETE_ARCHIVED_MESSAGES (similar name as RSXMB_DELETE_MESSAGES, but totally different messages are handled!)
Message are Archived from PI persistency layer if they
are no longer in retention period
are flagged for Archiving
have appropriate messages state
have appropriate adapter state
Archiving XML messages process involves two steps before XML messages are deleted, during first step messages are read from PI persistence layer and stored to archive files and in the second step reads the previously written message from archiving files in order to verify the data integrity. Finally messages are deleted from PI persistence layer.
ERROR Messages: Error messages or messages that are not in complete/final status have to be cancelled first thus cancelling a message forces the field ITFACTION to be set to ARCH in a productive system i.e. this message MUST be archived and it cannot be deleted by report RSXMB_DELETE_MESSAGES anymore, even if ITFACTION initially was set to DEL. Which is as per thumb rule - Only messages in a final status can be deleted / archived. Messages with errors have to be cancelled first. After that these cancelled messages can only be archived and *will not be deleted*!
As mentioned above, error messages have to be cancelled first before they then can be removed (archive/delete) from the database. Canceling is required for asynchronous messages in error state only. By canceling a message its state changes from error to cancel/final and at this state message is allowed to be archived. In contrast synchronous messages are final right when processing fails.
If in case of bulk error messages, all these error messages can either be cancel them manually by selecting these set of messages in SXMB_MONI and then cancel them by pressing button "Cancel Processing of Messages With Error" (CTRL+F8). or report RSXMB_CANCEL_MESSAGES is the tool of choice for mass data handling, i.e. for canceling a large set of messages. This report handles message that already are in error state.
Report RSXMB_SHOW_STATUS helps in getting an overview of persistence layer situation. In general, messages status with 003 is consider final state in Integration Engine. You may refer to either SXMBPMAST table column ITFACTION to know the status overview of XI messages or refer to OSS note# 944727 to know in detail about different status of XI messages. If messages is not in status cancel or any other state (other than final) even after running RSXMB_CANCEL_MESSAGES report as an extreme step, you may use report RXWF_XI_UNUSED_MSGS to convert these messages status to final state (i.e. 003), consequently the subsequent run of Archive and Delete jobs will take care of next course of action. *Important Note - You are highly recommended to experiment running RXWF_XI_UNUSED_MSGS report in Testing Environment before running the same in Production environment.
If RSXMB_SHOW_STATUS shows messages with status no 001 and 008 (adapter status) and not either archiving or delete then schedule report SXMB_REFRESH_ADAPTER_STATUS. This report applicable only if system configured to use 'outbound IDoc Adapter or ccBPM'.
HISTORY Entries: Deleting history entries for processed messages, History entries are spots for observing message processing. They are generated by persisting at the end of a processing step for a message and contain the current status and the execution date.
History entries remain in the database for an unspecified length of time and must be deleted at some stage so that the database table does not overflow. The deletion process only deletes history entries for messages that have already been archived or deleted. The history entry is kept in the database for at least seven days after a message has been deleted. This is necessary since history entries are also required for the quality of service Exactly Once. Deleting these History entries are taken care by standard house-keeping job SAP_BC_XMB_HIST_DELETE_.
Different Archiving/Delete related Reports at glance:
RSXMB_ARCHIVE_MESSAGES Archive XML Messages
RSXMB_CANCEL_NOT_REST_MESSAGES Cancel XI Messages With Errors That Cannot Be Restarted
RSXMB_DELETE_ARCHIVED_MESSAGES Delete Archived XML Messages
RSXMB_DELETE_MESSAGES Delete XML Messages from the Persistency Layer marked for DEL
RSXMB_CANCEL_MESSAGES Mass cancellation of error messages - XI
RSXMB_SHOW_REORG_STATUS & RSXMB_SHOW_STATUS provide an overview for all XML messages in XI persistency layer.
RSXMB_MESSAGE_STATISTICS2 which provides the processing statistics. This report basically analyzed the history table.
RXWF_XI_UNSED_MSGS report helps in converting messages states other than 003 to final state i.e. 003.
SXMB_REFRESH_ADAPTER_STATUS report helps in converting message sates like 001/008 into final state.
For more detailed information please refer to below SAP notes and URLs
Note 872388 - Troubleshooting Archiving and Deletion
Note 960240 - Messages with adapter type 'PE' are not deleted
Note 1042379 - BPE-HT: Deleting messages that are no longer used
Note 944727 - Status overview of XI messages
URLs of help.sap.com
1) http://help.sap.com/saphelp_nw04s/helpdata/en/f5/d347ddec72274ca2abb0d7682c800b/frameset.htm
2) http://help.sap.com/saphelp_nw04s/helpdata/en/41/b715015ffc11d5b3ea0050da403d6a/frameset.htm
3) http://help.sap.com/saphelp_nw04s/helpdata/en/0e/80553b4d53273de10000000a114084/frameset.htm