The third part of the series on Integrating SAP using the SAP IDoc Snaps will discuss the IDoc Listener and the IDoc Document Listener Snap and touch on the SAP IDoc Read Snap. We will revisit the prerequisites already discussed in part 1 of the series, show you where to configure the connection inside the SAP Gateway, test it, and give you an insight into the various transactions inside SAP that allow you to send out IDocs. Lastly, we will use the IDoc Write Snap again to send back a status IDoc used by the SAP system to change the status of an IDoc that we initially received through the IDoc Listener Snaps. The status IDoc will allow us to integrate with the monitoring systems for IDoc inside the SAP system and report successes or failures back to SAP.
Revisiting the pre-requisites to use IDoc Listener Snaps
We discussed the requirements in part 1 of the series already but we wanted to remind everyone here quickly about them. Both parts, adding service names to the /etc/services file on every node of your Groundplex as well as ensuring that the Access Control Lists on the SAP Gateway allow the GroundPlex to connect to the SAP Gateway are essential to make both, the IDoc Listener Snap as well as the IDoc Document Listener Snap work. Please have a look at the Prerequisites for IDoc Listener and IDoc Document Listener Snaps section of part 1 of the series.
Understanding the IDoc Listener and Document Listener Snaps
Either one of the SAP IDoc Listener Snaps listens for incoming IDocs by registering at the SAP Gateway. That means pipelines built with the Snaps will not finish but run continuously until a user stops them. To register at the gateway, you must specify the Gateway Host, Gateway Service (translating to the port the gateway runs on), and a Program ID that uniquely identifies the Snap on the SAP Gateway. If your SAP system is behind an SAP router, you must also specify the SAP router string. The team operating your SAP systems can give you the host, gateway service name, and the SAP router string and define a unique name for the Snap to use as the Program ID.
While every SAP System has a Gateway built in, you can also set up a stand-alone Gateway close to your Groundplex nodes. SAP stand-alone Gateways are typically used in high-availability setups where an SAP system connects to a third-party application like our Snaps.
On the SAP side you must use transaction SM59 to create a RFC destination that will be used by the SAP ALE layer to send data to the Snap in your pipeline. The RFC destination must be of type T - TCP/IP Connection and use a Registered Server Program with the same Program ID that you specified in the Snap Settings.
When you Execute the pipeline and navigate to the Dashboard, you will see it running. You can use the Status to open the Pipeline Execution Statistics and the number of Documents to verify that IDocs are being sent from the SAP system to the pipeline.
Once the pipeline runs, you can use the Connection Test button in transaction SM59 to test the connection and also see your connection registered in transaction SMGW. The Program ID can be seen using the Goto -> Logged on Clients menu entry in SMGW.
A tip on retrieving the IDoc structure from the IDoc Listener Snap
As the IDoc Listener Snap can receive any IDoc from the SAP system and forward it in binary format, building downstream pipelines will depend on your understanding of the IDoc(s) structure. One trick we often use to understand the structure of the IDocs is to attach an HTTP Client Snap with a binary input directly to the SAP IDoc Listener Snap output and forward the content to an API mock service that displays the received request.
When doing that, you will get the XML document below for the demo IDoc that we use throughout the blog series. You can then use this information to build your pipeline.
<?xml version="1.1"?>
<ZPERSON01>
<IDOC BEGIN="1">
<EDI_DC40 SEGMENT="1">
<TABNAM>EDI_DC40</TABNAM>
<MANDT>001</MANDT>
<DOCNUM>0000000000007004</DOCNUM>
<DOCREL>754</DOCREL>
<STATUS>30</STATUS>
<DIRECT>1</DIRECT>
<OUTMOD>2</OUTMOD>
<IDOCTYP>ZPERSON01</IDOCTYP>
<MESTYP>ZPERSONDATA</MESTYP>
<SNDPOR>SAPA4H</SNDPOR>
<SNDPRT>LS</SNDPRT>
<SNDPRN>A4HCLNT001</SNDPRN>
<RCVPOR>A000000002</RCVPOR>
<RCVPRT>LS</RCVPRT>
<RCVPRN>SNPCLNT000</RCVPRN>
<CREDAT>20240528</CREDAT>
<CRETIM>231206</CRETIM>
<SERIAL>20240528231206</SERIAL>
</EDI_DC40>
<ZPERSONDATA01 SEGMENT="1">
<FIRSTNAME>Elmo</FIRSTNAME>
<LASTNAME>Monster</LASTNAME>
<TITLE>Monster</TITLE>
<GENDER>male</GENDER>
<STREET>123 Sesame Street</STREET>
<CITY>New York City</CITY>
<ZIP>10023</ZIP>
<COUNTRY>United States</COUNTRY>
<EMAIL>elmo@sesame.org</EMAIL>
<PHONE>+12125953456</PHONE>
</ZPERSONDATA01>
</IDOC>
</ZPERSON01>
The SAP IDoc Document Listener Snap
Other than the IDoc Listener Snap, the IDoc Document Listener Snap will allow you to specify the IDoc Basic Type in the Settings tab of the Snap. Doing so will allow the Snap to retrieve the IDoc structure during validation from the SAP system and allow you to use it in a downstream Snap like a Mapper Snap. If you only ever intend to receive one Basic IDoc Type into the pipeline this is very helpful as standard SAP IDocs are typically very large and contain many segments. Being able to get the structured passed down makes the configuration of the pipeline much simpler. The IDoc Document Listener Snap will however not accept and process different IDoc Basic Types very well in comparison to the IDoc Listener Snap.
The attached sap-idoc-document-listener-to-http pipline makes use of the IDoc Document Listener Snap and simply maps the output to an HTTP Client Snap that is configured to post the IDoc to an HTTP endpoint.
After Validating the pipeline you can see the downstream structure of the IDOC appear in the Mapper Snap and use it to make the mapping you need.
The HTTP Endpoint used in this Snap would receive a JSON document in the body of the HTTP POST request that would look like the below JSON structure.
{
"IDOC":{
"@BEGIN":"1",
"EDI_DC40":{
"@SEGMENT":"1",
"TABNAM":"EDI_DC40",
"MANDT":"001",
"DOCNUM":"0000000000007020",
"DOCREL":"754",
"STATUS":"30",
"DIRECT":"1",
"OUTMOD":"2",
"IDOCTYP":"ZPERSON01",
"MESTYP":"ZPERSONDATA",
"SNDPOR":"SAPA4H",
"SNDPRT":"LS",
"SNDPRN":"A4HCLNT001",
"RCVPOR":"A000000002",
"RCVPRT":"LS",
"RCVPRN":"SNPCLNT000",
"CREDAT":"20240529",
"CRETIM":"002032",
"SERIAL":"20240529002032"
},
"ZPERSONDATA01":{
"@SEGMENT":"1",
"FIRSTNAME":"Elmo",
"LASTNAME":"Monster",
"TITLE":"Monster",
"GENDER":"male",
"STREET":"123 Sesame Street",
"CITY":"New York City",
"ZIP":"10023",
"COUNTRY":"United States",
"EMAIL":"elmo@sesame.org",
"PHONE":"+12125953456"
}
}
}
Enabling Trace files
All SAP Snaps that make use of the SAP JCo library allow you to enable the SAP JCo trace by selecting the checkbox in Enable Trace in the configuration. This is shown in the configuration screenshot of the IDoc Listener Snap above. Once enabled the SAP JCo will write a trace file for the communication between the SAP system and the GroundPlex to /opt/snaplogic on the node that executes the connection. The files start with JCO and end in .trc. You can configure the level in the connection by using the jco.trace_level configuration parameter with a value between 0 and 10.
What can I do with the IDoc Read Snap
With the IDoc Read Snap, you can read an existing IDoc in Snaplogic pipelines. This can be useful if you build pipelines that create IDocs configured in SAP that are not to be processed immediately but through a scheduled background job running later. In that case, the IDoc will, in the event of a successfully created IDoc, receive status code 64 - IDoc ready to be passed to application directly after creating the IDoc. You might then create a second pipeline that might run periodically to check if the status has changed in SAP and if it has proceeded with further data processing. You could also use the IDoc Read Snap to retrieve all statuses that an IDoc has passed and the timestamps for when the changes happened. They are contained in the STATUS field of the objects in the INT_EDIDS array as shown below. The IDOC_CONTROL object also includes a STATUS field with the current status at the time of writing identical to the last object in the INT_EDIDS array.
Optimizing the Function Module that reads the IDOC in SAP
By optimizing the standard IDOC_READ_COMPLETELY function module, which we used as the foundation for our RFC-enabled custom version in part 1 of this blog, you can significantly enhance performance. This can be achieved by modifying the function module to return only the control record and excluding the data and/or status records. This allows you to create three distinct function modules that can be used interchangeably in the IDoc Read and the IDoc Write Snaps, depending on the specific information you need from the SAP system. The three function modules can use the same ABAP code, but only differ in name and their table selections (see example below).
* read data records - if requested
IF INT_EDIDD IS REQUESTED.
...
ENDIF.
* read status_records - if requested
IF INT_EDIDS IS REQUESTED.
...
ENDIF
To do this, remove either one or both tables shown in the screenshot below from your function module and the interface calling the IDOC_READ_COMPLETELY function module inside your module. For example, you might create these three function modules.
- ZIDOC_READ_COMPLETELY (includes everything just like IDOC_READ_COMPLETELY)
- ZIDOC_READ_COMPLETELY_C (includes no tables only the control structure is returned)
- ZIDOC_READ_COMPLETELY_CS (includes control structure and the INT_EDIDS table)
Sending back status information to SAP for an existing IDoc
In the last part of the blog, we will go full circle with the processing of IDocs by looking at changing the status of a created IDoc in SAP from a SnapLogic pipeline. Once an IDoc is successfully passed over to a SnapLogic pipeline, the SAP system changes the status of the IDoc from status 30 - IDoc ready for dispatch to 03 - IDoc Data passed to port OK. From an SAP point of view, this means that the SAP system successfully handed the IDoc over to the partner configured to receive the IDoc. What happened to the IDoc after that is, at that point, unknown to the SAP system. If you are using IDoc monitoring, the tools used would never be able to tell what happened to the data that was sent via the IDoc. Leaving the IDocs in status 03 is, therefore, typically an unsatisfactory state as you would naturally want to understand if the data has reached its destination successfully or if something went wrong along the way to be able to possibly trigger alerts or take other actions to resolve the missing information.
To change the status of the IDoc in SAP you have to send a newly constructed SYSTAT01 IDoc of Message Type STATUS back to the SAP system.
We attached a sample pipeline below that receives our ZPERSON01 IDoc we used throughout the blog, writes the content to a MSSQL Server and then continues to construct a new SYSSTAT01 IDoc containing status code 12 - Dispatch OK if the database write was successful and 11 - Error during dispatch if it failed.
The JSON below contains the fields relevant for changing the status of the IDoc, such as the DOCNUM containing the IDoc number of the IDoc to change the STATUS, text information in the field STATXT, as well as the date and time the change occurred.
{
"EDI_DC40": {
"MESTYP" : "STATUS",
"SNDPRT" : "LS",
"SNDPRN" : "SNPCLNT000",
"SNDPOR" : "A000000002",
"RCVPRT" : "LS",
"RCVPRN" : "A4HCLNT001"
},
"E1STATS": [
{
"DOCNUM": "1009",
"STATUS": "12",
"STATXT": "Message Processessed Successfully in Snaplogic",
"STAPA4": "status",
"LOGDAT": "20240307",
"LOGTIM": "000000"
}
]
}
To successfully receive the Message Type, your ALE configuration in SAP must contain the STATUS Message Type as an incoming message type processed by Process Code STA1, as shown in transaction WE20 blow.
In the last part of our blog series on SAP IDoc Snaps, we will look at securing the communication between SAP and SnapLogic by configuring Secure Network Communication (SNC). This configuration is technically not part of IDoc processing but is used by any SAP Snap that communicates with the SAP system via the SAP JCo Libraries. It is also relevant for the SAP Snaps executing RFC, such as the SAP Execute and SAP RFC Listener Snaps.
