Ultra task with Salesforce Read snap: merge output into single document

I have a question similar to the one asked here Merge documents into one big document

Specifically, I have a pipeline with a Salesforce Read Snap which returns 0 or more documents as the result. I’d like the output of the Read Snap to contain a single document containing an array with results returned by the Read Snap.

Based on my reading of the forum posting:

  • I cannot use the Group By N snap as it isn’t supported in Ultra tasks
  • It is possible to use the script snap in an Ultra snap to do this.

I’ve attached a pipeline that can do this:
SFReadUltraPipeline.slp (13.6 KB)

The pipeline returns the following output from a triggered (non-ultra task):

[
    {
        "theOriginal": {
            "Id": "0036A00000Vatj6QAB",
            "FirstName": "Andrew",
            "LastName": "Ramirez",
            "hed__AlternateEmail__c": "aramirez@destinsolutions.com",
            "original": {
                "FirstName": "Andrew",
                "Email": "aramirez@destinysolutions.com"
            }
        },
        "duplicates": [
            {
                "Id": "0036A00000AbbdpQAB",
                "FirstName": "Andrew",
                "LastName": "Ramirez",
                "hed__AlternateEmail__c": "aramirez@destinysolutions.com",
                "original": {
                    "FirstName": "Andrew",
                    "Email": "aramirez@destinysolutions.com"
                }
            },
            {
                "Id": "0036A00000AbbeOQAR",
                "FirstName": "Andrew",
                "LastName": "Mayzak",
                "hed__AlternateEmail__c": "4541474710648498@destinysolutions.com",
                "original": {
                    "FirstName": "Andrew",
                    "Email": "aramirez@destinysolutions.com"
                }
            },
            {
                "Id": "0036A00000Vatj6QAB",
                "FirstName": "Andrew",
                "LastName": "Ramirez",
                "hed__AlternateEmail__c": "aramirez@destinsolutions.com",
                "original": {
                    "FirstName": "Andrew",
                    "Email": "aramirez@destinysolutions.com"
                }
            }
        ]
    }
]    

with the dashboard view demonstrating that documents are merged correctly:

A request sent to an ultra task based on the pipeline times out, with the dashboard indicating the script snap never returns:

I suspect that the issue is that document lineage is not being maintained by the script snap. However, I don’t know how to maintain document lineage and also get the snap to output a single document with an array containing the results of the Salesforce Read Snap.

Can someone tell me how to alter this pipeline to make it ultra compatible while still returning a single output document containing the results of the Salesforce Read snap?

@omair

I am also facing same issue with NS. I have tried different option (child pipeline,task execution etc). Looks like ultra is only returning single value not any response array. I found in ultra no batch mode is supported. But we need a confirmation from Snap team. It will be great if you found something and post it on this forum.

@nmaniyar

Hi Nidhi - we are seeking your help on this.

  1. Is ultra can send response as array/ multiple document ?
    2.Is ultra can be deployed at cloud? My understanding about ultra is- when third party trying to access internal system with high volume data set then we can design ultra. is there any way through snaplogic UI portal we can whitelist IP for external application?