Snaplogic Last one before record

[sravankunadi]

Identify the record if the entity.success = false then get the one last before record.

For example: if we have 50 records and the last record we got as false, so i want the 49 record value

What i am trying to do here is if the last record entity.success =false then capture the before record entity.nextpageToken

1. If entity.success =false
   

2. if entity.success=true record looks like below and we would like to get the entity.nextpagetoken
   

How can we achieve this in Snaplogic?

[sravankunadi]

Thanks again for your help

I am little bit confused. I have attached the pipeline. i could not figure out the exact logic that you mentioned.
Marketo recursive_2023_01_16.slp (55.6 KB)

2nd mapper in the pipeline:

Router in the pipeline:

3rd mapper could not figure out the logic:

[AleksandarAngel]

@sravankunadi

Basically,
after the Rest Get (Get Lead Changes) Snap, you use the Group By N Snap with group size set to 0, to gather all of the responses in an array.

Once you have all of them in an array, use:
$group.findIndex(x => x.entity.success == false)
expression to get the index of the response from the array where entity.success == false.

This expression will return -1 if there is no response with entity.success == false.

Then by using Router:

So, if we there is no response with entity.success == false, you can just map the nextPageToken from the latest response from the group (the array), by using $group.pop().entity.nextPageToken.

Otherwise, if there is a response with entity.success == false, you can get the nextPageToken from the latest response with entity.success == true from the group (the array) by using the following expression $group[$ind-1].entity.nextPageToken.

Hope, this helps.

BR,
Aleksandar.

[sravankunadi]

Thank you @AleksandarAngelevski

I could achieve whatever i need it, but for the below scenario

So, if we there is no response with entity.success == false , you can just map the nextPageToken from the latest response from the group (the array), by using $group.pop().entity.nextPageToken .

I would like to extract the complete payload and not just one latest response. Is there a way we could get all the records if there is no response with entity.success==false and load the data to target systems

Appreciate your help on this.

[AleksandarAngel]

@sravankunadi

Yes, just split the group using a JSON Splitter instead of using Mapper with $group.pop().entity.nextPageToken after the Router for that route (entity.success == false).

BR,
Aleksandar.

[sravankunadi]

@AleksandarAngelevski

If the pipeline is running for more than 1 hour then we see always that route entity.success ==false and we are missing those records where we had the entity.success == true and it is not helping even if we use the json splitter.

As we are storing everything in the array initially we get the records where entity.success==true and after time that turns to false. In this case the router always goes to false. If the pipeline is running for less than 60 min then the records will load to target. I am not sure when we will receive the entity.success==false it could be anywhere between 1 and 60 min

I have attached the pipeline and some screenshots

Marketo recursive_2023_01_23.slp (357.2 KB)