Snaplogic Last one before record

Identify the record if the entity.success = false then get the one last before record.

For example: if we have 50 records and the last record we got as false, so i want the 49 record value

What i am trying to do here is if the last record entity.success =false then capture the before record entity.nextpageToken

  1. If entity.success =false

  2. if entity.success=true record looks like below and we would like to get the entity.nextpagetoken

How can we achieve this in Snaplogic?

Hi @sravankunadi,

You can try using Group By N Snap with Group Size set to 0 to collect all of the records in an array.

Next, you can use the following expression in a Mapper Snap to get the nextPageToken from the last record where success == true.

$group[$group.findIndex(x => x.entity.success == false) - 1].entity.nextPageToken.

Let me know if this helps you.

BR,
Aleksandar.

3 Likes

Thanks @AleksandarAngelevski for your help. Iam getting the below error

I would like to mention one point here. I do not know when i will receive the entity.success == false it could be anywhere between 1 min and 60 min. Once i receive the entity.success==false then i would like to the flow from that nextpageToken and would like to loop untill all the records are fetched

Hi @sravankunadi,

This would be a case if there is no response with a entity.success == false.

You can separate the process of getting the index and later you can check if the index != -1.

So, If the index != -1 then you can access the latest response with entity.success == true with $group[index-1]. And if the index == -1 then you can just use $group.pop().

BR,
Aleksandar.

1 Like

Thanks again for your help

I am little bit confused. I have attached the pipeline. i could not figure out the exact logic that you mentioned.
Marketo recursive_2023_01_16.slp (55.6 KB)

2nd mapper in the pipeline:

Router in the pipeline:

3rd mapper could not figure out the logic:

@sravankunadi

Basically,
after the Rest Get (Get Lead Changes) Snap, you use the Group By N Snap with group size set to 0, to gather all of the responses in an array.

Once you have all of them in an array, use:
$group.findIndex(x => x.entity.success == false)
expression to get the index of the response from the array where entity.success == false.

This expression will return -1 if there is no response with entity.success == false.

Then by using Router:

image

So, if we there is no response with entity.success == false, you can just map the nextPageToken from the latest response from the group (the array), by using $group.pop().entity.nextPageToken.

Otherwise, if there is a response with entity.success == false, you can get the nextPageToken from the latest response with entity.success == true from the group (the array) by using the following expression $group[$ind-1].entity.nextPageToken.

Hope, this helps.

BR,
Aleksandar.

1 Like

Thank you @AleksandarAngelevski

I could achieve whatever i need it, but for the below scenario

So, if we there is no response with entity.success == false , you can just map the nextPageToken from the latest response from the group (the array), by using $group.pop().entity.nextPageToken .

I would like to extract the complete payload and not just one latest response. Is there a way we could get all the records if there is no response with entity.success==false and load the data to target systems

Appreciate your help on this.

@sravankunadi

Yes, just split the group using a JSON Splitter instead of using Mapper with $group.pop().entity.nextPageToken after the Router for that route (entity.success == false).

image

BR,
Aleksandar.

1 Like

@AleksandarAngelevski

If the pipeline is running for more than 1 hour then we see always that route entity.success ==false and we are missing those records where we had the entity.success == true and it is not helping even if we use the json splitter.

As we are storing everything in the array initially we get the records where entity.success==true and after time that turns to false. In this case the router always goes to false. If the pipeline is running for less than 60 min then the records will load to target. I am not sure when we will receive the entity.success==false it could be anywhere between 1 and 60 min

I have attached the pipeline and some screenshots

Marketo recursive_2023_01_23.slp (357.2 KB)

any inputs on this please?

Cant you use Head/Tail snaps for this?