Identify the record if the entity.success = false then get the one last before record.
For example: if we have 50 records and the last record we got as false, so i want the 49 record value
What i am trying to do here is if the last record entity.success =false then capture the before record entity.nextpageToken
If entity.success =false
if entity.success=true record looks like below and we would like to get the entity.nextpagetoken
How can we achieve this in Snaplogic?
You can try using Group By N Snap with Group Size set to 0 to collect all of the records in an array.
Next, you can use the following expression in a Mapper Snap to get the nextPageToken from the last record where success == true.
$group[$group.findIndex(x => x.entity.success == false) - 1].entity.nextPageToken.
Let me know if this helps you.
Thanks @AleksandarAngelevski for your help. Iam getting the below error
I would like to mention one point here. I do not know when i will receive the entity.success == false it could be anywhere between 1 min and 60 min. Once i receive the entity.success==false then i would like to the flow from that nextpageToken and would like to loop untill all the records are fetched
This would be a case if there is no response with a entity.success == false.
You can separate the process of getting the index and later you can check if the index != -1.
So, If the index != -1 then you can access the latest response with entity.success == true with
$group[index-1]. And if the index == -1 then you can just use
Thanks again for your help
I am little bit confused. I have attached the pipeline. i could not figure out the exact logic that you mentioned.
Marketo recursive_2023_01_16.slp (55.6 KB)
2nd mapper in the pipeline:
Router in the pipeline:
3rd mapper could not figure out the logic:
after the Rest Get (Get Lead Changes) Snap, you use the Group By N Snap with group size set to 0, to gather all of the responses in an array.
Once you have all of them in an array, use:
$group.findIndex(x => x.entity.success == false)
expression to get the index of the response from the array where entity.success == false.
This expression will return -1 if there is no response with entity.success == false.
Then by using Router:
So, if we there is no response with entity.success == false, you can just map the nextPageToken from the latest response from the group (the array), by using
Otherwise, if there is a response with entity.success == false, you can get the nextPageToken from the latest response with entity.success == true from the group (the array) by using the following expression
Hope, this helps.
Thank you @AleksandarAngelevski
I could achieve whatever i need it, but for the below scenario
So, if we there is no response with entity.success == false , you can just map the nextPageToken from the latest response from the group (the array), by using
I would like to extract the complete payload and not just one latest response. Is there a way we could get all the records if there is no response with entity.success==false and load the data to target systems
Appreciate your help on this.
Yes, just split the group using a JSON Splitter instead of using Mapper with
$group.pop().entity.nextPageToken after the Router for that route (entity.success == false).
If the pipeline is running for more than 1 hour then we see always that route entity.success ==false and we are missing those records where we had the entity.success == true and it is not helping even if we use the json splitter.
As we are storing everything in the array initially we get the records where entity.success==true and after time that turns to false. In this case the router always goes to false. If the pipeline is running for less than 60 min then the records will load to target. I am not sure when we will receive the entity.success==false it could be anywhere between 1 and 60 min
I have attached the pipeline and some screenshots
Marketo recursive_2023_01_23.slp (357.2 KB)
any inputs on this please?
Cant you use Head/Tail snaps for this?