โ04-11-2022 10:58 AM
Certain flow snaps like โFilterโ, โHeadโ, โTailโ when used in a complex pipeline doesnโt generate a preview while validating the entire pipeline however these generate a preview if the pipeline is simple.
For instance, if Iโm trying to read data from an excel file with 100k+ records and use either โfilterโ or โhead/tailโ, I can see the preview when the pipeline has been validated but it doesnโt generate one when the pipeline is huge. Any specific reason behind this?
I would want to see the preview as the pipeline that Iโm currently developing is a complex one (multiple snaps prior to โflowโ snaps), thereafter it needs a โfilter + head/tailโ (i.e. flow snaps) and then it will again require a bunch of other snaps after those โfilter + head/tailโ making it complex again.
Help on this matter would be highly appreciated.
Regards,
Darsh
Solved! Go to Solution.
โ04-11-2022 11:22 AM
What @dmiller probably meant, is, that the 50 records validating doesnโt meet the criteria in the filter.
Try increasing the preview count.
Iโm pretty sure thereโs nothing wrong with the Filter snap, and to prove that, you can try and add true
as a filter expression and re-validate. You should see all 50 records passed through.
โ04-11-2022 11:04 AM
Keep in mind that Preview only uses a subset of data (50 records by default), so the data being used my not meet the criteria being used.
An org admin can change the number of records used during preview in Manager > Settings. See https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438287/Data+Preview+from+a+Pipeline#DataP...
โ04-11-2022 11:08 AM
Agreed @dmiller on the 50 records by default on preview.
Preview has been working on all the snaps but for certain it doesnโt so I was curious to know why. There is always a workaround like exporting the data in an excel file right after those โflow snapsโ but it is time consuming so even 50 records by default gives me an idea whether those flow snaps have worked correctly or not.
Head/trim might not require a preview however filter snap does need one; it works well if the pipeline is not complex but as soon as the pipeline grows, filter wouldnโt generate a preview which is kind of weird to me.
โ04-11-2022 11:11 AM
Sharing a snap below for your reference:
Even the successive snaps are not generating any preview which was happening before the flow snaps.
โ04-11-2022 11:15 AM
And here I do have a preview on Filterโฆ
Sometimes I think you need to shift-click Validate to refresh the data.