04-11-2022 10:58 AM
Certain flow snaps like “Filter”, “Head”, “Tail” when used in a complex pipeline doesn’t generate a preview while validating the entire pipeline however these generate a preview if the pipeline is simple.
For instance, if I’m trying to read data from an excel file with 100k+ records and use either ‘filter’ or ‘head/tail’, I can see the preview when the pipeline has been validated but it doesn’t generate one when the pipeline is huge. Any specific reason behind this?
I would want to see the preview as the pipeline that I’m currently developing is a complex one (multiple snaps prior to “flow” snaps), thereafter it needs a ‘filter + head/tail’ (i.e. flow snaps) and then it will again require a bunch of other snaps after those ‘filter + head/tail’ making it complex again.
Help on this matter would be highly appreciated.
Regards,
Darsh
Solved! Go to Solution.
04-11-2022 11:22 AM
What @dmiller probably meant, is, that the 50 records validating doesn’t meet the criteria in the filter.
Try increasing the preview count.
I’m pretty sure there’s nothing wrong with the Filter snap, and to prove that, you can try and add true
as a filter expression and re-validate. You should see all 50 records passed through.
04-12-2022 10:38 AM
Where did you put the condition as TRUE?
04-12-2022 10:55 AM
@amit.saroha: Condition = True was kept in the filter snap. If you’re following the thread, the snap after finalOutput is the filter snap (that’s where the condition was put).
04-11-2022 12:36 PM
Also, you can always change the number of records the platform processes during validation: