No luck with that approach :). Will have to just keep at it.
So did you come up with a decent solution?
BTW, any explanation for why SnapLogic doesn’t support a simple loop flow?
I’ve given my boss, who is our SnapLogic admin, information needed for an enhancement request. I am new to SnapLogic, so I hesitate to critique. I’d just say that every other ETL tool and every other usable scripting or coding language I’ve used implements loops. So it’s a shocker that SL doesn’t have this - or that you have to use nested pipelines. So I wholeheartedly encourage them to look into this.
It’s very useful to have a visual that implements a loop and shows you, nicely, what the exit condition is and what variables/fields are getting modified as one iterates. It’s possible to use recursion (pipelines). But given that we can’t click through to child pipelines or see easily what’s getting passed to them (setup of paramaters is very clicky, on both the parent and child ends) - given all this, a loop construct is really needed.
It’s also important to improve recursion and make those parameters more visible. Having to click around the parent, and the child, and having to manually ensure that everything matches, is really hard to do. And recursion is difficult for the citizen-integrators that SL targets. It’s a hard concept. If SL wants to make this easy, a simple loop will help - along with visibility improvements, as noted.
I was never able to get looping to work. Even with the “callback” part of the loop separated out to a separate pipeline and using Pipeline Execute, it appears that Snaplogic can look ahead and detect if the logic in the datastream will create a potential loop…and refuses to enter the loop from the entry point (that would be looped back TO). I was able to get delay working via the suggested approach with a little Ruby script though.
We have not set it up yet but I anticipate that we will end up closing the loop logically by using a REST interface.
