Forum Discussion

mnyolt's avatar
mnyolt
New Contributor
4 years ago

Which pattern for recursive/DFS traversal of tree?

I’m quite new to SnapLogic and struggling to create a pipeline which traverses a tree structure.

The use-case is that as an input, I get a tree-structure of items, and want to replicate this in another system.
For creating sub-items, I first need to create the parent item and specify its Id (within the target system) during the create of the sub-items. Thus a DFS traversal is essential.

I tried to implement a basic traversal like this:

The JSON Generator creates a simple test document:

[
    {
        "a": 0,
        "children": [
            { "a": 1 }
        ]
    }
]

The splitter reads the children property and feeds the children back to the union.
The union serves as the “loop” and combines the original document with any nested child documents.
The filter is configured with $.get('children') != null and ensures that the splitter does not create empty documents.

The logic of the pipeline should be sound, since at some point no children are left, and no more documents are created by the splitter.
But I get an StackOverflowError.
(Which surprises me, since outputViews just write and inputViews just poll - if connected properly, no infinite recursion should happen - it’s basically just streams connected, right?)

It seems this is not the way to traverse such a tree structure.
I also read that recursive pipeline execution is not supported.

So, what is the usual pattern of traversing such a tree structure recursively in a DFS manner?

  • darshthakkar's avatar
    darshthakkar
    Valued Contributor

    Thanks @robin, I haven’t had a chance to test the solution you provided but I’m sure it would have worked.
    I went with a different route for my development so the concern I was having has been resolved.

    XML Generator does intake dynamic values if they have been binded properly prior to it (using Aggregate or Group By), the only challenge I found was ingesting native html tags because <a href ="$URL"> wasn’t generating the desired output as the tags gets converted to snapLogic’s predefined symbols like < = &lt;,so on and so forth. Check these links for detailed explanation on tags conversion:

    Link 1
    Link 2

    What I found and worked for me was defining the html format (if you’re trying to generate html files; could be any format you’re interested into) in the mapper itself/file writer and then let the format do its job.

    Thanking @robin for his suggestions, time and help on this. Signing off from this thread.