I am in need of pipeline(s) querying close to 20 tables (may increase in future) and consolidate and write as flat json files in the snaplogic server which will be used by different pipelines for lookup data during processing.
The idea is to use only one or as minimal connection during the execution. Hence I am not going for that simple solution of having 20 SQL Selects and consolidate and write the file.
So, I designed a child pipeline which will accept table name, required columns (need only 4 column names from all the 20 tables) as parameters and use the same in the SQL select snap to configure the table name and the output fields. The output may be n number of rows. I need to group all of them in to one document. So I thought of using Group by N snap. But the issue is since we cant set the Target Field in Group by N field dynamically, I will not be able to differentiate the table records. Is there a way to set the Target Field Dynamically from either pipeline parameter or from input document?
I am moving this to enhancement tab as this may be a useful feature to have the value dynamically populated in the Target Field of Group by N snap.
Thanks and regards,