rriley99
2 years agoNew Contributor III
For each entry run a pipeline
I want to create a "load" pipeline that will ingest a json with the pipeline name and the parameters to pass to the pipeline. So for example I'd start with a json like this:
{
"targets": [
{
"redshift": {
"pipeline_name": "redshift_pipeline",
"connection_string": "foo"
}
},
{
"s3": {
"pipeline_name": "s3_pipeline",
"bucket": "bar"
}
}
]
}
And for each item in that list I would want to pass the values in each dict to the "EXECUTE PIPELINE" Snap.
So my process has been
1. build the iterative item (json)
2. Somehow parse and iterate
3. Use the execute pipeline snap.