Using Azure Storage/Azure Blob Storage
I am trying to create an Azure Blob Reader but I keep encountering this error:
Error: Directory not found or access denied: wasb:///storage@janoswsl.blob.core.windows.net/; Reason: com.snaplogic....
Forum Discussion
Can I ask why they need to be run exclusively and cannot be run in parallel? Are they accessing some common resource?
There’s no explicit support for this in the PipelineExecute snap. The simplest option would be to put the two PipeExec snaps in their own pipeline and have the parent run that new pipeline. The root will wait for the new middle pipeline to finish before running the next one and the middle will wait for the two children to finish before it returns.
Welcome @ausman - you are correct that you cannot “save” a value back to pipeline parameters, but there are a some easy ways to do what you are looking to do.
One simple method is to have a main pipeline that reads from the SQS queue then calls a child pipeline to do the work on that file. Then the main pipeline can use the $original object value that returns from the Pipeline Execute snap to get the object that was originally passed into the child pipeline.
Another way (which in my opinion is a little more cumbersome) is to use the Copy snap after the SQS read and a Join snap using the Merge merge type after you have completed the work on the file so you can re-use that original value that was returned from the SQS queue.
I’m sure some other clever folks could give you other ways to solve the same issue, but these are pretty simple to understand and implement.
Hope this helps!
