Only execute child pipeline if BigQuery result is not empty

[Colossal8333]

I have a pipeline that starts with a BigQuery Execute (BQE) Snap, followed by several other Snaps.

I only want the step(s) following the BQE Snap to execute if that Snap returns data i.e. if no rows are returned, do nothing, otherwise continue with the rest of the pipeline

[Colossal8333 · Accepted solution]

Figured it out - I can use a Data Validator Snap and set the constraint for one of the column names i.e. Required = true

[SpiroTaleski]

@Colossal8333 

With the Data Validator Snap (and the constraint that you've put), if no rows are returned, the snap (Data validator) will throw an error and the pipeline will fail. - or maybe that's the behavior that you want to achieve. 🙂   

Another option that you can check is Filter Snap. If the filter expression evaluates to true, the document is passed through. If the filter expression evaluates to false, the document is dropped in the stream.

BR, 

Spiro

[Colossal8333]

Yes, sorry could've made that clearer - I only want the pipeline to execute if there is data, because the second phase sends a Slack notification, and I only want that notification sent if the data is present (error logs).

I like the filter recommendation - thank you!