Data preview and schema suggest in child pipelines

Designing Pipelines
#1

I have a child pipeline that opens with a JSON Formatter that utilizes a *.schema file. This works very well for schema propagation up to the parent, however it doesn’t seem to do much when it comes to designing the child pipeline itself. It seems the lack of expected incoming documents in validation prevents the data preview and schema suggestions from being accessible when designing/maintaining the child pipeline. Note that my *.schema file does include example data, but validation doesn’t seem to utilize it.

The best solution I’ve come up with is to start the child pipeline with a JSON Generator with some sample data for validation/design, and then disable it (and its connection) when it needs to utilized as child. But this is obviously less than idea - one has to maintain that sample data, and if one forgets to disable it, it breaks its usage as a child.

  1. Is there a better way to validate child pipelines with an open document input such that data preview and schema suggest are available in design? My hope would be that that could propagate down from the parent validation, but that doesn’t seem to be the case.

  2. Is there a better way to receive strongly typed documents into a child pipeline than utilizing a JSON Formatter with *.schema file?

image

Thanks, Ryan

#2

@rrogers, for question #1, you might look at the Record Replay snap. You can use it to capture your input data from the parent and then use it to replay that data during validation at design time.

#3

Thank you very much! That seems to do exactly what I’m looking for. I even discovered that it doesn’t have to be the first snap in the child pipeline, so that I can keep the JSON Formatter in place for schema propagation back up to the parent (assuming that is the best way to accomplish that - question #2).

image
image753×119 19 KB