Simple document viewer during testing

[dwhansen-cbg]

I was wondering if there is a simple way of viewing documents made it down a certain path during testing of a pipeline. I know that I can view how many documents flow in and out of a Snap using the statistics viewer, but I’d like to actually see the documents. Is there a quick file writer or something like that that could help? Is it possible to stop a pipeline at a certain step for debugging purposes?

[dmiller]

Have you used the preview feature? When you validate a pipeline, the circle between two Snaps gets a document icon; click that to preview the data that is passing through to the other Snap.

[dwhansen-cbg]

Yes, but unfortunately those don’t work consistently throughout my pipelines. Sometimes they stop at Join snaps, sometimes after inserts, sometimes at other times. I just tried a simple CSV formatter with a File Writer and that seemed to work. Ideally the preview feature would always work, but it doesn’t.

[dmiller · Accepted solution]

It works on a subset of data (default is 50 documents, but configurable to some degree by the admin), so if the data is changing, what displays in preview will change as well.

As for stopping a pipeline, you should be able to edit a Snap and set Snap execution to Disabled which would stop the pipeline at that Snap (assuming a linear pipeline).

[dwhansen-cbg]

@dmiller how can I change the default of 50 documents? I’ve looked on the Manager Dashboard and haven’t seen anything there.

If i disable a snap in the middle of a pipeline, do I need to disable subsequent snaps, too?