cancel
Showing results for 
Search instead for 
Did you mean: 

Count documents / get value of last document

Henchway
Contributor

Hi all,

i’m trying to save a csv file with a file writer and include the number of documents / rows within the name.

I’ve tried giving the csv formatter a binary header (snap.in.totalCount), so i can access the value in the file writer, however it looks like the writer just takes the first value (1).

image

Is there any simple way of doing this?

Best regards
Thomas

2 REPLIES 2

j_angelevski
Contributor III

Hi @Henchway,
I don’t know if it is possible to passthrough the last “snap.in.totalCount” number, but I found a workaround for this.
Add a Group by N snap right before you want to write the input documents to a file. (Name the grouped field ‘doc’ for example, with group size = 0 )
image
Then add a pipeline execute snap with the follwogin parameter:
image
This will give you the exact number of input documents from the file. And then simply inside the pipeline execute add JSON Splitter to split the grouped docs, CSV Formatter and finally File Writer, but now you can access the _count parameter to get the number of documents from the input file.
image
image
image

Thanks for the suggestion, i’ll give it a try!

Best regards
Thomas