Hi all,
i’m trying to save a csv file with a file writer and include the number of documents / rows within the name.
I’ve tried giving the csv formatter a binary header (snap.in.totalCount), so i can access the value in the file writer, however it looks like the writer just takes the first value (1).
Is there any simple way of doing this?
Best regards
Thomas
Hi @Henchway,
I don’t know if it is possible to passthrough the last “snap.in.totalCount” number, but I found a workaround for this.
Add a Group by N snap right before you want to write the input documents to a file. (Name the grouped field ‘doc’ for example, with group size = 0 )
Then add a pipeline execute snap with the follwogin parameter:
This will give you the exact number of input documents from the file. And then simply inside the pipeline execute add JSON Splitter to split the grouped docs, CSV Formatter and finally File Writer, but now you can access the _count parameter to get the number of documents from the input file.
