Hi,
I’m trying to use two counters snap.in.totalCount, snap.out.totalCount to compare the total record count before the snap parsed and after parsed like to catch if the count is differ.The above variable gives running counter document like 1,2,3… but i need the variable value “20” records if it has 20 records to use in router for writing the different files.
Like
if $in == $out
output0
else
output1.
can you suggest how can i attain this.
Documents (records) are generally processed through snaps (including the router) in a streaming fashion, independent of each other. To accomplish what you’re looking for, perhaps you could group all the documents together using one of the group by snaps to get a “before” count to copy down a side path. Then split them back apart into separate documents again and do whatever processing. Then afterwards group them back together again to get an “after” count, which you could merge back with the “before” count to compare and take action on. Best if you can leave things streaming though, especially if it might be a lot of documents.
Thanks for response Chris.
is it possible to use script file where i can do the count at two steps. If so can you please provide an example.
Hi Ravi,
There’s a new snap object under flow category called Gate… basically it will collect all the documents and will put into the array, maybe you can use this and count the length of the array created.
Note: Putting a “Gate” snap into your pipeline will only have 1 output, the documents were put in an array, so if you’re streaming those documents individually you have to use the json splitter snap and split the array
