How to write each document to a file rather than all documents to a single file

@mohamadelmardini

While it is not possible to individually break the document messages into separate individual files on the same pipeline, I dont understand why you want to increase the I/O resources for lets say 10000 messages.

It is plain and clear that writing so many messages into so many files require that many I/O streams to be opened on the network. It is very process intensive.

I am not getting your concept here. What is the reasoning?

Another way of implementing the Queue, is write the whole message to a database table.

id, message
1, JSON.stringify(jms.message1)
2, JSON.stringify(jms.message2)
3, JSON.stringify(jms.message3)

When you want to process them, you can individually read the message, parse it and do whatever you want to further process it. Hope it makes sense.

Getting back to your original question, you can write separate documents with the JSON-Formatter, XML-Formatter, and DocumentToBinary snaps. You can then configure the FileWriter snap with an expression for the file name, for example:

'out_' + Date.now() + '.json'

Or, if you just want a sequence number, you can use the ‘snap.in.totalCount’ variable:

'out_' + snap.in.totalCount + '.json'

So, every document generated by the formatter snap will be written to its own file with the name computed by the expression.

To configure the formatters:

• JSON Formatter - Select the ‘Format each document’ option.
• XML Formatter - Clear the ‘Root element’ field. Note that the input document should only have a single field that represents the root element.
• Document To Binary - Set ‘Encode or Decode’ to DOCUMENT.

@mohamadelmardini For breaking up each document from JMS queue we are following the same method as they suggested above,do we still having issue breaking up the documents ?also the JMS acknowledge we were able to acknowledge the messages based JMS Message ID as suggested above only thing we didn’t do is having the reject acknowledge .