How to write each document to a file rather than all documents to a single file

[mohamadelmardin]

I am reading multiple messages using JMS consumer into one big binary file then using a binary to document snap I am breaking it into multiple documents. How can I write each document to a separate file with timestamp rather than writing all the documents at the end to one file?

I know there is a way to do that using ForEach pipeline where you can send each document seperately but I don’t want to use a master pipeline and a child pipeline. I want all operations to be done in one single pipeline.

[mohamadelmardin]

@Bhavin
Thank you this what I ended up doing and the closest thing to my solution. I have one clarification though regarding using two JMS Acknowledge snaps (Acknowledge, Reject) instead of just one. Isn’t the other JMS Acknowledge snap for ‘Reject’ redundant? Why cannot we use just one Acknowledge snap and if the message processed successfully then it will be acknowledged but the message wasn’t processed and didn’t make it till the end then the message won’t be acknowledged therefore it will remain in the queue for future processing. I don’t see what difference would it make to add a Reject snap with Recover message box checked since the first Acknowledge snap can cover both scenario.

Regarding your request to share the JMS account configuration I will be more than happy to share it below 🙂

[Bhavin]

Isn’t the other JMS Acknowledge snap for ‘Reject’ redundant?
yes, mine was geared more towards “asthetics” 🙂

Regarding your request to share the JMS account configuration I will be more than happy to share it below
much obliged

[tstack]

mohamadelmardini:

How can I write each document to a separate file with timestamp rather than writing all the documents at the end to one file?

Getting back to your original question, you can write separate documents with the JSON-Formatter, XML-Formatter, and DocumentToBinary snaps. You can then configure the FileWriter snap with an expression for the file name, for example:

'out_' + Date.now() + '.json'

Or, if you just want a sequence number, you can use the ‘snap.in.totalCount’ variable:

'out_' + snap.in.totalCount + '.json'

So, every document generated by the formatter snap will be written to its own file with the name computed by the expression.

To configure the formatters:

• JSON Formatter - Select the ‘Format each document’ option.
• XML Formatter - Clear the ‘Root element’ field. Note that the input document should only have a single field that represents the root element.
• Document To Binary - Set ‘Encode or Decode’ to DOCUMENT.

[sandeepkasaram]

mohamadelmardini:

essages using JMS consumer into one big binary file then using a binary to document snap I am breaking it into multiple documents. How can I write each document to a separate file with timestamp rather than writing all the documents at the end to one file?

I know there is a way to do that using ForEach pipeline where you can send each document seperately but I

@mohamadelmardini For breaking up each document from JMS queue we are following the same method as they suggested above,do we still having issue breaking up the documents ?also the JMS acknowledge we were able to acknowledge the messages based JMS Message ID as suggested above only thing we didn’t do is having the reject acknowledge .

[mohamadelmardin]

Sandeep the original solution wasn’t working when I tested it out. I updated the approach and it is working now. You can check it out at DEV environment.

All is good now.