File writer in Sequence Snap

Hi Good Day,

I was trying to use sequence snap in child pipeline so to split the days sequentially which is passed from parent pipeline . To make it understandable , we have a field called Total_Days and start_date from the parent pipeline . For example, if the start_date is 28/01/2021 and Total_Days is 3 , I need an output of 28/01/2021 , 29/01/2021 , 29/01/2021 . This particular logic I obtained using sequence snap in the child . But my concern here is I was not able to write the result in a file as only the last entries are captured . Is there any way to capture the complete entries obtained from sequence snap in a file writer
Parent Snap details:

image ----Parent main flow

-----Entries in Mapper 4

image ----Pipeline execute entries

Child Snap Details:

----Child Pipeline

Thanks,
Harshavardhana

Please elaborate on what you mean by this.

Hi ptaylor,

Thanks for the heads-up,

I actually mean to say since we use file overwrite in file writer in child pipeline flow , each time the value will be overwritten with the latest value which comes from the parent . So is there any way to capture all the values associated with this child pipeline flow containing sequence snap so that we can write the values to the file once after the execution of this child pipeline .

Thanks

So can I use trying other.options instead of overwrite feature

Sorry, I’m still having trouble understanding what the issue is.

Are you saying that every time the child pipeline runs, it’s overwriting the same file? Then why not just change the file name so that it’s unique per execution? There are many ways to do that, such as appending pipe.startTime or pipe.ruuid to the name.

Also, even if the file is being overwritten every time, why can’t you just look at the latest version written?

You can also just validate the child pipeline by itself to test it by setting the pipeline parameters to default values that test the functionality you want to test.

Hi ptaylor,

I would like to find a way of splitting the value unit and then add the start date accordingly . I will put it simple we have two fields say units and start date in the mapper . We have many values for the units starting from 1 to n and start date with standard value like yyyy-mm-dd . My expectation is we need to split the start date based on the value units . Let me give an example

Say we have unit 2 and start date as 2020-12-28 and I need two entries like 2020-12-28 and 2020-12-29 . This holds for all unit values . So here I am adding the day of the date each by one until we reach the value unit-1 ( add day of start date with unit starting 0 till unit-1)

I did this before using the sequence snap please suggest is there any other effective ways to achieve this so that I can write in a file completely at once.

Thanks

Try this pipeline:

Community9347_2021_01_29.slp (6.2 KB)

It looks like this:

Exactly I was using the same thing before. Thanks for your patience though . Now last assume you are getting multiple values and start date from parent pipeline . This time it becomes impossible to write all the values to the file writer . I shall share the pipeline prent and child I developed in a short while.

Hi Ptaylor,

Please find my Parent and Child pipeline and source file known as Excel Date

Pipelines:

Generator.Holiday files_2021_01_30 (1).slp (11.3 KB) Parent Date Pipeline_2021_01_30.slp (7.3 KB)

Source File:Date Generator.xlsx (18.7 KB)

My Expected Output:

My Obtained Output:

image

May be I am using file overwrite here may be a concern . If we store the same in a oracle table through insert method we could capture all of them . But I need a file writer to this job.

Thanks

The trick is moving the Excel Formatter and File Writer from the child pipeline to the parent pipeline so that they can operate on the cumulative output of the child executions:


Here are the pipelines:
Parent Date Pipeline_2021_02_01.slp (9.6 KB)
Holiday files_2021_02_01.slp (4.1 KB)

Note, this would be simpler (no child pipeline necessary) if the Sequence snap supported setting the “Number of documents” using an expression evaluated against each input document, but unfortunately it doesn’t work that way.

Hi Ptaylor,

Thanks so much for the reply it is working now . I did not think about this approach . Yes as you mentioned sequence snap has to be passed with the number of documents from the source keeping it is a child .