Automating data chunking date file
I am reading a file in a pipeline to break a data load into six-month increments. Twice a year, I need to update the file to accommodate dates for the next six months. My question is this: is ther...
Forum Discussion
omiaye - Please download the attached ZIP file, decompress it, and import the SLP as a new pipeline.
The Sequence snap is pretty self-explanatory. We're just getting the list of years we want records for.
In the Mapper, we create both the Jan and Jul start/end dates as a document with and array containing two objects of start/end dates.
Then we use JSON Splitter so each start/end date is a single document.
Finally, the Filter is used to prevent any future dates (i.e. when we're running in the first 6 months of the year.
As with any solution, there are other ways to solve this, and I'm sure others in the Community can come up with a more concise way to do the same, but I wanted to give you an easily supported and understandable solution.
Hope this helps!
