Automating data chunking date file

[omiaye]

I am reading a file in a pipeline to break a data load into six-month increments. Twice a year, I need to update the file to accommodate dates for the next six months. My question is this: is there a way to build this date increment logic into my snap and avoid using this file? Thanks.

Current file format:
Start Date, End Date
2021-01-01, 2021-06-30
2021-07-01, 2021-12-31
2022-01-01, 2022-06-30
2022-07-01, 2022-12-31
2023-01-01, 2023-06-30
2023-07-01, 2023-12-31
2024-01-01, 2024-06-30

[koryknick]

@omiaye - Please download the attached ZIP file, decompress it, and import the SLP as a new pipeline.  

The Sequence snap is pretty self-explanatory.  We're just getting the list of years we want records for.

In the Mapper, we create both the Jan and Jul start/end dates as a document with and array containing two objects of start/end dates.

Then we use JSON Splitter so each start/end date is a single document. 

Finally, the Filter is used to prevent any future dates (i.e. when we're running in the first 6 months of the year.

As with any solution, there are other ways to solve this, and I'm sure others in the Community can come up with a more concise way to do the same, but I wanted to give you an easily supported and understandable solution.

Hope this helps!