How to pass a variable along the pipeline?
I have a pipeline that looks like this:
The Snaps are:
List all files in a given Box directory
Filter the files according to a mask
Read the matching Excel file into the pipeline
Con...
Forum Discussion
I’m just playing with regex ideas for this (because who doesn’t like to - regex is like play doh! 😄), I think this will work for a mapper if you know it is a single delimiter being a space or a dash:
Date.parse($RPT_MONTH.replace(/^([^ -]+).*(\d{2})(\d{2})$/,"$1-$2$3"),'MMM-yy').toLocaleDateString({"format":"MMM yyyy"})
But you could cover your bases in case the delimiter is not a space or dash or no delimiter at all (ex. Aug.2022 or Aug22) with either one of these two:
Date.parse($RPT_MONTH.replace(/^([^\d]{3}).*(\d{2})(\d{2})$/,"$1-$2$3"),'MMM-yy').toLocaleDateString({"format":"MMM yyyy"})
Date.parse($RPT_MONTH.replace(/^([a-zA-Z]+).*(\d{2})(\d{2})$/,"$1-$2$3"),'MMM-yy').toLocaleDateString({"format":"MMM yyyy"})
Disclaimer: testing works for me with the values of the original post and a few I threw in of my own. All potential cases are not fully tested.
Follow-up to my post: In my testing, I found that Date.parse() format of “MMM-yy” works correctly with both 2-digit and 4-digit years, so it may not be necessary to differentiate in your logic.
Correct, there is no way to pass a variable “up”. To elaborate on that a bit, the snaps all run in parallel, so there would be a race between the snap that is passing the variable “up” and the snap that is trying to read the variable. For example, if there were two files coming in, the Write snap might see the first file name in one execution and the second file name in another execution.
I can see how that would happen, but could that not be controlled with a ForEach Snap that serializes the processing?
The use case you are thinking of would require a child pipeline due to the race condition, however it is not a requirement for the pipeline I am developing as (a) there will only be one file, and (b) it is acceptable to stop processing and throw an error if there is more than one file.
Hmm, I’m not quite sure what you mean here. The PipeExec snap basically obsoletes the ForEach snap.
That’s not going to be true in the general case. So, we can’t add something so error prone and hope that people only use it in the right situations.
