Day 5 was the first day that really becomes difficult on SnapLogic, and it’s mostly because you need to loop with saving state from each iteration in order to process the individual moves. I also had to do this with the creation of the initial data configuration which required its own loop with state being preserved. Now, I used the same file for both of these, so it’s probably not a huge deal, but this is one of the situations where you effectively have to create a pipeline that reads from a file, and writes to that same file to obtain and preserve state between iterations (via input documents).
I think some callouts here are that when doing my internal pipeline testing, I used an internal json generator, because state is/was preserved in a spot different from the pipeline itself, this made it somewhat hard to do, but allowed me to change values as necessary, when it came time for integrating the pipeline, I’d just disconnect and disable the JSON generator and run the pipelines as required. Seems it might be nice to have some sort of way to debug that more readily, I know we have the replay snap, but I found it didn’t really work, because unlike a normal pipeline, I have to wait for full pipeline executions to run before I run another round of it, so that became somewhat more cumbersome than just testing a single instance, etc.
I think maybe the big callout here is using local (to snapplex) files rather than files in sldb or on the web somewhere, while it SEEMS fast to use those files, the time to access a file local to the plex is much faster. This might be one of those cases where it would be nice to have like a plex-centric internal cache to be able to leverage specifically in order to deal with state-based looping within SnapLogic, being able to set that state prior to a pipe exec (or as maybe part of the pipe exec?) and then synchronizing the output of the pipe exec to read the final output at that time. This form of looping becomes VERY cumbersome, but going through it now means it should be easier in the future?
I also found that parsing the moves "move x from y to z" was a two-snap process using regex, the first to get the regex output and the second to map the output appropriately. I’m not sure how common this type of parsing is used, but it might be something where it’s worth potentially putting some sort of document-based parser logic into a snap to make this kind of parsing easier in the future.
Screenshots
SLP Files:
Main Pipeline:
day05_2022_12_05.slp (78.3 KB)
Puzzle Generation pipeline:
day05_generator_2022_12_05.slp (13.6 KB)
Puzzle Processing pipeline:
day05_processor_2022_12_05.slp (16.8 KB)
