Order of Execution Not Right

Hello,

A pipleine I’m working on seems to attempt to execute the Upsert Content (2) snap before the Create Deal/Job snap is executed. Since the Content record is a child of the Deal/Job, the absence of the Deal/Job Id is causing the Upsert Content (2) to fail.

I’ve tried to route the documents down the one branch but the pipeline fails because Salesforce (the endpoint), doesn’t recognise the additional variables in the Map Deal/Job snap if the Pass Through option is selected.

I’m not sure how to best approach this.
Please note my understanding of SnapLogic is beginner level and I’m not a developer by trade.

What is the Join type?

Just seen this on the Join Snap article here:

Known Issue : When the upstream Snaps of the Join Snap contains Copy, Router, Aggregate, or similar Snaps, it is likely that the data flow of a branch in a Pipeline gets blocked until another branch completes streaming the document. The Join Snap might hang if its upstream Snaps in a Pipeline has a blocked branch.

Workaround : Set Sorted streams to Unsorted in the Join Snap to effectively buffer all documents in all input views internally—this unblocks the document flow of all the upstream branches. The internal sorters sort the input documents from the input views into the local temporary stage.

I’ll do some investigating

@NAl - if you want all the data to be inserted into the Create Deal/Job before any data is sent to the Upsert Content, you could simply place a Sort just before the Upsert. This will effectively cache all the data until the upstream snaps have completed.

You are basically running into a race condition where the Deal/Job records may not have been committed to the database before the Upsert Content tries to insert a new record that is reliant on that relationship.

Hi @koryknick, apologies for the delayed response I was working on project that went live.

Have added the Sort snap just before the Upsert but this issues seems to continue.

Merge is not joining the streams based on a field, so my theory is that it’s mixing up the documents from the lower and upper inputs. Try sorting out just before the Join (Merge), but make sure you sort on the same field.

1 Like

@NAl - I just noticed that you are looking at a pipeline validation - it appears that you perhaps have the “Snap Execution” mode set to “Execute Only” meaning that it wouldn’t allow those documents to actually update the endpoint unless you actually execute the pipeline. If that is the case, you are correct that the Upsert would be failing due to those missing references.

Hi @koryknick sorry that’s not it, it’s set to Validate & Execute and am testing with Executing the pipeline.

I’ve actually taken @bojanvelevski’s comments onboard and it seems to be working okay for now but more tests need to be run before signing off.

This all very useful information so appreciate all of your inputs so far!