In-Memory Lookup cache

[sarathmattam]

Hi, I am trying out In-Memory Lookup snap. I have a pipeline that loads a DB lookup table on the right path and left path has a mapper snap with an open input view that can accept a parameter

Now this pipeline is invoked from a parent pipeline using Pipeline Execute snap. I am passing 10 country codes for lookup and the flow as such is working fine.

But my concern is, the total execution time is way beyond my expectations. Being accessing cached data, I was expecting the child pipeline to respond faster except the first invocation(under the assumption that cache is getting loaded)

So, I would like to know what is the life time of cached data in In-Memory Lookup snap? At point cache gets wiped out and new copy is taken? is there a mechanism to load cache based on a predefined interval or a trigger ?

Thanks & Regards,
Sarath Kallayil

[sarathmattam]

An update … when I checked the details of child pipeline execution from Dashboard, I could see that, for every execution of child pipeline the SQL snap is executed. So, if that is true, then it beats the purpose of using In-Memory cache here. Or is my design wrong ? In that case, how should I design my pipeline so that SQL snap is not getting executed for every invocation of the child pipeline ?

Thanks & Regards,
Sarath

[akidave]

Do you have the “Reuse executions to process documents” option enabled in the Pipeline Execute snap https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438684/Pipeline+Execute ? Without that option being enabled, a new child pipeline is started per input document and the cache will not be effective.

[sarathmattam]

@akidave, I will try that option and let you know. However, If I understand it ryt, you were talking about a case where in multiple documents are available as part of a single transaction. Will the cache in in-memory lookup snap live beyond a transaction?

Thanks & Regards,
Sarath Kallayil

[akidave]

The cache in the in-memory lookup snap will be active for the duration of the pipeline the snap is added to. For child pipelines without reuse option, that is one document. For child pipelines with reuse enabled, the cache duration could be as long as the parent pipeline execution duration.