How to access the expression library loaded in the parent pipeline from a child pipeline

[patan]

I have a scenario as explained below:

I have an expression library which is 176kb to compare with the static data(we have a performance issue with JSON Generator and then performing the join). Which needs to be accessible from the child pipeline which will be executed multiple times, since I have to pass the pipeline properties I cannot configure to reuse executions.

Since the child pipeline will be launching individual pipeline instances which is resulting in loading the expression library every time and causing the performance of Memory usage(avg of 40%).

Expression Library → compareandfetchfilter_new.expr (433 Bytes)
Parent pipeline → invoke-child-pipe-lib-with-170kb_2023_01_02.slp (4.2 KB)
Child Pipeline → compare and get result from Expression Library_2023_01_02.slp (208.9 KB)

Any suggestions would be appreciated.

[Aleksandar_A]

Hi @patan,

I’ve managed to make it run using the Pipeline Parameter but I had to make small modification in the expression library itself.

Expression Library →
compareandfetchfilter_new.expr (553 Bytes)

Next, in the child pipeline in the mapper snap swap lib.arrayLib.getorgId($) with eval(JSON.parse(_expLib).getorgId($))

Child pipeline →
compare and get result from Expression Library_2023_01_03.slp (210.3 KB)

Let me know if this helps you.

BR,
Aleksandar.

[patan]

@AleksandarAngelevski Thank you for your feedback. but this solution will not work as I have more than 1000 records to be compared. A ternary operator solution might not fit my scenario.

[Aleksandar_A]

@patan What you want to say is that the Comparator object will contain more documents than it has right now (2)?

[patan]

@AleksandarAngelevski

The Comparator object will have more documents to be compared than it has now.