Cache Issue & In memory issues in Ultra

[kamalaker_pinna]

Hi,

Below is the requirement

1. Accept input JSON
2. Fetch Credentials from credential store (Centrify)
3. Run POST API to connect to DocuSign and create Envelope.

We have built 2 approaches to solve our requirement and We are facing cache issue while running the ultra in 1st approach and In Memory Issue in 2nd approach.

Approach 1:

i) Pass the input JSON from application
ii) child pipeline (Get Credentials) to fetch the credentials from Centrify application (Using API)
a) If success, pass to Create Envelop API.
b) If Failure, route to logging or output response.

Both the steps run parallelly.

The problem we are facing is for failure scenario. We have removed/modified the user in Centrify hoping that the pipeline fails but it succeeded for multiple times.

Manually, when we validate the child pipeline we get the null response.

Are we missing something here?
Below is the pipeline screenshot for reference.

Approach 2.

We have made a copy of input JSON (425) so that if input is received then we can run “Get Credentials” step and the remaining process runs as is like approach 1.

The problem we are facing is In memory issue as we are receiving multiple (6-7) JSON for the same test case. Application receives “504 Gateway timeout” as it crosses the threshold but once the In memory step is processed API runs fine but not solving the purpose.
Not sure why introducing copy is a problem here.

Approach 2 works fine with a triggered task (takes 1 input) but with Ultra it receives multiple Input JSON and results in In memory issue.

Screenshot for 2nd approach.

Thanks in advance.

Thanks,
Kamal

[vineesha]

Hi,

is the Single document output checked in the in-memory lookup snap settings?

[kamalaker_pinna]

Hi Vineesha,

Yes, it is set as Single Document output. However, the issue we encounter is in the Input JSON itself. As we are receiving multiple input for JSON, In memory lookup is taking time.

Thanks,
Kamal

[vineesha]

Did you try having the In-memory lookup snap in a child pipeline? As it provides Limited support in Ultra Pipelines when the Single document output field is selected.