Pipeline Execute Pool size
I am trying to use the pipeline execute snap to launch child pipelines. currently the throughput of this is relatively slow, to improve this I would like to increase the pool size to allow multiple childs to run at the same time however regardless of changing this setting it seems the run continues to only process one document at a time. for example please see screenshots below: In the screenshots above i would expect 10 documents to be on the input section, with 10 instances of the DELME pipeline running, however only 1 is running at a time?48Views0likes4CommentsPreview not available for Flow snaps
Certain flow snaps like “Filter”, “Head”, “Tail” when used in a complex pipeline doesn’t generate a preview while validating the entire pipeline however these generate a preview if the pipeline is simple. For instance, if I’m trying to read data from an excel file with 100k+ records and use either ‘filter’ or ‘head/tail’, I can see the preview when the pipeline has been validated but it doesn’t generate one when the pipeline is huge. Any specific reason behind this? I would want to see the preview as the pipeline that I’m currently developing is a complex one (multiple snaps prior to “flow” snaps), thereafter it needs a ‘filter + head/tail’ (i.e. flow snaps) and then it will again require a bunch of other snaps after those ‘filter + head/tail’ making it complex again. Help on this matter would be highly appreciated. Regards, DarshSolved31KViews0likes13CommentsError while executing sequence - NotAcceptableIOException : Not acceptable for testsnap2lsat: { "error_map" : null }
Hi…I’m getting the below error in my sequence on random pipelines. It automatically gets resolved and repeats again, however the child pipelines when executed individually works fine. Does anyone have an idea on this? Any information would be helpful! com.snaplogic.snap.api.SnapDataException: Failed to execute pipeline at com.snaplogic.snaps.flow.PipeExec.executePipeline(PipeExec.java:1121) at com.snaplogic.snaps.flow.PipeExec.process(PipeExec.java:802) at com.snaplogic.snaps.flow.PipeExec.processSafely(PipeExec.java:627) at com.snaplogic.snaps.flow.PipeExec.execute(PipeExec.java:486) at com.snaplogic.cc.snap.common.SnapRunnableImpl.executeSnap(SnapRunnableImpl.java:812) at com.snaplogic.cc.snap.common.SnapRunnableImpl.execute(SnapRunnableImpl.java:586) at com.snaplogic.cc.snap.common.SnapRunnableImpl.doRun(SnapRunnableImpl.java:877) at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:436) at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:120) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.snaplogic.api.ExecutionException: Prepare on neighbor failed at com.snaplogic.cc.pipeline.PipelineManager.plexExecute(PipelineManager.java:2395) at com.snaplogic.cc.pipeline.PipelineManager.execute(PipelineManager.java:2448) at com.snaplogic.snapi.pipeline.builder.PipelineExecutionBuilderImpl.execute(PipelineExecutionBuilderImpl.java:220) at com.snaplogic.snapi.pipeline.builder.PipelineExecutionBuilderImpl.retry(PipelineExecutionBuilderImpl.java:185) at com.snaplogic.snaps.flow.PipeExec.executePipeline(PipeExec.java:1021) ... 14 more Caused by: com.snaplogic.util.NotAcceptableIOException: Not acceptable for testsnap2lsat: { "error_map" : null } at com.snaplogic.util.HttpUtils.getInputStreamFromResponse(HttpUtils.java:250) at com.snaplogic.util.HttpUtils.getInputStreamFromResponse(HttpUtils.java:164) at com.snaplogic.cc.pipeline.PipelineManager.plexExecute(PipelineManager.java:2334) ... 18 more Reason: Not acceptable for testsnap2lsat: { "error_map" : null } Error Fingerprint[0] = efp:com.snaplogic.snaps.flow.VugOJsmW Error Fingerprint[1] = efp:com.snaplogic.cc.pipeline.fqgzTyoF Error Fingerprint[2] = efp:com.snaplogic.util.aWBrAYmz2.6KViews0likes2CommentsLooping through Api call
Hi We have a requirement where, If we do an api call , we get response something like below { records_remaining : “200”, records_processed: “100”, failed: “0” }, “entity”: [ "rec1: “abc”, “rec2”:“xyz” ] Pagination does not support by Api . we need to wait for 10 sec for every api call until we get records_remaining is 0 for example https:abc.com/8080/getfullload?batch=50 -----> wait for 10 sec and do the same call until we get records remainining to 0 @bojanvelevski Any best way we can achieve this? please help!Solved3.7KViews0likes3CommentsParsing the string as a csv
Hi, I want to parse my dataset which is coming out as a string. Here is the example of the dataset [“field_name1,field_name2,field_name3,field_name4,field_name5,field_name6,field_name7\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}\nValue1,Value2,Value3,Value4,Value5,Value6,{Val7.1:val,Val7.2:val}”] field 1-6 are normal fields field 7 is a json field. Appreciate all you help I want to create a csv file from this dataset which is the result from a rest api callSolved4.2KViews0likes3CommentsPerform a Post API OAuth 2.0 call to get token and pass token to a GET API Basic Auth Header
I need to make a Post Rest API call using OAuth 2.0 with client_id and client_secret and then pass the token to a Get Rest API snap to make the api call Client Authentication ‘Send as Basic Auth Header’. How can I do this using the Rest Snaps?3.6KViews0likes2CommentsFlatten multilevel/nested JSON Objects and Arrays
Hello there, Is there a way to flatten a nested JSON (multiple objects inside objects/arrays) without using a Script Snap? Take this input for example: { "msg":"Hello, World", "num":1, "fruits":{ "apple":1, "orange":4, "inner2":{ "message":"found" }, "cars":[ "bmw", "audi", "dacia" ], "cars2":[ { "bmw":"something" }, { "audi":"else" } ] } } The output should look like this: { "msg": "Hello, World", "fruits.cars2.1.audi": "else", "fruits.orange": 4, "num": 1, "fruits.cars.1": "audi", "fruits.cars.0": "bmw", "fruits.apple": 1, "fruits.cars.2": "dacia", "fruits.inner2.message": "found", "fruits.cars2.0.bmw": "something" } So every key name is constructed from the inner objects/arrays of key names delimited with “.” in the case of objects. In the case of arrays, the key name should also have the index number of the current item. One way to do this is using the script snap using any available language. A function that loops through the root object and check the type of the current property. If it’s of type Object or array then I can call the function again. I’ve tried using expression libraries but with no luck. Any ideas?5.2KViews0likes4CommentsSalesforce SOQL - Pass Object Type as Parameter
Hello, Is it possible to pass the “Object Type” into a Salesforce SOQL Snap (or Salesforce lookup or Salesforce Read) I want to dynamically pull data from multiple salesforce objects and pull them into my SQL DB using a lookup table. However, when trying to use a parameter in the “Object Type” field, i get an error, despite the parameter not being null. See attached. ThanksSolved7.7KViews0likes8Comments