SnapLogic SDK - Accessing pipeline properties and parameters

[sg_sripathi]

Hi,

I am writing a custom snap in which I would like to get the pipeline name, runtime id and few predefined path parameter. I do not want to capture these as snap level properties for user to configure. Could you please let me know how to evaluate a expression like “pipe.label” without defining them as pipeline properties?

I stumbled across ExpressionUtil.compile(“pipe.uuid”); code which returns SnapLogicExpression object. But nor sure how to evaluate the expression, which seems to require additional paramters like ScopeStack and DataValueHanler.

Could you please help if there is easy way to access pipeline properties or execute a expression in custom snap?

Regars,
Sripathi

[robin · Accepted solution]

Hello @sg_sripathi, good question and yes this can be done.

Inject the expression utility class:

@Inject
private ExpressionUtils expressionUtils;

String pipeLabel;

then in the configure() method:

@Override
public void configure(PropertyValues propertyValues)
            throws ConfigurationException {
  Document emptyDoc = documentUtility.newDocument();
  pipeLabel = expressionUtils.createExpressionProperty(
  propertyValues, true, "pipe.label").eval(emptyDoc);
}

And then you can just use the variable when writing an output document e.g.:

Map<String, String> data = new LinkedHashMap<String, String>() {{
  put("pipeLabel", pipeLabel);
}};
outputViews.write(documentUtility.newDocument(data));

The Mapper Snap can be used to validate whatever expression value you are trying to use in the custom Snap:

[bgiles]

There’s quite a few additional values you may find interesting. Skimming the code I see

• pipe.args.get(*), e.g., pipe.args.get(‘CONTENT_TYPE’)
• pipe.flags.immediate_mode
• pipe.flags.is_suggest
• pipe.instance_version
• pipe.property_map.instance_version
• pipe.property_map.info.label.value
• pipe.plexPath
• pipe.projectPath
• pipe.ruuid
• pipe.suspendable
• pipe.target_runtime
• pipe.tmpDir
• pipe.update_time
• pipe.update_user_id

There’s already convenience functions for some of these values, e.g., propertyValues.inImmediateMode(), but I don’t have a list of them at hand.

Related - if you haven’t seen it before java can be bad at deleting temporary files even if you set the ‘deleteOnExit()’ flag and explicitly delete it in your finalizer. At least on Linux systems - and that’s what the groundplexes and cloudplexes usually run on. The ‘pipe.tmpDir’ is at a location managed by the snap executor and it deletes everything in that directory after all of the child threads (snaps) have exited.

It’s a really good idea to use it instead of java.getProperty(“java.io.tmpdir”) since these undeleted files can cause problems over time.

[sg_sripathi]

Thanks @robin, @bgiles, this is very helpful. I think the SDK documentation could use some of this.

Basically we are building a custom snap for logging at document level, the snap logs to a file when running on groundplex and to s3 when running on cloudplex. The log itself is very specific to our need for monitoring certain complex pipeline designs. The snap itself work in transparent mode, i.e. input is passed to output without any change, hence this logging step can be added anywhere in the pipeline with different log levels which can be controlled by pipeline parameter. Hence I had the requirement to run the expressions without defining snap properties.

Thanks much for you help, this been very useful.

[Ramesh]

Hi Sripathi,

Did you implement the change which Robin suggested in below ?  If yes, can you please tell me on which snap you implemented this code ?  If possible can you provide a sample pipeline to achieve this logic ?