Slow PostgreSQL Update Snap
I have a PostgreSQL Update snap at the end of a pipeline. It’s job is to update a column in a row if nothing in an inbound row (matching that data) has been modified. This means that most of the time...
Forum Discussion
Thank you, Diane. Unfortunetly, FileName is not coming through after Json Parser and therefore, there is no way to get to it in JSON Splitter.
ZipFile Read produces following output:
JSON Parser follows directly after ZipFile Read and outputs only content of the data file itself:
Therefore, JSON Path does not have FileName:
Any ideas how I can pass FileName from ZipFile Read to JSON Parser and then to JSON Splitter?
Thank you!
(As you found out, using BinaryToDocument is one approach, I’m just offering another for reference)
I can offer a solution that is possible if you’re on the new 4.16 release. Since the JSONParser doesn’t pass through the header from the binary document, you can use a child pipeline to do the parsing and pass the filename as a pipeline parameter that can be referenced anywhere in the child pipeline. This approach is possible in 4.16 since the PipelineExecute snap can now support binary views.
You can switch the type of view by going into the “Views” tab and changing the value under the “Type” column for the given view:
So, your pipeline will look something like the following with the binary documents from the zip file going into a child pipeline:
And, the $['content-location'] field containing the file name being passed as a parameter to the child execution in the PipelineExecute:
The child pipeline would then have the JSONParser and a Mapper to add in the $FileName field:
I’m attaching the parent and child pipelines as well:
ParseJsonWithFilename_2019_02_18.slp (4.5 KB)
ReadZipFileAndParse_2019_02_18.slp (3.7 KB)
You should be able to reference the other library through the
libvariable. Is that not working?For example, if you this as the library
helper.expr:{ chefify: x => x + ' bork! bork! bork!' }You should be able to reference it from another library, like so:
{ msg: () => lib.helper.chefify('Hello, World!') }Note that if the references are not in functions, but directly in the top-level expression, you’ll need to ensure the libraries are listed in the right order in the pipeline properties.
I found a better way to achieve what i was look for.
But thanks for highlighting this.@tstack
I have a use case where a child pipeline has multiple expr files.
I want to pass a value to the child pipeline using which my chiled pipeline would choose one of the expression library.
is there a way to do it ?In the child pipeline properties, add a parameter that you’ll use to specify the library to load. Then, in the “Expression Libraries” section, add a path that uses the parameter you just created. You’ll probably also want to set the name of the imported library using the “As” column. In the screenshot below, the parameter is
configand defaults to the library filedev.exprand the library can be referenced vialib.confin expressions:In the parent pipeline, you’ll pass the
configparameter with the library paths that you want. In the screenshot below, there are two documents entering the PipeExec with “dev.expr” and “prod.expr” as the paths. So, the child will execute first with the “dev” library and then with the “prod” library.
Is there a way to declare methods with global scope in expression library.
Is nesting multiple libraries together a good preactice ?Do you mean without the
lib.libnameprefix? No, libraries are always imported under thelibobject.I’m not sure what you mean by this, can you give an example?
