Generate expression file from database query
For some data transformations I would like to use an expression file that is generated each night, instead of querying a SQL database everytime the pipeline is started. I already have data available in the database and now I need to get the data transformed in the expression file JSON format, but I am stuck on getting the right ouput. Coming from a XML oriented environment (with extensive knowledge in XSL but not so much JSON) I have quite some issues with switching to snaps and JSON... Data sample (JSON) from the database [ { "code": "ARTICLEGROUP", "source": "JLG", "target": "10" }, { "code": "COMMODITYCODE", "source": "31251501", "target": "0" }, { "code": "COUNTRYCODE", "source": "AF", "target": "AF" }, { "code": "COUNTRYCODE", "source": "AL", "target": "AL" }, { "code": "COUNTRYCODE", "source": "DZ", "target": "DZ" }, { "code": "COUNTRYCODE", "source": "AS", "target": "AS" }, { "code": "COUNTRYCODE", "source": "AD", "target": "AD" }, { "code": "COUNTRY_ISOCODE", "source": "ARE", "target": "AE" }, { "code": "COUNTRY_ISOCODE", "source": "AFG", "target": "AF" }, { "code": "COUNTRY_ISOCODE", "source": "ALA", "target": "AX" }, { "code": "COUNTRY_ISOCODE", "source": "ALB", "target": "AL" }, { "code": "UOM", "source": "EA", "target": "pi" }, { "code": "UOM", "source": "M", "target": "me" }, { "code": "UOM", "source": "BG", "target": "za" } ] Desired output { "ARTICLEGROUP" : { "JLG": "10" }, "COMMODITYCODE" : { "31251501": "0" }, "COUNTRYCODE" : { "AF": "AF", "AL": "AL", "DZ": "DZ", "AS": "AS", "AD": "AD" }, "COUNTRY_ISOCODE" : { "ARE": "AE", "AFG": "AF", "ALA": "AX", "ALB": "AL" }, "UOM" : { "EA": "pi", "M": "me", "BG": "za" } , getValue : (type, source) => this[type][source] } Anyone can point me in the right direction? Have tried multiple things already, but I can't get the "arrays" right for some reason.
Enable editing of expression library file
When designing pipelines we have environment specific parameters (Dev, Test, Prod) in expression libraries. This renders our pipeline development independent of having to alter pipeline parameters or tasks between environments and deploys. However, the managing of the expression library files are a bit tedious. The file has to be downloaded, edited and uploaded again. It would be awesome to enable editing of the content of expression library file in the preview window. Example mockup: Of course having environment variables in a more accessible and standardised way would be the ideal solution, such as a new entity in SL manager, there could be an environment variables section in the projects as well as having the possibility to access shared env vars.Dynamically change the Delimiter in CSV parser
I’m designing ah pipeline that will handle multiple delimiters in a single CSV parser. I’m using a expression library that has the delimiter details of the respective files as below. [ { “src_file”: “aab”, “Delimit”: “PIPE”, “tgt_table”: “T_STG_AAB” } }, { “src_file”: “abc”, “Delimit”: “TAB”, “tgt_table”: “T_STG_ABC” } }, { “src_file”: “efg”, “Delimit”: “COMMA”, “tgt_table”: “T_STG_EFG” } } ] On the mapper snap I can write expression to refer this expression library lib.sample.find(x => x.src_file==“incoming_filename”).get(‘Delimit’). But when I try to use the same in CSV parser it doesnt accept. can we really do this.I am trying to perform a lookup and extend the object using the expression library
I wanted to perform a extend by performing a lookup using the expression library and it is giving me “this.comparator is undefined” error. Below is my expression Library Content: { embedbusinessUnit: x => x.extend({"Demo":this.comparator[0]}), embedbusinessUnit1: x => x.merge(this.comparator[0]), "comparator": [ { "KSCHL": "EPE", "VKORG": "1000", "BusinessType": "E", "KSCHLN": "LPE", "VKORGN": "1000" }, { "KSCHL": "EP0", "VKORG": "RG", "BusinessType": "N", "KSCHLN": "LP0", "VKORGN": "1000" } ] } When I access the lib.businessUnitExpr.embedbusinessUnit($) gives me a comparator undefined error. But when I access lib.businessUnitExpr.embedbusinessUnit1($) then it is able to identify the comparator Any help would be appreciated. BusinessUnitComparator.expr (382 Bytes)
Using expression libraries in JSON Generator
Hello, I need to generate a JSON with a bunch of dynamic keys, which are stored in an expression library. I haven’t found a well rounded solution so far, except pushing it all together in a single mapper field, however i was hoping to find something more graceful and readable. So my idea was to use a JSON Generator and piece together the information, however I run into two issues: How can I access the expression libraries in a JSON Generator? How can I set the values of the expression library as element key? I’ve tried several combinations to achieve the first thing: [ { "test" : $lib.env.jiraSyncDatefield[pipe.plexPath.split('/')[1]] } ] [ { "test" : ${lib.env.jiraSyncDatefield[pipe.plexPath.split('/')[1]]} } ] Once I have this working, the question arises on how i can set these values as element keys, I hope that will be working by enclosing them in square brackets. Did anyone have a similar task in the past? EDIT: Found a simpler approach, i wasn’t aware that the expression evaluation worked in the target path: Still, it would be good to know if the described use case is possible. Best regards Thomas
Dynamic expression library path
I have a requirement where I need a path to a parameter within a expression library. I have a number of grouped collections, which have the same parameter name, something like this: "Search": { "Material": { "serverIPAdrress": "198.168.0.25", "localId": "1d524a85-7f01-4341-a923-0cc706478562" }, "Price": { "serverIPAdrress": "198.168.0.202", "localId": "b99d7b0e-5d13-4394-bffc-7951df46c292" }, "Customer": { "serverIPAdrress": "198.168.2.131", "localId": "98f0c2a3-d96a-400d-b48b-7b4f8752dc4e" } } I would like to use the Price or Customer, which I have as a input value ($original.searchType) in my mapper, within my expression library path e.g.: lib.config.Environment[pipe.projectPath.split('/')[1]].Search.$original.searchType.localId is there a way to include this in my mapping table, to dynamically allow to pass in generic parameters to the next snap? Thank you Jay
Referencing snap, pipe variables from within an expression library
I’m trying to create an expression library function that needs access to both the snap and pipe variables. Can these variables be accessed directly from within the expression library function or must they be passed in as parameters? Right now my function looks like this: getHeader: (name) => match(pipe.flags.immediate_mode) { true => snap.original.load().get(name), _ => pipe.get("args",{}).get(name) } When I run this pipeline, I get the following error: {"error":"snap is undefined. Perhaps you meant: isNaN, jsonPath, JSON, String, _personsURL, parseInt","reason":null,"resolution":"Check the spelling of the variable"}