Array Rename

[Majid]

Hi Team,

I am trying to achieve below. I would need directions on how to achieve below in a mapper or any other way.

Source :
{
“UBER_ID”:[
“1”,
“Integer_pattern”
],
“First_name”:[
“Majid”,
“TextOnly_pattern”
],
“Last_name”:[
“”,
“TextOnly_pattern”
]
}

Target :

I would like to rename the above fields based on the value in the array. Below is the output I am looking at.

{
“Integer_pattern”:[
“1”,
],
“TextOnly_pattern”:[
“Majid”,
],
" TextOnly_pattern":[
“”,
]
}

[del]

@Majid,

Attached is a sample pipeline with a couple of options using the Pivot Snap to produce results similar to what your sample pipeline appears to be attempting. Both options use the same logic - the second just brings the grouping back together if desired.

It may not be your target solution, and my expressions could need a little work, but I hope it helps some with the direction you want to go.

Community.10510_2021_08_05.slp (30.4 KB)

[Majid]

Thank You @del … This works but in my case the number and name of fields will be different from one file to other as I am creating a generic pipeline. Is there anyway to make the Pivot snap number of fields and field names dynamic.

I am also not sure about performance as this method is going to divide each record into x records based on the number of fields.

I appreciate you looking into the use case and providing directions.

[del]

@Majid,

Here is a version 2 of the pipeline that uses a Mapper and Splitter (instead of Pivot snap) for a dynamic pivot of the data. Community.10510_2021_08_05 (v2).slp (32.8 KB)

I’m not sure about performance, either, but I don’t know how you use your validator snap as-is without splitting/pivoting the data - because of the duplicate key issues.

As this is to be a generic pipeline, I think you might be better off using an Expression Library
in place of (or in conjunction with) your Validator. I think you could avoid the pivot, then.

[Majid]

@del Thank you so much… this is what I was exactly looking for. I will verify the performance and update you…

Can you provide example if possible about the way it can be done with expression library. As data validator does not allow any expression library or parameters used I am not sure how this can be achieved in a mapper using expression library.

[del]

@Majid I’m glad the above helped.

I think I may be derailing too much by suggesting the expression library. It was a creative thought, but would require readjusting downstream logic to reach desired end result.

But, for an exercise, I put this together to show where my thought was leaning.
It basically changes your source:

	{
		"UBER_ID": [
			"1",
			"Integer_pattern"
		],
		"First_name": [
			"Majid",
			"TextOnly_pattern"
		],
		"Last_name": [
			"",
			"TextOnly_pattern"
		]
	}

to this:

	{
		"UBER_ID": [
			"1",
			"Integer_pattern",
			"valid"
		],
		"First_name": [
			"Majid",
			"TextOnly_pattern",
			"valid"
		],
		"Last_name": [
			"",
			"TextOnly_pattern",
			"invalid"
		]
	}

But you’d still have to handle this creatively in downstream snaps to get your desired results.

Community.10510.v3_2021_08_06.slp (4.9 KB)
community10510v3.expr.txt (376 Bytes)