Forum Discussion

Pretpark's avatar
Pretpark
New Contributor II
10 months ago
Solved

Remove XML duplicate key from JSON list in XML formatter

Hello, As you can see from the title, i am trying to remove a duplicate key in XML which is created by the XML formatter. The JSON generator contains JSON something along the lines of this: ...
  • GanguP's avatar
    GanguP
    2 years ago

    This worked perfectly fine. I do have questions though,
    1) will this methodology work for multiple tags(arrays)?
    2) Adding the second JSON splitter makes it easier,  I did provide a small number of columns in the sample but it has lot morefields. If we have to add 100's of fields it becomes tedious.  Is there another option or an easier way to do this?

     



  • koryknick's avatar
    koryknick
    2 years ago

    GanguP - rather than specify the individual columns to retain, you can select the "Include scalar parents" checkbox.  As long as the element is a primitive datatype (i.e. not array or object), it will automatically let it pass through.

GanguP's avatar
GanguP
New Contributor III
2 years ago
Copy LinkReport

I am trying to extract the values including the array list and load into Snowflake. Below are the fields

abandoned
abandonSeconds
acwSeconds
agentId
agentSeconds
tagid
tagname

Aleksandar_A's avatar
Aleksandar_A
Contributor III
2 years ago
Copy LinkReport

Attached below is a sample pipeline that will flatten on tags array, so for every element in the tag array you will have a document with the above information from the parent contacts object.

Let me know if this is what you are trying to achieve.

Regards,

Aleksandar.

  • Aleksandar_A's avatar
    Aleksandar_A
    Contributor III
    2 years ago
    Copy LinkReport

    Hello GanguP.

    Would you be able to provide us a sample output, so we can better understand what you want to achieve? 

    Regards,

    Aleksandar.

    • GanguP's avatar
      GanguP
      New Contributor III
      2 years ago
      Copy LinkReport

      I am trying to extract the values including the array list and load into Snowflake. Below are the fields

      abandoned
      abandonSeconds
      acwSeconds
      agentId
      agentSeconds
      tagid
      tagname

      • Aleksandar_A's avatar
        Aleksandar_A
        Contributor III
        2 years ago
        Copy LinkReport

        Attached below is a sample pipeline that will flatten on tags array, so for every element in the tag array you will have a document with the above information from the parent contacts object.

        Let me know if this is what you are trying to achieve.

        Regards,

        Aleksandar.

  • koryknick's avatar
    koryknick
    Employee
    2 years ago
    Copy LinkReport

    GanguP - rather than specify the individual columns to retain, you can select the "Include scalar parents" checkbox.  As long as the element is a primitive datatype (i.e. not array or object), it will automatically let it pass through.

    • GanguP's avatar
      GanguP
      New Contributor III
      2 years ago
      Copy LinkReport

       Thank you so much @Kory  .  Appreciate your quick response