Forum Discussion

mithsrini's avatar
mithsrini
New Contributor II
5 years ago
Solved

JSON Lines with each entity in individual line - Json formatter

Hello,

My pipelines are in two stages.

Pipeline 1 . File pipeline I’m loading 5 records of Json data to a s3 file with ‘Json Lines’ attribute checked.
Pipeline 2 . Using the s3 file created in Pipeline 1 in a REST POST, to call an API by passing it as a Query parameter. But only first Json line is taken for processing. All the other lines are ignored that is my issue.

My input file expected to be of RAW format

*[*
*{"uri":"entities/aaa","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_DDD"}],"FirstName":[{"value":"FIRST_GGG"}],"LastName":[{"value":"LAST_GGG"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_GGG"},{"value":"TITLE_GGG"}],"Summary":[{"value":"SUMMARY_GGG"}]}}],"Address":[{"value":{"City":[{"value":"CITY_GGG"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_aaa","sourceTable":"SOURCE_TABLE_GGG","type":"configuration/sources/LNKD"}]},*
*{"uri":"entities/bbb","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_HHH"}],"FirstName":[{"value":"FIRST_HHH"}],"LastName":[{"value":"LAST_HHH"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_HHH"},{"value":"TITLE_HHH"}],"Summary":[{"value":"SUMMARY_HHH"}]}}],"Address":[{"value":{"City":[{"value":"CITY_HHH"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_bbb","sourceTable":"SOURCE_TABLE_HHH","type":"configuration/sources/LNKD"}]}*
*]*
*```*


Here there is a newline got added after each entity. How to bring Raw format of a valid json with newline values?

Thanks,
Mithila JT
  • Hi,

    Based on the information you have provided, have you tried reading the contents of the file and doing a JSON splitter on the records and calling the API ?

3 Replies

  • mithsrini's avatar
    mithsrini
    New Contributor II

    Any suggestions to this will be appreciated. Thanks.

    • rajendraj's avatar
      rajendraj
      New Contributor II

      Hi,

      Based on the information you have provided, have you tried reading the contents of the file and doing a JSON splitter on the records and calling the API ?

      • mithsrini's avatar
        mithsrini
        New Contributor II

        Hi Rajendraj,

        Thank you for checking. Yes I used JSON lines and Binary to Document to add comma to the JSON lines to call the API. Its working.

        Regards,
        Mithila JT