JSON Lines with each entity in individual line - Json formatter

[mithsrini]

Hello,

My pipelines are in two stages.

Pipeline 1 . File pipeline I’m loading 5 records of Json data to a s3 file with ‘Json Lines’ attribute checked.
Pipeline 2 . Using the s3 file created in Pipeline 1 in a REST POST, to call an API by passing it as a Query parameter. But only first Json line is taken for processing. All the other lines are ignored that is my issue.

My input file expected to be of RAW format

*[*
*{"uri":"entities/aaa","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_DDD"}],"FirstName":[{"value":"FIRST_GGG"}],"LastName":[{"value":"LAST_GGG"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_GGG"},{"value":"TITLE_GGG"}],"Summary":[{"value":"SUMMARY_GGG"}]}}],"Address":[{"value":{"City":[{"value":"CITY_GGG"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_aaa","sourceTable":"SOURCE_TABLE_GGG","type":"configuration/sources/LNKD"}]},*
*{"uri":"entities/bbb","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_HHH"}],"FirstName":[{"value":"FIRST_HHH"}],"LastName":[{"value":"LAST_HHH"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_HHH"},{"value":"TITLE_HHH"}],"Summary":[{"value":"SUMMARY_HHH"}]}}],"Address":[{"value":{"City":[{"value":"CITY_HHH"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_bbb","sourceTable":"SOURCE_TABLE_HHH","type":"configuration/sources/LNKD"}]}*
*]*
*```*


Here there is a newline got added after each entity. How to bring Raw format of a valid json with newline values?

Thanks,
Mithila JT

[mithsrini]

Any suggestions to this will be appreciated. Thanks.

[rajendraj · Accepted solution]

Hi,

Based on the information you have provided, have you tried reading the contents of the file and doing a JSON splitter on the records and calling the API ?

[mithsrini]

Hi Rajendraj,

Thank you for checking. Yes I used JSON lines and Binary to Document to add comma to the JSON lines to call the API. Its working.

Regards,
Mithila JT