JSON lines

Hello,

My pipelines are in two stages.

Pipeline 1. File pipeline I’m loading 5 records of Json data to a s3 file with ‘Json Lines’ attribute checked.
Pipeline 2. Using the s3 file created in Pipeline 1 in a REST POST, to call an API by passing it as a Query parameter. But only first Json line is taken for processing. All the other lines are ignored that is my issue.

My input file expected to be of RAW format

*[*
*{"uri":"entities/aaa","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_DDD"}],"FirstName":[{"value":"FIRST_GGG"}],"LastName":[{"value":"LAST_GGG"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_GGG"},{"value":"TITLE_GGG"}],"Summary":[{"value":"SUMMARY_GGG"}]}}],"Address":[{"value":{"City":[{"value":"CITY_GGG"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_aaa","sourceTable":"SOURCE_TABLE_GGG","type":"configuration/sources/LNKD"}]},*
*{"uri":"entities/bbb","type":"configuration/entityTypes/HCP","attributes":{"Name":[{"value":"NAME_HHH"}],"FirstName":[{"value":"FIRST_HHH"}],"LastName":[{"value":"LAST_HHH"}],"MiddleName":[{"value":"VAN"}],"Employment":[{"value":{"Title":[{"value":"TITLE_HHH"},{"value":"TITLE_HHH"}],"Summary":[{"value":"SUMMARY_HHH"}]}}],"Address":[{"value":{"City":[{"value":"CITY_HHH"}]}}],"Identifiers":[{"value":{"Type":[{"value":"TYPE2"}],"ID":[{"value":"ID1"}]}}]},"crosswalks":[{"value":"CROSSWALK_bbb","sourceTable":"SOURCE_TABLE_HHH","type":"configuration/sources/LNKD"}]}*
*]*
*```*


Here there is a newline got added after each entity. How to bring Raw format of a valid json with newline values?

Thanks,
Mithila JT

Any suggestions to this will be appreciated. Thank you.

@mithsrini - I’ve created a simplified example that I believe does what you are asking.

JSON_Lines_Call.zip (3.8 KB)

There are two pipelines - one to make the REST POST call and the other that provides that API to call, which simply dumps back the records it finds. You will need to create a task for the API and ensure that name matches in the REST POST snap.