Designing and Running Pipelines

Formatting JSON data according to given scenarios

Hi, I need to know the ways of performing the restructuring of the JSON payload based on the requirement.for ex. quiz_id value we need to distribute to each type of question. below are input and expected output need to achieve through snaplogic pipel...

Extracting a large data set

Hi community, I hope everyone is keeping safe. I have a request to design a data pipeline to extract ~100M rows of data, with about 10 columns in the table. What’s the best practice approach to carrying this out? The source system is a DB2 database, ...

Count of successful rows from Oracle Merge

Hi, I have a requirement to record the successful row count from Oracle Merge Snap. this is to maintain the Data Log table. Is there any way to know the successful rows from Oracle Merge. i saw something in Mapper (snap.out.totalCount). Please someon...

Resolved! Getting the Status from a REST call

Hi Folks, I’m having trouble getting a proper handling from Snaplogic on a REST call. So the rest call will return on success: “entity”: { “MessageId”:“0101017165ec422f-dfa2bb38-faac-4c48-bcd7-0e2869d80fb6-000000”, …} And on Error: “entity”: "ERROR: ...

Parquet Writer

Hi I was trying to use parquet writer to write file into S3 folder - where i was unable to do it as it is throwing an error. I tried with other files where i could successfully write into S3. Kindly help me. It says that invalid path or aws account b...

Make same change to multiple ‘Pipeline Execute’ snaps

Hi everybody, I want to make a blanket setting on the ‘Execute On’ option in all ‘Pipeline Execute’ snaps across many pipelines. Is there a way to make the change in one place so it can be reflected in all the existing snaps? Or should I open every s...

How to read and fetch incoming documents in Script snap while using in JavaScript mode

i am planning to use script snap to read incoming data from json generator. through script snap i need to apply some complex logic on this document and then output it, how the incoming document can be refereed in script snap ? also is there any limit...

Updating the status in database after pipeline is successfull

The usecase is to generate a uniqueID and load the uniqueID and status as “inprogress” in database. After that call the child pipelines.If all the child pipeline gets executed successfully update the status of uniqueID to success in database. Is ther...

SnapLogic - Integration Nation

Forum Posts

Formatting JSON data according to given scenarios

Extracting a large data set

Count of successful rows from Oracle Merge

Resolved! Getting the Status from a REST call

Parquet Writer

Make same change to multiple ‘Pipeline Execute’ snaps

How to read and fetch incoming documents in Script snap while using in JavaScript mode

Updating the status in database after pipeline is successfull

What account should be used for API with query par...

How to dynamically update an account?

Execute each child pipeline multiple times

Routing files from S3

Clean duplicated lines in CSV datasource

How to get all values of a key from objects in a l...

The "Pipeline execute" is changing data types?

Remove XML duplicate key from JSON list in XML for...

Generating individual files based on 'n' records f...

Convert array with objects into object with key/va...