Designing and Running Pipelines

Extracting a large data set

Hi community, I hope everyone is keeping safe. I have a request to design a data pipeline to extract ~100M rows of data, with about 10 columns in the table. What’s the best practice approach to carrying this out? The source system is a DB2 database, ...

Count of successful rows from Oracle Merge

Hi, I have a requirement to record the successful row count from Oracle Merge Snap. this is to maintain the Data Log table. Is there any way to know the successful rows from Oracle Merge. i saw something in Mapper (snap.out.totalCount). Please someon...

Resolved! Getting the Status from a REST call

Hi Folks, I’m having trouble getting a proper handling from Snaplogic on a REST call. So the rest call will return on success: “entity”: { “MessageId”:“0101017165ec422f-dfa2bb38-faac-4c48-bcd7-0e2869d80fb6-000000”, …} And on Error: “entity”: "ERROR: ...

Parquet Writer

Hi I was trying to use parquet writer to write file into S3 folder - where i was unable to do it as it is throwing an error. I tried with other files where i could successfully write into S3. Kindly help me. It says that invalid path or aws account b...

Make same change to multiple ‘Pipeline Execute’ snaps

Hi everybody, I want to make a blanket setting on the ‘Execute On’ option in all ‘Pipeline Execute’ snaps across many pipelines. Is there a way to make the change in one place so it can be reflected in all the existing snaps? Or should I open every s...

How to read and fetch incoming documents in Script snap while using in JavaScript mode

i am planning to use script snap to read incoming data from json generator. through script snap i need to apply some complex logic on this document and then output it, how the incoming document can be refereed in script snap ? also is there any limit...

Updating the status in database after pipeline is successfull

The usecase is to generate a uniqueID and load the uniqueID and status as “inprogress” in database. After that call the child pipelines.If all the child pipeline gets executed successfully update the status of uniqueID to success in database. Is ther...

S3 Dynamic Credentials

Does anyone have an example of how to use the ‘S3 Dynamic’ form of credentials?

SnapLogic - Integration Nation

Forum Posts

Extracting a large data set

Count of successful rows from Oracle Merge

Resolved! Getting the Status from a REST call

Parquet Writer

Make same change to multiple ‘Pipeline Execute’ snaps

How to read and fetch incoming documents in Script snap while using in JavaScript mode

Updating the status in database after pipeline is successfull

S3 Dynamic Credentials

Help to parse Json data into csv

Updating Salesforce contact object and Account Loo...

Slicing Data from JSON

Connecting Confluent Kafka using OAuth

Hexadecimal to binary conversion

array.map if field1 meets criteria, I want field2

Filter Array data

How to create timestamps and out put them only usi...

Do JSON mapping in a single mapper

Problem with nested mappings