Designing and Running Pipelines

How to capture Error records in Redshift Bulk load

I have a requirement to load around ~300k records to a Redshift table.I have used Redshift Bulk snap for this purpose with the error count setting to 20000.I have an error pipeline called in this main pipeline. After executing this pipeline I was hop...

KERBEROS issues - HDFS cluster

Hi , We’ve implemented Kerberos in our hadoop cluster, but we are having issues with our pipelines. We are getting this error "Reason: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBER...

Calculate file size while reading using File Reader snap

I want to calculate the size of input file read by a File Reader snap, to use it as a parameter to define my pool size for child pipelines. Is there a way we can do this using File Reader snap ? Thanks!!

Kafka Consumer Crash

Hi All, If I am using a Confluent Kafka Consumer Snap and it crashes in between, Will it commit an offset? For example if there are 500 documents in Kafka and after reading 250 documents Kafka crashes. Will it read from the 251st document when I run ...

Reusability Component in Snaplogic

Mapplet is a resusable component in IDQ in which we can pass generic fields to check the same condition for different fields. Suppose I want to make a generic mapping ( checking fields for Nullability and produce valid or invalid flag). Input fields ...

How to create soap xml from json

how to map all the data present in json format to soap xml format

Json to soap envelope format

I have a requirement of converting JSON to a soap envelope format(XML). Any suggestions?

Oracle Connection and Number of session

Hi , I am developing a load plan for loading CSV into Oracle Tables and few Oracle to Oracle. How many pipeline can be executed in parallel, which will load CSV into oracle table? where is the configuration settings which manages the number of sessio...

SnapLogic - Integration Nation

Forum Posts

How to capture Error records in Redshift Bulk load

KERBEROS issues - HDFS cluster

Calculate file size while reading using File Reader snap

Kafka Consumer Crash

Reusability Component in Snaplogic

How to create soap xml from json

Json to soap envelope format

Oracle Connection and Number of session

Generating individual files based on 'n' records f...

Convert array with objects into object with key/va...

Converting a string to JSON doesn't work

Rest Post for Multipart form type content type wit...

Best way to perform multiple search and replace in...

Obtain Rest API curl to run in postman directly

CSV parser cannot parse data

Parsing XML using a Python script returns an encod...

multiple array objects

String replace method where pattern is dynamic