Designing and Running Pipelines

How to capture Error records in Redshift Bulk load

I have a requirement to load around ~300k records to a Redshift table.I have used Redshift Bulk snap for this purpose with the error count setting to 20000.I have an error pipeline called in this main pipeline. After executing this pipeline I was hop...

KERBEROS issues - HDFS cluster

Hi , We’ve implemented Kerberos in our hadoop cluster, but we are having issues with our pipelines. We are getting this error "Reason: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBER...

Calculate file size while reading using File Reader snap

I want to calculate the size of input file read by a File Reader snap, to use it as a parameter to define my pool size for child pipelines. Is there a way we can do this using File Reader snap ? Thanks!!

Kafka Consumer Crash

Hi All, If I am using a Confluent Kafka Consumer Snap and it crashes in between, Will it commit an offset? For example if there are 500 documents in Kafka and after reading 250 documents Kafka crashes. Will it read from the 251st document when I run ...

Reusability Component in Snaplogic

Mapplet is a resusable component in IDQ in which we can pass generic fields to check the same condition for different fields. Suppose I want to make a generic mapping ( checking fields for Nullability and produce valid or invalid flag). Input fields ...

How to create soap xml from json

how to map all the data present in json format to soap xml format

Json to soap envelope format

I have a requirement of converting JSON to a soap envelope format(XML). Any suggestions?

Oracle Connection and Number of session

Hi , I am developing a load plan for loading CSV into Oracle Tables and few Oracle to Oracle. How many pipeline can be executed in parallel, which will load CSV into oracle table? where is the configuration settings which manages the number of sessio...

SnapLogic - Integration Nation

Forum Posts

How to capture Error records in Redshift Bulk load

KERBEROS issues - HDFS cluster

Calculate file size while reading using File Reader snap

Kafka Consumer Crash

Reusability Component in Snaplogic

How to create soap xml from json

Json to soap envelope format

Oracle Connection and Number of session

Fetch dashboard/monitor details via a pipeline

mapValues returns null when adding certain conditi...

What account should be used for API with query par...

How to dynamically update an account?

Execute each child pipeline multiple times

Routing files from S3

Clean duplicated lines in CSV datasource

How to get all values of a key from objects in a l...

The "Pipeline execute" is changing data types?

Remove XML duplicate key from JSON list in XML for...