Designing and Running Pipelines

How to use Deduplicate snap?

I plan to use Deduplicate snap to remove duplicate records from a CSV file. In my CSV file, let’s say I have 3 records with the same ID. I just want to keep 1 record. So I am using Deduplicate snap and set it up like this It works but it also remove...

Different parameter for Pipeline Validation vs Execute Pipeline

Hi, Is there a way to define default values (parameters, mapper, elsewhere) based on the run mode of the pipeline? During pipeline development I wish for some parameter to default to X during validation runs, but have it default to Y during Execution...

Reference pipeline properties from python in a Script Snap

I’d like to be able to reference pipe.projectPath in a python script, in a Script Snap? Is this possible?..what is the syntax?

How to read and delete specific AWS SQS message in snaplogic

Hi, We have a requirement to read specific SQS message (by passing sqs message id) and delete it after entire pipeline process gets completed. When I use the SQS Consumer snap by passing sqs message id, it is reading all the messages from Queue inst...

Empty CSV File With Header Row

I have a need to write an “empty” CSV file that includes only the header row information in the case that there’s no data to populate the file. Does anyone know a good way to do this? Anything to do with the CSV formatter and the header row options d...

Using Python script with Unix snap

Hi Team, I want to run python scripts as it is running on external editor and read about unix snap can call the Python Script. Can you please let me know the steps and how can it is used to call the script. Can we upload it manually on server or do I...

Airflow scheduler for SnapLogic

Hello, Can any one help me on this for Snaplogic and Airflow Integration for scheduling the jobs. Appreciate your help. Thanks Sudheer

Increasing batch and fetch size in Oracle account

I am noticing some performance issues when processing larger volumes of data (10 million records) using the Oracle Bulk Loader in Snaplogic. I am able to compare the job as it runs in its current form (another integration tool) to Snaplogic and a j...

SnapLogic - Integration Nation

Forum Posts

How to use Deduplicate snap?

Different parameter for Pipeline Validation vs Execute Pipeline

Reference pipeline properties from python in a Script Snap

How to read and delete specific AWS SQS message in snaplogic

Empty CSV File With Header Row

Using Python script with Unix snap

Airflow scheduler for SnapLogic

Increasing batch and fetch size in Oracle account

What account should be used for API with query par...

How to dynamically update an account?

Execute each child pipeline multiple times

Routing files from S3

Clean duplicated lines in CSV datasource

How to get all values of a key from objects in a l...

The "Pipeline execute" is changing data types?

Remove XML duplicate key from JSON list in XML for...

Generating individual files based on 'n' records f...

Convert array with objects into object with key/va...