Designing and Running Pipelines

How would I use Python scripts to transform data from CSVs or Oracle?

Hi all, I need to transform data in ways that I would typically use pandas or a similar package for. It is manipulation of the data itself with joins, transformations, etc. It would be nice to be able to bring in the data directly from Oracle, join w...

Resolved! Route based on column name

I have a pipeline where I’m pulling data from a table with ~600 columns. About 50 of these columns are storing multiple values for each row (semi-colon delimited). I need to route those 50 columns so I can process/handle them differently. How can I r...

Resolved! Transforming JSON Data

Hello All, Trying to transform the JSON data into a mapper. Can anyone help me to get this resolved without using the script snap? Attached the pipeline with JSON generator and expected output. JSON Mapper_2020_02_12.slp (10.3 KB) Thanks!

Omit missing fields in a Mapper

Hi community, I would like to create a Master Mapper of sorts that will accept an input stream of let’s say field1 and field2, and just output those fields. Mapper: TEST_MasterMapper_2020_02_11.slp (4.9 KB) I’ve uploaded a sample pipeline of what I...

Snowflake - Bulk Load giving error

Hi, I am trying to use Snowflake bulk load snap to write data to a Snowflake table from input view. But when I have tried to validate or execute the pipeline I got the following error, Pipeline Execute: Error details: What is this temp file here? C...

Resolved! Discard all documents in pipeline

At the very end of a pipeline I would like to update a single row in a logging table. I’m going to do that in an Execute Task snap, but the problem I’m having is that it wants to run the child snap 20k times instead of simply running a single update ...

Pass parameter from child pipeline to parent pipeline

I have a parent pipeline that executes a child pipeline. The child pipeline’s sole purpose to write a single row to a logging table. After writing to the logging table, the child pipeline runs a Select Snap to get the newly inserted log_id. My questi...

Convert an existing groundplex to support ultra pipeline execution

Hello Community, we currently have a Groundplex consisting of 2 nodes. We have use cases that require ultra pipelines. The questions are: Can we extend the existing groundplex to support ultra pipelines? Is this possible by adding at least one feedma...

SnapLogic - Integration Nation

Forum Posts

How would I use Python scripts to transform data from CSVs or Oracle?

Resolved! Route based on column name

Resolved! Transforming JSON Data

Omit missing fields in a Mapper

Snowflake - Bulk Load giving error

Resolved! Discard all documents in pipeline

Pass parameter from child pipeline to parent pipeline

Convert an existing groundplex to support ultra pipeline execution

Data Validator Pattern error

Deleting multiple files at once from SFTP folder

SQS -pipeline stalling

Best Approach for Handling CSV Import with Truncat...

Fetch dashboard/monitor details via a pipeline

mapValues returns null when adding certain conditi...

What account should be used for API with query par...

How to dynamically update an account?

Execute each child pipeline multiple times

Routing files from S3