Designing and Running Pipelines

Pipelines failing with read time error on pipeline execute snap

Hi, our production pipelines are facing a severe impact after 4.19 upgrade. we saw lot of pipelines are failing with read timeout error : this kind of error was not observed during 4.18 or previous versions and it started coming very frequently afte...

How would I use Python scripts to transform data from CSVs or Oracle?

Hi all, I need to transform data in ways that I would typically use pandas or a similar package for. It is manipulation of the data itself with joins, transformations, etc. It would be nice to be able to bring in the data directly from Oracle, join w...

Resolved! Route based on column name

I have a pipeline where I’m pulling data from a table with ~600 columns. About 50 of these columns are storing multiple values for each row (semi-colon delimited). I need to route those 50 columns so I can process/handle them differently. How can I r...

Resolved! Transforming JSON Data

Hello All, Trying to transform the JSON data into a mapper. Can anyone help me to get this resolved without using the script snap? Attached the pipeline with JSON generator and expected output. JSON Mapper_2020_02_12.slp (10.3 KB) Thanks!

Omit missing fields in a Mapper

Hi community, I would like to create a Master Mapper of sorts that will accept an input stream of let’s say field1 and field2, and just output those fields. Mapper: TEST_MasterMapper_2020_02_11.slp (4.9 KB) I’ve uploaded a sample pipeline of what I...

Snowflake - Bulk Load giving error

Hi, I am trying to use Snowflake bulk load snap to write data to a Snowflake table from input view. But when I have tried to validate or execute the pipeline I got the following error, Pipeline Execute: Error details: What is this temp file here? C...

Resolved! Discard all documents in pipeline

At the very end of a pipeline I would like to update a single row in a logging table. I’m going to do that in an Execute Task snap, but the problem I’m having is that it wants to run the child snap 20k times instead of simply running a single update ...

Pass parameter from child pipeline to parent pipeline

I have a parent pipeline that executes a child pipeline. The child pipeline’s sole purpose to write a single row to a logging table. After writing to the logging table, the child pipeline runs a Select Snap to get the newly inserted log_id. My questi...

SnapLogic - Integration Nation

Forum Posts

Pipelines failing with read time error on pipeline execute snap

How would I use Python scripts to transform data from CSVs or Oracle?

Resolved! Route based on column name

Resolved! Transforming JSON Data

Omit missing fields in a Mapper

Snowflake - Bulk Load giving error

Resolved! Discard all documents in pipeline

Pass parameter from child pipeline to parent pipeline

Execute each child pipeline multiple times

Routing files from S3

Clean duplicated lines in CSV datasource

How to get all values of a key from objects in a l...

The "Pipeline execute" is changing data types?

Remove XML duplicate key from JSON list in XML for...

Generating individual files based on 'n' records f...

Convert array with objects into object with key/va...

Converting a string to JSON doesn't work

Rest Post for Multipart form type content type wit...