Designing and Running Pipelines

Resolved! Kafka consumer Skip messages when fail

Hi , We pull the data from Kafka and put it and the database, but we realized the Kafka consumer skips the data/offsets if the pipeline fails. for example: in a run Kafka consumer is supposed to read offset 3,4,5 but it pipeline fails so it skips the...

Resolved! Change the key to the value of the value of the key of a String For identifying columns

Hello, So I have a DataSet that looks like this: { Name : testvalue1, Name : testvalue2 } But I want to change the key ‘Name’ into the value it contains: output { testvalue1 : testvalue1, testvalue2 : testvalue2 } It’s needed because after that I w...

org.joda.time.LocalDateTime error for DB2 Insert

Hi, I had an issue with DateTime conversion issues Insert records when migrating data from MySQL to DB2. MYSQL dateformat : 2022-03-17 05:10:10 DB2 timeformat 2022/03/17 05:10:10 I am getting the following error Failure: SQL operation failed, Reason...

Extracting lineage from snapLogic pipelines

How can lineage–source information, target information, field transformations/calculations/derivations, etc.–be extracted from a group of pipelines? I need to be able to do so, to adhere to our company’s data governance rules around data movement. ...

Passing Body Content in Post Snap

Hi, This is a very novice question, but I am having trouble with a post request snap. This is just to authenticate with the RingCentral platform to receive a token for API consumption. I am able to successfully retrieve an access token when calling t...

To replace more than one character in string by looking up another field value

I have a business requirement, where we have below JSON in that, have to update object’s field called TRANSLATION_ID by looking up another fields called VARIABLE_ID and VARIABLE_VALUE. Here if both VARIABLE_ID and VARIABLE_VALUE are not null then rep...

Resolved! Snaplogic Snap Pack To Create, Update, Delete Scheduled Tasks

Hi, I would like to have the ability to create, update, delete Snaplogic scheduled tasks via REST API from a web application. I understand that the [Snaplogic Metadata] snap pack only allows the creation of triggered tasks? Can Snaplogic facilitate t...

Resolved! PipeLine Execute- File Writter- Getting Batch Number

We are currently creating a Pipeline where we are using a Pipeline Execute. Pipeline Execute we have a batch number defined. In the pipeline that is triggered from PE we are writing a file. I would like to know if there is a way to get the batch numb...

SnapLogic - Integration Nation

Forum Posts

Resolved! Kafka consumer Skip messages when fail

Resolved! Change the key to the value of the value of the key of a String For identifying columns

org.joda.time.LocalDateTime error for DB2 Insert

Extracting lineage from snapLogic pipelines

Passing Body Content in Post Snap

To replace more than one character in string by looking up another field value

Resolved! Snaplogic Snap Pack To Create, Update, Delete Scheduled Tasks

Resolved! PipeLine Execute- File Writter- Getting Batch Number

What account should be used for API with query par...

How to dynamically update an account?

Execute each child pipeline multiple times

Routing files from S3

Clean duplicated lines in CSV datasource

How to get all values of a key from objects in a l...

The "Pipeline execute" is changing data types?

Remove XML duplicate key from JSON list in XML for...

Generating individual files based on 'n' records f...

Convert array with objects into object with key/va...

Resolved! Kafka consumer Skip messages when fail

Resolved! Change the key to the value of the value of the key of a String For identifying columns

Resolved! Snaplogic Snap Pack To Create, Update, Delete *Scheduled* Tasks

Resolved! PipeLine Execute- File Writter- Getting Batch Number

Resolved! Snaplogic Snap Pack To Create, Update, Delete Scheduled Tasks