Solved
Kafka consumer Skip messages when fail
Hi ,
We pull the data from Kafka and put it and the database, but we realized the Kafka consumer skips the data/offsets if the pipeline fails.
for example: in a run Kafka consumer is supposed to ...
Forum Discussion
I think there is no direct way to “Inner join” both Input as the Snowflake Bulk Load snap does not passthrough what is inserted or there are no way to pass through the parent snap`s documents.
I think for now “Merge Join” is the closest to verifying the data inserted into the Snowflake or not, if I am afraid about Validating a pipeline would “Skip” some data so I should change the “Snap Execution” of Kafka Aknowladge snap to Execute only.
let me know if you think the same.
Yes, just change the Ack’s mode to Execute Only.
