Welcome to the new Integration Nation!

Kafka Consumer Snap: Issue with Multiple Input Documents

·Jan 05, 2026 11:35 AM·

Hello Team ! I was experimenting with Kafka Consumer Snap, when i tried to open the input view for the snap and provide a list of partition and offset to read from it only processed a single record and rejected all other records saying: "Multiple input documents are present.". So is it not possible to feed the consumer snap with a list of partitions and offsets to get the data ?

9 comments

· Sorted by Oldest

Patrick T.
·
·
No. It is definitely not intended to be used that way.
James H.
·
·
I haven't used this snap but based on the reply from Patrick, I would try to make a child pipeline and then push your list of documents from the parent to the child one at a time.
Patrick T.
·
·
Yes, that's a workaround but I have questions about why you even want to use Kafka this way.
Patrick T.
·
·
It's not intended as a random-access database.
Patrick T.
·
·
Please tell us more about the use case.
Patrick T.
·
·
It seems like you're not taking advantage of Kafka's offset tracking with consumer groups. Virtually all production use cases don't specify a value for the Partition or Offset properties.
Patrick T.
·
·
Kafka has very powerful capabilities for offset tracking and automatic balancing of partition assignments across the members of a consumer group. Our Consumer snap integrates well with these capabilities.
P S.
·
·
Patrick T. Agreed, Kafka shouldn't be treated as a database. We normally rely on group IDs, but in this case, the downstream target specifically requested re-processing for a select list of offsets.
P S.
·
·
That’s why we’re feeding the CSV data to the consumer. For now, I am using the same workaround suggested by James H. i.e. to read the CSV in a parent pipeline and send the offsets one at a time to the child pipeline to handle the read.
👍1
🆗1