โ05-07-2018 11:59 AM
Hi
I wanted to read all the records from the kafka queue in first run and then latest records from the queue from kafka consumer snap.I start reading for the first time from the queue,what setting of โseek typeโ and โauto offset resetโ should I use in this scenario.
โ05-09-2018 09:38 AM
Hi,
Based on your scenario, when first run the consumer snap, you may use a new group ID in the Consumer, and set โauto offset resetโ to earliest. This would let the Consumer to read from earliest at the first running (when initialize the new group). For โseek typeโ, you may set it to End, so that it would continue reading when resume. Hope it helps ๐
โ11-05-2018 04:30 AM
Hi @kzhou,
Very thanks for your above statements. Those will be work irrespective of partition number right?
Please confirm the same. Thanks in advance!
โ03-11-2019 09:30 AM
@GBekkanti, sorry for the late response. The offset in Kafka is a per-partition based, and the configuration โauto offset resetโ and โseek typeโ would be in the partition where the Consumer is assigned to.