找到你要的答案

Q:does kakfka-console-consumer consumes from multiple partitions

Q:kakfka控制台消费者消费从多个分区呢

I have a kafka topic with multiple partitions.

I want to dump the topic into a file to conduct some analysis there, therefore i think the easiest way to do it is to use the kafka-console-consumer.

My question is, does the kafka-console-consumer is able to read to all the topic partitions, or it will be assigned to a single partition?

If not, how do assign the kafka-console-consumer to a specific partition, therefore a would have to start as many kafka-console-consumers as partitions.

我有一个多分区的卡夫卡主题。

我想将这个主题转储到一个文件中进行一些分析,因此我认为最简单的方法就是使用卡夫卡控制台用户。

我的问题是,卡夫卡控制台消费者是否能够读取所有的主题分区,或将被分配到一个单一的分区?

如果没有,如何将卡夫卡控制台用户分配给特定的分区,因此必须将许多卡夫卡控制台用户作为分区启动。

answer1: 回答1:

Yes, kafka-console-consumer reads from all available partitions. Also take a look at kafkacat, it has some more advanced but usefull features.

是的,卡夫卡控制台用户从所有可用分区读取。也要在kafkacat一看,它有一些更先进但有用的功能。

answer2: 回答2:

As previously mentioned by Lukas, you can use kafkacat to dump all messages from a topic to a local file:

kafkacat -b a_broker -t your_topic -o beginning > all_msgs.bin

This will consume messages from all partitions, if you want to limit to a single partition use the -p <partition> switch.

The default output message separator is newline (\n), if you want something else use the -f "<fmt>" option.

由卢卡斯先前所提到的,你可以用kafkacat转储所有的邮件从一个主题到本地文件:

kafkacat -b a_broker -t your_topic -o beginning > all_msgs.bin

这将消耗的信息从所有分区,如果你想限制到一个单一的分区使用的P & lt;分区>;开关。

默认的输出消息分离器是换行符(\n),如果你想让别人使用-f”& lt的东西;FMT >;”选项。

apache-kafka