問題1
安裝配置kafka ---> flume ---> hdfs報(bào)錯(cuò)"responseBody={error_code=15,coordinator={node_id=-1,host=,port=-1}}"
flume配置文件如下
agent.sources = kafkaSource
agent.channels = memoryChannel
agent.sinks = hdfsSink
# The channel can be defined as follows.
agent.sources.kafkaSource.channels = memoryChannel
agent.sources.kafkaSource.type=org.apache.flume.source.kafka.KafkaSource
agent.sources.kafkaSource.kafka.bootstrap.servers = slave01:9092,slave02:9092,slave03:9092
agent.sources.kafkaSource.topic=maxwell
#agent.sources.kafkaSource.groupId=flume
agent.sources.kafkaSource.kafka.consumer.timeout.ms=100
agent.channels.memoryChannel.type=memory
agent.channels.memoryChannel.capacity=1000
agent.channels.memoryChannel.transactionCapacity=100
# the sink of hdfs
agent.sinks.hdfsSink.type=hdfs
agent.sinks.hdfsSink.channel = memoryChannel
agent.sinks.hdfsSink.hdfs.path=hdfs://master01:8020/flume
agent.sinks.hdfsSink.hdfs.writeFormat=Text
agent.sinks.hdfsSink.hdfs.fileType=DataStream
講道理這個(gè)配置完成以后，我只要輸入如下命令立刻妥妥地跑起來，但是一直報(bào)錯(cuò)"responseBody={error_code=15,coordinator={node_id=-1,host=,port=-1}}"
flume-ng agent --conf conf --conf-file example.conf --name kafka -Dflume.root.logger=INFO,console
解決
參考如下文章：https://stackoverflow.com/questions/42362911/kafka-high-level-consumer-error-code-15/42416232#42416232逗扒，我把kafka中的信息刪除完后解決問題蛛砰。
kafka系列之在zookeeper中存儲(chǔ)結(jié)構(gòu)參看下圖：
kafka系列之在zookeeper中存儲(chǔ)結(jié)構(gòu)
