yellowmarlboro
V2EX  ›  问与答

kafka 的连接问题...

  •  
  •   yellowmarlboro · Aug 21, 2018 · 6784 views
    This topic created in 2830 days ago, the information mentioned may be changed or developed.

    目前是 我要通过我的服务器连远程服务器的 kafka,我的服务器只作 producer,远程服务器把我的 ip 加入白名单了, 我用 telnet 也能通,可是用代码(python: ptkafka\kafka-python)就报这种错:

    pykafka: 'Failed to connect newly created broker for ip:host'
    kafka-python: 'NoBrokersAvailable'
    

    目前是 broker 或者配置问题? 另外我服务器不需要安装 kafka 了吧?

    我才看了半天 kafka 菜鸡 求教大佬们

    Supplement 1  ·  Aug 21, 2018

    后来加上api_version参数后就如下错误,找了好多说是kafka配置问题..

    INFO:kafka.client:Bootstrap succeeded: found 1 brokers and 35 topics.
    INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <connected> [IPv4 ('host', port)]>: Closing connection.
    DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <disconnecting> [IPv4 ('host', port)]>: reconnect backoff 0.05978240071782098 after 1 failures
    DEBUG:kafka.metrics.metrics:Added sensor with name bufferpool-wait-time
    DEBUG:kafka.metrics.metrics:Added sensor with name batch-size
    DEBUG:kafka.metrics.metrics:Added sensor with name compression-rate
    DEBUG:kafka.metrics.metrics:Added sensor with name queue-time
    DEBUG:kafka.metrics.metrics:Added sensor with name produce-throttle-time
    DEBUG:kafka.metrics.metrics:Added sensor with name records-per-request
    DEBUG:kafka.metrics.metrics:Added sensor with name bytes
    DEBUG:kafka.metrics.metrics:Added sensor with name record-retries
    DEBUG:kafka.metrics.metrics:Added sensor with name errors
    DEBUG:kafka.metrics.metrics:Added sensor with name record-size-max
    DEBUG:kafka.producer.sender:Starting Kafka producer I/O thread.
    DEBUG:kafka.producer.kafka:Kafka producer started
    DEBUG:kafka.producer.kafka:Requesting metadata update for topic enmonitor
    DEBUG:kafka.client:Initializing connection to node 0 for metadata request
    DEBUG:kafka.client:Initiating connection to node 0 at host_1:port
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-sent
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-received
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.latency
    
    Supplement 2  ·  Aug 21, 2018
    附上一条附言
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 None]>: creating new socket
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
    ERROR:kafka.conn:Connection attempt to <BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]> timed out
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: Closing connection. KafkaConnectionError: timeout
    WARNING:kafka.client:Node 0 connection failed -- refreshing metadata
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnecting> [IPv4 ('host_1', port)]>: reconnect backoff 0.04183829594241324 after 1 failures
    DEBUG:kafka.client:Initializing connection to node 0 for metadata request
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: creating new socket
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
    Traceback (most recent call last):
    kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.
    3 replies    2018-10-18 21:16:04 +08:00
    nilai
        1
    nilai  
       Aug 21, 2018
    advertised.listeners= 这个项有没有配置上??
    yellowmarlboro
        2
    yellowmarlboro  
    OP
       Aug 21, 2018
    @nilai 这个配置是在远程服务器 server.properties 配置的是吧.. 远程没配置好的问题么?
    我指定了 api_version 更改了以后就有附言的问题,远程服务器人告诉我是 api 版本问题 可是我模块已经最高了啊并且官网支持现在 kafka 版本的。。
    Pythonerxiaobai
        3
    Pythonerxiaobai  
       Oct 18, 2018
    @yellowmarlboro 楼主这个问题解决了么?我也遇到了相同的问题
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   3156 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 29ms · UTC 14:15 · PVG 22:15 · LAX 07:15 · JFK 10:15
    ♥ Do have faith in what you're doing.