Skip to content
This repository was archived by the owner on Apr 24, 2023. It is now read-only.
This repository was archived by the owner on Apr 24, 2023. It is now read-only.

Kafka out failure in name resolution #30

@Lubard

Description

@Lubard

Hi,
we are using this docker image version 1.2.1, and trying to connect send out to Kafka. And we host the kafka brokers on our own servers.
Here is the log we got:

[2020/02/25 10:24:36] [ info] [storage] initializing...

[2020/02/25 10:24:36] [ info] [storage] in-memory

[2020/02/25 10:24:36] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128

[2020/02/25 10:24:36] [ info] [engine] started (pid=1)

[2020/02/25 10:24:36] [ info] [out_kafka] brokers='10.37.XXX.XX:9092,10.37.XXX.YY:9092' topics='cpu_check'

[2020/02/25 10:24:36] [ info] [sp] stream processor started

[2020/02/25 10:24:56] [error] [out_kafka] fluent-bit#producer-1: [thrd:clrv0000160257.ic.ing.net:9092/0]: clrv0000XXX.net:9092/0: Failed to resolve 'clrv0000XXX.net:9092': Temporary failure in name resolution (after 3461538871ms in state INIT)

[2020/02/25 10:24:56] [error] [out_kafka] fluent-bit#producer-1: [thrd:clrv0000160257.ic.ing.net:9092/0]: clrv0000XXX.net:9092/0: Failed to resolve 'clrv0000XXX.net:9092': Temporary failure in name resolution (after 3461538871ms in state INIT)

[2020/02/25 10:24:56] [error] [out_kafka] fluent-bit#producer-1: [thrd:clrv0000160263.ic.ing.net:9092/1]: clrv0000YYY.net:9092/1: Failed to resolve 'clrv0000YYY.net:9092': Temporary failure in name resolution (after 3461538871ms in state INIT)

[2020/02/25 10:24:56] [error] [out_kafka] fluent-bit#producer-1: [thrd:clrv0000160263.ic.ing.net:9092/1]: clrv0000YYY.net:9092/1: Failed to resolve 'clrv0000YYY.net:9092': Temporary failure in name resolution (after 3461538871ms in state INIT)

clrv0000XXX.net is the machine name of the first broker 10.37.XXX.XX
clrv0000YYY.net is the machine name of the second broker 10.37.XXX.YY

It seems the connection was successful as Fluent-bit has created the intended topic on Kafka side, but cannot push data through.

After few minutes, we received this:

[2020/02/25 10:29:45] [ warn] [out_kafka] message delivery failed: Local: Message timed out
[2020/02/25 10:29:45] [ warn] [out_kafka] message delivery failed: Local: Message timed out
[2020/02/25 10:29:45] [ warn] [out_kafka] message delivery failed: Local: Message timed out

Any ideas why?
Our Kafka servers are running just fine with other applications.


Update:
Update the advertised.listeners to brokers' own IP address fixed this issue.
But it does not optimised as all machines are running under the same network...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions