Skip to content

in_tcp: add source address even when format is none#10264

Open
jadeKim1993 wants to merge 2 commits intofluent:masterfrom
jadeKim1993:in_tcp_source_address
Open

in_tcp: add source address even when format is none#10264
jadeKim1993 wants to merge 2 commits intofluent:masterfrom
jadeKim1993:in_tcp_source_address

Conversation

@jadeKim1993
Copy link
Copy Markdown

@jadeKim1993 jadeKim1993 commented Apr 28, 2025

Modify to add source_address even when input format is none instead of JSON


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
[INPUT]
    Name        tcp
    Listen      0.0.0.0
    Port        55555
    Format      none
    Source_Address_Key source_address

[OUTPUT]
    Name        stdout
    Match       *
  • Debug log output from testing the change
[2026/01/08 14:42:27] [ info] Configuration:
[2026/01/08 14:42:27] [ info]  flush time     | 1.000000 seconds
[2026/01/08 14:42:27] [ info]  grace          | 5 seconds
[2026/01/08 14:42:27] [ info]  daemon         | 0
[2026/01/08 14:42:27] [ info] ___________
[2026/01/08 14:42:27] [ info]  inputs:
[2026/01/08 14:42:27] [ info]      tcp
[2026/01/08 14:42:27] [ info] ___________
[2026/01/08 14:42:27] [ info]  filters:
[2026/01/08 14:42:27] [ info] ___________
[2026/01/08 14:42:27] [ info]  outputs:
[2026/01/08 14:42:27] [ info]      stdout.0
[2026/01/08 14:42:27] [ info] ___________
[2026/01/08 14:42:27] [ info]  collectors:
[2026/01/08 14:42:27] [ info] [fluent bit] version=4.0.2, commit=3a17947d29, pid=63465
[2026/01/08 14:42:27] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2026/01/08 14:42:27] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2026/01/08 14:42:27] [ info] [simd    ] disabled
[2026/01/08 14:42:27] [ info] [cmetrics] version=1.0.0
[2026/01/08 14:42:27] [ info] [ctraces ] version=0.6.4
[2026/01/08 14:42:27] [ info] [input:tcp:tcp.0] initializing
[2026/01/08 14:42:27] [ info] [input:tcp:tcp.0] storage_strategy='memory' (memory only)
[2026/01/08 14:42:27] [debug] [tcp:tcp.0] created event channels: read=25 write=26
[2026/01/08 14:42:27] [debug] [downstream] listening on 0.0.0.0:55555
[2026/01/08 14:42:27] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2026/01/08 14:42:27] [ info] [sp] stream processor started
[2026/01/08 14:42:27] [ info] [output:stdout:stdout.0] worker #0 started
[2026/01/08 14:42:31] [debug] [task] created task=0x7fde14029420 id=0 OK
[2026/01/08 14:42:31] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] tcp.0: [[1767850950.628888995, {}], {"log"=>"Hello TCP", "source_address"=>"tcp://127.0.0.1:60994"}]
[2026/01/08 14:42:31] [debug] [out flush] cb_destroy coro_id=0
[2026/01/08 14:42:31] [debug] [task] destroy task=0x7fde14029420 (task_id=0)
  • Attached Valgrind output that shows no leaks or memory corruption was found
==63490== Memcheck, a memory error detector
==63490== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==63490== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==63490== Command: ./fluent-bit -c test.conf
==63490==
Fluent Bit v4.0.2
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2026/01/08 14:54:08] [ info] Configuration:
[2026/01/08 14:54:08] [ info]  flush time     | 1.000000 seconds
[2026/01/08 14:54:08] [ info]  grace          | 5 seconds
[2026/01/08 14:54:08] [ info]  daemon         | 0
[2026/01/08 14:54:08] [ info] ___________
[2026/01/08 14:54:08] [ info]  inputs:
[2026/01/08 14:54:08] [ info]      tcp
[2026/01/08 14:54:08] [ info] ___________
[2026/01/08 14:54:08] [ info]  filters:
[2026/01/08 14:54:08] [ info] ___________
[2026/01/08 14:54:08] [ info]  outputs:
[2026/01/08 14:54:08] [ info]      stdout.0
[2026/01/08 14:54:08] [ info] ___________
[2026/01/08 14:54:08] [ info]  collectors:
[2026/01/08 14:54:08] [ info] [fluent bit] version=4.0.2, commit=3a17947d29, pid=63490
[2026/01/08 14:54:08] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2026/01/08 14:54:08] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2026/01/08 14:54:08] [ info] [simd    ] disabled
[2026/01/08 14:54:08] [ info] [cmetrics] version=1.0.0
[2026/01/08 14:54:08] [ info] [output:stdout:stdout.0] worker #0 started
[2026/01/08 14:54:08] [ info] [ctraces ] version=0.6.4
[2026/01/08 14:54:08] [ info] [input:tcp:tcp.0] initializing
[2026/01/08 14:54:08] [ info] [input:tcp:tcp.0] storage_strategy='memory' (memory only)
[2026/01/08 14:54:08] [debug] [tcp:tcp.0] created event channels: read=25 write=26
[2026/01/08 14:54:08] [debug] [downstream] listening on 0.0.0.0:55555
[2026/01/08 14:54:08] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2026/01/08 14:54:08] [ info] [sp] stream processor started
[0] tcp.0: [[1767851652.262490636, {}], {"log"=>"Hello TCP", "source_address"=>"tcp://127.0.0.1:59940"}]
[2026/01/08 14:54:12] [debug] [task] created task=0x54af260 id=0 OK
[2026/01/08 14:54:12] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2026/01/08 14:54:12] [debug] [out flush] cb_destroy coro_id=0
[2026/01/08 14:54:12] [debug] [task] destroy task=0x54af260 (task_id=0)
^C[2026/01/08 14:54:15] [engine] caught signal (SIGINT)
[2026/01/08 14:54:15] [ warn] [engine] service will shutdown in max 5 seconds
[2026/01/08 14:54:15] [ info] [engine] service has stopped (0 pending tasks)
[2026/01/08 14:54:15] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2026/01/08 14:54:15] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==63490==
==63490== HEAP SUMMARY:
==63490==     in use at exit: 0 bytes in 0 blocks
==63490==   total heap usage: 1,701 allocs, 1,701 frees, 900,312 bytes allocated
==63490==
==63490== All heap blocks were freed -- no leaks are possible
==63490==
==63490== For lists of detected and suppressed errors, rerun with: -s
==63490== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Please refer to the official documentation at the following link
(https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/inputs/tcp.md)

Backporting

  • [N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • TCP input plugin now supports capturing the source address from incoming client connections when using format=none payload format. Configure this feature using the source_address_key option to store the source address in a custom field.
  • Tests

    • Added test coverage verifying source address capture functionality works correctly with format=none.

Signed-off-by: GeunTae Kim <gtkim100@gmail.com>
@jadeKim1993 jadeKim1993 force-pushed the in_tcp_source_address branch from 93c419c to 3a17947 Compare April 28, 2025 23:43
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 7, 2025

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the Stale label Sep 7, 2025
@jadeKim1993
Copy link
Copy Markdown
Author

I have updated the PR description. Please review it at your convenience.
thank you.

@leonardo-albertovich
Copy link
Copy Markdown
Contributor

@edsiper this PR is pretty simple and it's covered by a test case, it's ready for the next merge window.

@github-actions github-actions bot removed the Stale label Jan 16, 2026
@leonardo-albertovich
Copy link
Copy Markdown
Contributor

@edsiper could you please make the test cases run so the code doesn't go stale for the next release cycle? It's a really simple change that shouldn't cause any problems.

@edsiper
Copy link
Copy Markdown
Member

edsiper commented Feb 20, 2026

not sure why CI is now not running, will check.

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: jade <54811879+jadeKim1993@users.noreply.github.com>
@jadeKim1993 jadeKim1993 requested a review from cosmo0920 as a code owner March 31, 2026 04:16
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 31, 2026

📝 Walkthrough

Walkthrough

This PR extends the TCP input plugin to optionally include the remote source address in output records when format=none is configured. It introduces conditional logic in parse_payload_none() to append a source address field to the output record and includes a test case validating this behavior.

Changes

Cohort / File(s) Summary
TCP Payload Parsing
plugins/in_tcp/tcp_conn.c
Modified parse_payload_none() to conditionally retrieve and append source address information to the output record when ctx->source_address_key is configured.
Runtime Tests
tests/runtime/in_tcp.c
Added new test flb_test_format_none_with_source_address() that validates source address field inclusion in TCP messages with format=none configuration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • #11531 – Also modifies the parse_payload_none() function in the same file with type/bounds adjustments.

Suggested reviewers

  • edsiper

Poem

🐰 A whisker-twitching tcp tale,
Source addresses now set sail,
In none-formatted streams so bright,
Connection origins, shining light! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding source address support to the TCP input plugin when format is set to 'none', which is the primary focus of both the code and test changes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/runtime/in_tcp.c (1)

687-687: Make the assertion less order-dependent to avoid brittle test failures.

Line 687 expects "log" immediately followed by "source_host". Prefer checking only "source_host":"tcp:// here (the log field is already covered by flb_test_format_none) or validating both fields independently.

♻️ Minimal diff to reduce ordering sensitivity
-    cb_data.data = "\"log\":\"message\",\"source_host\":\"tcp://";
+    cb_data.data = "\"source_host\":\"tcp://";
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/runtime/in_tcp.c` at line 687, The test currently sets cb_data.data to
a string that requires the "log" field to appear immediately before
"source_host" which makes the assertion order-dependent; change the test so
cb_data.data only asserts the presence of the "source_host":"tcp://" fragment
(or validate "log" and "source_host" independently) instead of the exact
ordering—update the assignment to cb_data.data in tests/runtime/in_tcp.c and
adjust any assertions that rely on exact field order (note: flb_test_format_none
already covers "log", so keep that check separate).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/runtime/in_tcp.c`:
- Line 687: The test currently sets cb_data.data to a string that requires the
"log" field to appear immediately before "source_host" which makes the assertion
order-dependent; change the test so cb_data.data only asserts the presence of
the "source_host":"tcp://" fragment (or validate "log" and "source_host"
independently) instead of the exact ordering—update the assignment to
cb_data.data in tests/runtime/in_tcp.c and adjust any assertions that rely on
exact field order (note: flb_test_format_none already covers "log", so keep that
check separate).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f192e424-ced0-4ae8-8ab2-c0ffa81c1781

📥 Commits

Reviewing files that changed from the base of the PR and between 680f14c and f05593e.

📒 Files selected for processing (2)
  • plugins/in_tcp/tcp_conn.c
  • tests/runtime/in_tcp.c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants