[Q&A] exec command won't run a curl

[kyra-hrabal]

What is a problem?

I am trying to run an exec command that curls an endpoint but I only want the HTTP status from the header. No output is being returned, and there are no errors in the log. I am running fluentd in a docker container.

Here is an example of the curl:
curl -ik #{ENV['STATUS_ENDPOINT']}

here is my current setup, so that I am getting the http status every 60 seconds:

<system>
  log_level debug
</system>

<source>
  @type exec
  tag endpoint.status
  command "curl -ik #{ENV['STATUS_ENDPOINT']}"
  run_interval 60
  <parse>
    @type none
  </parse>
</source>

<label @FLUENT_LOG>
  <match **>
    @type stdout
  </match>
</label>

When this executes, no output is returned. All I get is:
fluent.debug: {"title":"exec_input","spawn":[{},"curl -ik https://xyz.com"],"mode":["read"],"stderr":"discard","message":"Executing command title=:exec_input spawn=[{}, \"curl -ik https://xyz.com\"] mode=[:read] stderr=:discard"}

I have verified:

• the environment variable is populated as expected
• i am using fluentd v1.19.0 and curl is installed in my image (this is the latest version my company has scanned and fixed vulnerabilities for, soI am using a company-provided image)
• the curl command works from inside the container and returns results
• the exec source does work, as I have other exec inputs that call the aws cli and works just fine

I have tried:

• changing the log level to error
• removing -ik from the command
• replacing -ik with -s
• using bash to execute the command: command /bin/bash -lc 'curl -ik "$STATUS_ENDPOINT"'
• writing a shell and bash script to execute the curl command which exec calls
• running curl with different endpoints

No errors pop up, there is just no output. I am frustrated.

Describe the configuration of Fluentd

<system>
  log_level debug
</system>

<source>
  @type exec
  tag endpoint.status
  command "curl -ik #{ENV['STATUS_ENDPOINT']}"
  run_interval 60
  <parse>
    @type none
  </parse>
</source>

<label @FLUENT_LOG>
  <match **>
    @type stdout
  </match>
</label>

Describe the logs of Fluentd

No error logs, only this debug log:
fluent.debug: {"title":"exec_input","spawn":[{},"curl -ik https://xyz.com"],"mode":["read"],"stderr":"discard","message":"Executing command title=:exec_input spawn=[{}, \"curl -ik https://xyz.com\"] mode=[:read] stderr=:discard"}

Environment

- Fluentd version: 1.19.0

[kenhys]

I am trying to run an exec command that curls an endpoint but I only want the HTTP status from the header. No output is being returned, and there are no errors in the log. I am running fluentd in a docker container.

You just throw away result of the command because there is no appropriate <match> section to match your endpoint.status tag.

If you insert the following snippet into fluent.conf, you can observe the processed message.

<match endpoint.status>
  @type stdout
</match>

[kyra-hrabal]

Sorry, I do have that actually. I was trying to just grab snippets from my conf files so I forgot to include it but I do have

<match endpoint.**>
    @type stdout
</match>

which prints out all the other exec sources for endpoint.* but not for the curl