lookup: missing config or lookup value

[tuttle]

Hi, thank you for this neat and handy project.

I installed the release v0.0.3 to /opt/botcheck and added this to Apache 2.4.66-1~deb11u1 site conf:

    Define BOTCHECK_DIR /opt/botcheck/apache
    Include ${BOTCHECK_DIR}/botcheck.conf

It appears to work (displays the Are you human? box on the first page load) but produces lots of these log lines to apache log:

[rewrite-map] invalid input "useragents.ri.list;": missing config or lookup value

Is there a problem on my side?

[tuttle]

I asked Opus to investigate. Here's the analysis:

This is a bug in the lookup program.

Root cause

The Apache config does this lookup for every request:

RewriteCond ${lookup:useragents.ri.list;%{HTTP_USER_AGENT}|NOT_FOUND} !=NOT_FOUND

Apache sends useragents.ri.list;%{HTTP_USER_AGENT} to the lookup program's stdin. When a client sends a request without a User-Agent header (or with an empty one), this resolves to just useragents.ri.list; — an empty lookup value.

The parser.go in parseInput() splits on ; and then rejects the input when either side is empty:

if config == "" || lookup == "" {
    return "", "", fmt.Errorf("missing config or lookup value")
}

This triggers the error log in main.go:

log.Printf("invalid input %q: %v", scanner.Text(), err)

which produces exactly the message you're seeing.

Why it's a bug

An empty User-Agent header is a perfectly normal condition — many HTTP clients, bots, and monitoring tools don't send one. The lookup program should treat an empty lookup value as a non-match and return NULL silently, rather than logging it as an error. The other two lookups (%{REQUEST_URI}, %{REMOTE_ADDR}) never hit this because those Apache variables are never empty.

Recommendation

File an issue with the project. The fix should be in parser.go: when the lookup value is empty, return a sentinel or handle it in main.go to simply respond NULL without logging an error. A one-line config workaround on your side would be to add a condition before the User-Agent check:

RewriteCond %{HTTP_USER_AGENT} .

right before the existing RewriteCond ${lookup:useragents.ri.list;...} line, so the lookup is skipped entirely when the header is absent.