Skip to content

web_search: fall back to DuckDuckGo when Bing returns zero results #2130

@Hmbown

Description

@Hmbown

Problem

web_search defaults to Bing HTML scraping. Bing works for simple one-word queries ("test") but silently returns zero results for technical/compound queries ("Claude Code tool architecture agent loop", "rust async traits stabilization"). This is a known Bing HTML search behavior — their search page is ad-driven and doesn't surface organic results for specific technical queries.

Meanwhile, web.run (which uses DuckDuckGo) returns 5 perfectly relevant results for the same queries.

Evidence (macOS, from US IP):

Tool Query Result
web_search (Bing) "test" 2 results
web_search (Bing) "Claude Code tool architecture..." empty
web.run (DuckDuckGo) "Claude Code tool architecture agent loop" 5 results, perfectly relevant

Proposed Fix

In crates/tui/src/tools/web_search.rs, the DuckDuckGo path already falls back to Bing when DDG returns a bot challenge or empty results (lines 254-281). Add the symmetric fallback: when Bing returns zero results, try DuckDuckGo before reporting empty.

This is a ~15-line change in the SearchProvider::Bing branch (around line 213-217):

// Current (line 213-217):
if matches!(context.search_provider, SearchProvider::Bing) {
    check_policy(decider, BING_HOST)?;
    let results = run_bing_search(&client, &query, max_results).await?;
    return search_tool_result(query, "bing", results, None);
}

// Proposed: add zero-result fallback to DDG
if matches!(context.search_provider, SearchProvider::Bing) {
    check_policy(decider, BING_HOST)?;
    let results = run_bing_search(&client, &query, max_results).await?;
    if !results.is_empty() {
        return search_tool_result(query, "bing", results, None);
    }
    // Bing returned zero — try DuckDuckGo as fallback
    check_policy(decider, DUCKDUCKGO_HOST)?;
    // ... run DDG search, return with "bing returned no results; used DuckDuckGo fallback" message
}

Acceptance Criteria

  • web_search "rust async traits stabilization" returns results (via DDG fallback)
  • web_search "test" still returns Bing results directly (no fallback needed)
  • Message includes provenance: "Bing returned no results; used DuckDuckGo fallback"
  • Network policy gates applied for both hosts independently
  • Spam detection (web_search / web.run returns unusable results — DDG blocked, Bing garbage #964) still runs on fallback results
  • Existing tests pass (cargo test -p codewhale-tui web_search)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    autonomous-readyGreenlit for the autonomous nightly loop to pick up, implement, PR, and mergebugSomething isn't workingenhancementNew feature or requestrustPull requests that update rust code

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions