task: Provider Config

[FWuermse]

Context

Providers need control over how their resource fields are indexed by the search operator. Currently all fields are auto-classified by heuristic.

Feature

A SearchConfig resource (one per APIResourceSchema) lives in the provider workspace and declares:

• excludedFields: not indexed
• semanticFields: vector/semantic search
• filterFields: keyword filtering

Unlisted fields default to full-text search. No SearchConfig = current heuristic behavior (backwards compatible).

Acceptance Criteria

• SearchConfig type exists in search.platform-mesh.io/v1alpha1 (cluster-scoped)
• One SearchConfig per APIResourceSchema, named identically, stored in provider workspace
• search-operator reads SearchConfig from provider workspace when creating/updating SearchIndex
• Falls back to default when no SearchConfig exists
• excludedFields are removed from default index field lists
• semanticFields map to SearchIndex.spec.semanticFields
• exactFields map to SearchIndex.spec.filterableFields
• SearchConfig changes trigger SearchIndex reconciliation (for each org that binds the workspace)
• status.conditions[SchemaValid] warns when referenced fields don't exist in schema
• searchconfigs added to search APIExport permission claims
• Document changes in new ADR

Open Questions

• Re-indexing strategy when config changes (eventual consistency vs full re-index?)
• Validation: reject invalid field refs or just warn via condition?

No response

Completion Checklist

• Code implemented and tested
• Documentation updated
• Changes reviewed

[pteich]

Additional open question:

• Naming pattern for the resources
• Reference to dedicated CRD/APIResource schema

For handling changes:

• Removing the search index to make sure references to excluded fields are fully removed
  OR
• Change mapping and trigger re-index (overwrite data but OpenSearch can save versions of documents)

[FWuermse]

Problem: Search Operator, Search Service use SearchIndex as source of truth for valid and indexable fields, SearchConfig only provides information about which fields to treat special. Only storing fields in the SearchConfig leads to expensive lookups for every search&index operation.

Solution for now: SearchConfig only contains information about what fields to treat special, SearchIndex stores all fields to be indexed including their type and acts as source of truth.

[FWuermse]

Additional open question:

• Naming pattern for the resources
• Reference to dedicated CRD/APIResource schema

For handling changes:

• Removing the search index to make sure references to excluded fields are fully removed
  OR
• Change mapping and trigger re-index (overwrite data but OpenSearch can save versions of documents)

Naming: Same as for SearchIndex given we store it in root:orgs aswell.
Retrigger: Reindex every 24h for config changes?