Skip to content

docs: update metadata-search config schema#10403

Closed
N-o-Z wants to merge 3 commits into
masterfrom
niro/docs-mds-config-schema
Closed

docs: update metadata-search config schema#10403
N-o-Z wants to merge 3 commits into
masterfrom
niro/docs-mds-config-schema

Conversation

@N-o-Z
Copy link
Copy Markdown
Member

@N-o-Z N-o-Z commented May 8, 2026

Summary

  • Rewrite the metadata-search Configuration Reference and sample to match the current metadata_search config struct in lakeFS Enterprise (pkg/config/config.go).
  • Collapse split keys (lakefs.* / metadata_settings.* / repositories) into a single top-level metadata_search block.
  • Rename endpoint → lakefs_mds_endpoint and secret_access_key → secret_key.
  • Document period (default 60s), concurrency (default 25), and the per-repo storage_namespace override.
  • Note that max_commits now defaults to 1000 and must be >= 1 (no longer 0-to-disable).
  • Add a Validation callout listing the strict checks the service performs at startup.
  • Note that as of treeverse/charts version 1.10.0 the MDS pod runs on the treeverse/lakefs-enterprise image via lakefs mds run, replacing the legacy standalone treeverse/mds container.

Source of truth used: treeverse/lakeFS-Enterprise pkg/config/config.go (MetadataSearch struct + Validate) and pkg/config/defaults.go (default values).

Test plan

  • Render docs/src/datamanagment/metadata-search.md locally with mkdocs serve and verify the Configuration Reference and Sample blocks display correctly.
  • Confirm the YAML in the Sample block is valid and matches a working lakeFS Enterprise config.
  • Cross-check field names/defaults against the latest MetadataSearch struct in lakeFS-Enterprise.

🤖 Generated with Claude Code

@N-o-Z @claude
docs: update metadata-search config schema
8f16553 
Refresh the metadata-search Configuration Reference and sample to match
the current `metadata_search` config struct in lakeFS Enterprise:

- Collapse split keys (`lakefs.*` / `metadata_settings.*` / `repositories`)
  into a single top-level `metadata_search` block.
- Rename `endpoint` -> `lakefs_mds_endpoint` and `secret_access_key` ->
  `secret_key`.
- Document `period` (60s), `concurrency` (25), and the per-repo
  `storage_namespace` override.
- Note that `max_commits` now defaults to 1000 and must be >= 1
  (no longer 0-to-disable).
- Add a Validation callout listing the strict checks the service
  performs at startup.
- Note that as of `treeverse/charts` 1.10.0 MDS runs on the
  `treeverse/lakefs-enterprise` image via `lakefs mds run`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the docs Improvements or additions to documentation label May 8, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026 •
edited
Loading

📚 Documentation preview at https://oss-pr-10403.docs-lakefs-preview.io/

(Updated: 5/11/2026, 7:58:48 PM - Commit: 83ceb45)

@N-o-Z N-o-Z self-assigned this May 8, 2026
@N-o-Z N-o-Z added exclude-changelog PR description should not be included in next release changelog minor-change Used for PRs that don't require issue attached mostly-ai labels May 8, 2026
@N-o-Z N-o-Z requested review from guy-har, nopcoder and talSofer May 8, 2026 02:26
@N-o-Z N-o-Z marked this pull request as ready for review May 8, 2026 21:57
talSofer
talSofer reviewed May 10, 2026
View reviewed changes
Comment thread docs/src/datamanagment/metadata-search.md Outdated
Use branch name prefixes (e.g., `feature-*`) to reduce the need for manual updates when new branches are added.

!!! warning "Validation"
The Metadata Search service refuses to start unless `lakefs_mds_endpoint` is non-empty and `period`,
Copy link
Copy Markdown
Contributor

@talSofer talSofer May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From lakefs Helm chart version 1.10.0

Copy link
Copy Markdown
Member Author

@N-o-Z N-o-Z May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand the comment. Is this a text suggestion?

Copy link
Copy Markdown
Contributor

@talSofer talSofer May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to suggest that we clarify that this condition holds for versions >= 1.10.0

Comment thread docs/src/datamanagment/metadata-search.md
3. Install or upgrade the Helm chart with the updated configuration.

!!! note
From [`lakefs` Helm chart](https://github.com/treeverse/charts) version **1.10.0**, the Metadata Search
Copy link
Copy Markdown
Contributor

@talSofer talSofer May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when customers already using earlier versions of metadata search upgrade to >= 1.10.0?

Copy link
Copy Markdown
Member Author

@N-o-Z N-o-Z May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They will need to update their mds config in the chart according to the new configuration fields. It's a small change with minor friction which I believe we can live with while we have a limited number of MDS users (on-prem)

Copy link
Copy Markdown
Contributor

@talSofer talSofer May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make sure to include it in the 1.10.0 release notes / change log (probably retroactively)

@N-o-Z
Copy link
Copy Markdown
Member Author

N-o-Z commented May 11, 2026

This PR is pending:
https://github.com/treeverse/cloud-controlplane/pull/5367
treeverse/charts#395

@N-o-Z N-o-Z requested a review from talSofer May 11, 2026 00:54
talSofer
talSofer approved these changes May 11, 2026
View reviewed changes
Comment thread docs/src/datamanagment/metadata-search.md
3. Install or upgrade the Helm chart with the updated configuration.

!!! note
From [`lakefs` Helm chart](https://github.com/treeverse/charts) version **1.10.0**, the Metadata Search
Copy link
Copy Markdown
Contributor

@talSofer talSofer May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make sure to include it in the 1.10.0 release notes / change log (probably retroactively)

Comment thread docs/src/datamanagment/metadata-search.md Outdated
Use branch name prefixes (e.g., `feature-*`) to reduce the need for manual updates when new branches are added.

!!! warning "Validation"
The Metadata Search service refuses to start unless `lakefs_mds_endpoint` is non-empty and `period`,
Copy link
Copy Markdown
Contributor

@talSofer talSofer May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to suggest that we clarify that this condition holds for versions >= 1.10.0

N-o-Z and others added 2 commits May 11, 2026 15:57
@N-o-Z
Copy link
Copy Markdown
Member Author

N-o-Z commented May 14, 2026

Closing in favor of: https://github.com/treeverse/docs-next-lakeFS/pull/2

@N-o-Z N-o-Z closed this May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@talSofer talSofer talSofer approved these changes

@nopcoder nopcoder Awaiting requested review from nopcoder

@guy-har guy-har Awaiting requested review from guy-har

Assignees

@N-o-Z N-o-Z

Labels

docs Improvements or additions to documentation exclude-changelog PR description should not be included in next release changelog minor-change Used for PRs that don't require issue attached mostly-ai

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@N-o-Z @talSofer