Skip to content

feat: migrate project storage from data.json to vault files#2324

Open
Emt-lin wants to merge 2 commits intomasterfrom
worktree-project-storage-migration
Open

feat: migrate project storage from data.json to vault files#2324
Emt-lin wants to merge 2 commits intomasterfrom
worktree-project-storage-migration

Conversation

@Emt-lin
Copy link
Copy Markdown
Collaborator

@Emt-lin Emt-lin commented Mar 23, 2026

Summary

  • Migrate project definitions and metadata from data.json (settings.projectList) to individual markdown files in the vault's projects directory
  • Add ProjectFileManager for CRUD operations with YAML frontmatter parsing/persistence
  • Add ProjectRegister for vault-event reconciliation (create/rename/delete/modify)
  • Add migration logic from legacy format with unsupported/ backup for failed entries
  • Add project UI components: AddProjectModal, ProjectList, ProgressCard, ProcessingStatus
  • Integrate chat history with new file-based project storage model
  • Add status sorting, cache-first ordering, and URL removal in processing pipeline
  • Use atomic cache operations (replaceCachedProjectRecordByFilePath) to prevent transient disappearance on rename
  • Await context cache cleanup on all delete/rename/folder-switch paths to prevent same-ID race
  • Parse frontmatter from raw file content first, fall back to metadataCache only when no frontmatter block exists
  • Add BOM/CRLF support in readFrontmatterViaAdapter and patchFrontmatter
  • Add aria-label to all icon-only buttons for accessibility

Test plan

  • Unit tests pass (106 suites, 1938 tests)
  • ESLint passes with 0 errors
  • Verify fresh install: no legacy projects → no migration dialog
  • Verify migration: existing data.json projects migrate to vault files correctly
  • Verify failed migration: unsupported projects backed up to unsupported/ folder
  • Verify project CRUD: create, edit, rename, delete projects via UI
  • Verify external file operations: rename/delete project files outside Obsidian
  • Verify folder switch: change projects folder in settings, confirm reload
  • Verify chat isolation: project chats remain separate from non-project chats
  • Verify hidden folder support: projects in dot-prefixed folders work correctly

Actual Effect

2026-03-22.00.20.26.mp4

@Emt-lin Emt-lin requested a review from logancyang March 23, 2026 10:27
@Emt-lin Emt-lin force-pushed the worktree-project-storage-migration branch from 1419138 to d8e0d55 Compare March 23, 2026 10:29
@Emt-lin
Copy link
Copy Markdown
Collaborator Author

Emt-lin commented Mar 23, 2026

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8e0d55fe6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/projects/projectRegister.ts
Comment thread src/utils/chatHistoryUtils.ts
@Emt-lin
Copy link
Copy Markdown
Collaborator Author

Emt-lin commented Mar 24, 2026

@codex review

Migrate project definitions and metadata from data.json to individual
markdown files in the vault's projects directory.

- Project file manager with YAML frontmatter parsing and persistence
- Migration logic from legacy data.json format with unsupported/ backup
- Project UI components (AddProjectModal, ProjectList, ProgressCard)
- Chat history integration for the new storage model
- Status sorting, cache-first ordering, and URL removal in processing
- Atomic cache operations to prevent transient disappearance on rename
- Awaited context cache cleanup on all delete/rename/folder-switch paths
- Raw-content-first frontmatter parsing to avoid stale metadataCache
- BOM/CRLF support in frontmatter read and write helpers
- Accessibility: aria-labels on all icon-only buttons
@Emt-lin Emt-lin force-pushed the worktree-project-storage-migration branch from d8e0d55 to ddbe5bf Compare March 24, 2026 02:23
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8e0d55fe6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/projects/projectRegister.ts
Comment thread src/projects/projectUtils.ts Outdated
@Emt-lin
Copy link
Copy Markdown
Collaborator Author

Emt-lin commented Mar 24, 2026

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 159d74f0da

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/projects/ProjectFileManager.ts
@Emt-lin
Copy link
Copy Markdown
Collaborator Author

Emt-lin commented Mar 24, 2026

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 159d74f0da

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/projects/ProjectFileManager.ts
Comment thread src/projects/projectMigration.ts Outdated
@Emt-lin
Copy link
Copy Markdown
Collaborator Author

Emt-lin commented Mar 24, 2026

@codex review

- Defer vault listener registration from constructor to initialize()
  to avoid premature cache mutations and frontmatter writes during
  initial vault load (before onLayoutReady).

- Coerce numeric YAML project IDs to strings so bare numeric values
  like `copilot-project-id: 123` are not silently dropped.
@Emt-lin Emt-lin force-pushed the worktree-project-storage-migration branch from 159d74f to 7c3f04f Compare March 24, 2026 03:28
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant