Skip to content

Use Mongodb replica set and transactions#4198

Merged
imnasnainaec merged 15 commits intomasterfrom
mongodb-replica-set-and-transactions
Mar 19, 2026
Merged

Use Mongodb replica set and transactions#4198
imnasnainaec merged 15 commits intomasterfrom
mongodb-replica-set-and-transactions

Conversation

@imnasnainaec
Copy link
Copy Markdown
Collaborator

@imnasnainaec imnasnainaec commented Mar 13, 2026

Resolves #4183

Combines #4184, #4194, and #4196 for full testing and review.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

https://app.devin.ai/review/sillsdev/TheCombine/pull/4198


This change is Reviewable

Summary by CodeRabbit

  • New Features

    • Single-node MongoDB replica set support and new DB startup script; import/merge/revert workflows for words via service APIs.
  • Bug Fixes

    • More reliable DB startup, replica-set initialization and shutdown across local, container and helm deployments.
  • Refactor

    • Repository frontier operations converted to transactional, action-driven workflows; controllers/services updated to use service-level word operations; undo/restore endpoints return standard HTTP statuses.
  • Tests

    • Added comprehensive integration tests and an ephemeral MongoDB test runner.
  • Documentation

    • Updated MongoDB connection and deployment/maintenance instructions.

@imnasnainaec imnasnainaec self-assigned this Mar 13, 2026
@imnasnainaec imnasnainaec added the 🟨Medium Medium-priority PR label Mar 13, 2026
@github-actions github-actions bot added documentation Improvements or additions to documentation backend frontend deployment docker test goal: MergeDup import/export audio project dependencies Pull requests that update a dependency file github_actions Pull requests that update GitHub Actions code labels Mar 13, 2026
@github-actions

This comment has been minimized.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 13, 2026

📝 Walkthrough

Walkthrough

Adds MongoDB replica-set startup and local orchestration, exposes explicit transaction APIs on the Mongo DB context, refactors frontier operations to action-driven, transactional flows across repository and service layers, adds an ephemeral Mongo test runner and integration tests, updates controllers/interfaces/frontend contracts, and replaces local DB startup scripts.

Changes

Cohort / File(s) Summary
Mongo replica-set & deployment
database/Dockerfile, database/init/00-replica-set.js, deploy/helm/thecombine/charts/database/templates/database.yaml, Backend/appsettings.json, README.md
Enable rs0 replica set, add replica-set init script, add Helm postStart lifecycle and env to advertise pod IP, bump Mongo image, and update connection strings to include replicaSet=rs0.
Local DB startup & tooling
.gitignore, .vscode/tasks.json, scripts/setupMongo.js, scripts/startDatabase.js, package.json
Replace simple startup with scripts/startDatabase.js (mongod + replSet orchestration), update npm database script and VS Code task, remove old setup helper and update .gitignore.
IMongoDbContext & transactions
Backend/Interfaces/IMongoDbContext.cs, Backend/Contexts/MongoDbContext.cs
Replace disposable context pattern with explicit transaction API: add IMongoTransaction, BeginTransaction, ExecuteInTransaction, ExecuteInTransactionAllowNull, and implement a transaction wrapper in the context.
Repository frontier refactor
Backend/Interfaces/IWordRepository.cs, Backend/Repositories/WordRepository.cs
Refactor frontier operations to action-driven, session/transactional flows: change DeleteFrontier to accept action callbacks, add RestoreFrontier, UpdateFrontier overloads, ReplaceFrontier, RevertReplaceFrontier, and session-backed helper methods.
WordService API & behavior
Backend/Interfaces/IWordService.cs, Backend/Services/WordService.cs
Add ImportWords, MergeReplaceFrontier, RevertMergeReplaceFrontier, single-item RestoreFrontierWord, timestamp/edit helpers, and factory actions that drive repository mutations.
Lift & Merge wiring
Backend/Interfaces/ILiftService.cs, Backend/Services/LiftService.cs, Backend/Controllers/LiftController.cs, Backend/Services/MergeService.cs
Switch Lift importer/exporter and merge/undo flows to use IWordService (instead of IWordRepository) and delegate merge/undo via WordService methods.
Controller response changes
Backend/Controllers/WordController.cs, Backend/Controllers/MergeController.cs
Change RestoreWord and UndoMerge endpoints to return Ok()/NotFound() (remove boolean payloads) and update response metadata and docs.
Frontend API wrappers & client
src/api/api/merge-api.ts, src/api/api/word-api.ts, src/backend/index.ts
Change undoMerge() and restoreWord() return types from boolean to void and update client wrappers to await without returning payloads.
Test infra & mocks
Backend.Tests/Backend.Tests.csproj, Backend.Tests/Mocks/MongoDbContextMock.cs, Backend.Tests/Mocks/WordRepositoryMock.cs, Backend.Tests/Repositories/MongoDbTestRunner.cs
Add ephemeral mongod test runner and platform runtime refs, add MongoDbContextMock with basic transaction support, and expand WordRepositoryMock to implement new action-based frontier API.
Integration & unit tests
Backend.Tests/Repositories/WordRepositoryTests.cs, Backend.Tests/... (Controllers/Services test files)
Add comprehensive integration tests for repository transactional flows; update many unit tests to use WordRepositoryMock, adjust async patterns and assertions.
Startup & config
Backend/Startup.cs, Backend/appsettings.json
Resolve admin repo from a new scoped provider during startup and append replicaSet=rs0 to Mongo connection strings.
Docs
docs/deploy/README.md, README.md
Update maintenance script paths and local inspect DB example to include ?replicaSet=rs0.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant WordService as WordService
    participant WordRepository as WordRepository
    participant MongoDbContext as MongoDbContext
    participant MongoDB as MongoDB

    Note over Client,MongoDB: Transactional frontier update flow (high-level)

    Client->>WordService: MergeReplaceFrontier / Update(userId, Word)
    WordService->>WordRepository: ReplaceFrontier / UpdateFrontier (passes action callbacks)
    WordRepository->>MongoDbContext: ExecuteInTransaction(operation)
    MongoDbContext->>MongoDB: BeginTransaction (start session)
    MongoDbContext->>MongoDB: Execute operation with session (modify frontier, update words)
    MongoDB-->>MongoDbContext: Operation result
    MongoDbContext->>MongoDB: CommitTransactionAsync()
    MongoDB-->>MongoDbContext: Commit confirmation
    MongoDbContext-->>WordRepository: result
    WordRepository-->>WordService: result
    WordService-->>Client: Ok / result
Loading

Estimated Code Review Effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • #4161 — Changes touching RestoreFrontier(s), WordRepository/WordRepositoryMock and WordService restore/merge APIs.
  • #4091 — Related refactor of LiftController/ILiftService/LiftService dependency shifting from repository to service.
  • #4148 — Related changes to frontier retrieval APIs and repository/frontier method updates.

Suggested reviewers

  • jasonleenaylor

"🐰✨ A Rabbit's Ode to Transactional Bliss
I hopped through replSet fields and scurried round the logs,
I stitched the frontier bits in sessions, neat as clogs.
When deletes become restores and replaces flip their fate,
The tests all cheer and hop along — transactions celebrate!"

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.92% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main focus of the changeset: converting MongoDB to use replica set and implementing transactions.
Linked Issues check ✅ Passed All coding requirements from issue #4183 are addressed: MongoDB replica set configuration, transaction support, and validation of both npm start and Docker-Kubernetes deployments.
Out of Scope Changes check ✅ Passed All changes align with the objective to enable MongoDB transactions via replica set migration. Test infrastructure updates and API adjustments support this core goal.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch mongodb-replica-set-and-transactions
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@imnasnainaec imnasnainaec force-pushed the mongodb-replica-set-and-transactions branch from f1a173d to d389fb3 Compare March 13, 2026 19:30
@imnasnainaec imnasnainaec force-pushed the mongodb-replica-set-and-transactions branch from d389fb3 to ec53acb Compare March 13, 2026 19:33
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 96.44444% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.70%. Comparing base (c6afad4) to head (7e705ee).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
Backend/Repositories/WordRepository.cs 95.62% 4 Missing and 2 partials ⚠️
Backend/Controllers/MergeController.cs 0.00% 1 Missing ⚠️
Backend/Services/WordService.cs 98.68% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4198      +/-   ##
==========================================
+ Coverage   75.15%   75.70%   +0.55%     
==========================================
  Files         302      303       +1     
  Lines       11099    11332     +233     
  Branches     1394     1411      +17     
==========================================
+ Hits         8341     8579     +238     
+ Misses       2357     2352       -5     
  Partials      401      401              
Flag Coverage Δ
backend 87.23% <96.44%> (+0.71%) ⬆️
frontend 66.34% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@imnasnainaec imnasnainaec removed the github_actions Pull requests that update GitHub Actions code label Mar 13, 2026
@imnasnainaec imnasnainaec marked this pull request as ready for review March 13, 2026 19:48
coderabbitai[bot]

This comment was marked as resolved.

@imnasnainaec imnasnainaec force-pushed the mongodb-replica-set-and-transactions branch from 47a7db2 to d457901 Compare March 16, 2026 13:42
coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as duplicate.

Copy link
Copy Markdown
Contributor

@jasonleenaylor jasonleenaylor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonleenaylor reviewed 38 files and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on imnasnainaec).


Backend/Repositories/WordRepository.cs line 384 at r6 (raw file):

        }

        // WITH-SESSION HELPER METHODS

Either remove this ugly comment, or alternatively turn it into a #region if you want to document this group of methods idiomatically.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
Backend/Repositories/WordRepository.cs (1)

336-338: ⚠️ Potential issue | 🟡 Minor

XML exception contract still overstates missing-id failures in replace flow.

The <exception> docs still say missing old IDs throw, but replacements run with createIfNotFound: true (Line 549), so some “missing old id” cases intentionally create instead of throwing. Please narrow these exception docs to guaranteed throw cases only.

✏️ Suggested doc fix
-        /// Thrown when an old word id doesn't exist in Frontier or a replacement word has a different project id.
+        /// Thrown when a replacement word has a different project id, or when a delete-target old id
+        /// cannot be found in Frontier.

Based on learnings: In Backend/Repositories/WordRepository.cs, ReplaceFrontierWithSession intentionally allows silent create/replace behavior via createIfNotFound: true; this behavior is accepted by the maintainer.

Also applies to: 531-532

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Backend/Repositories/WordRepository.cs` around lines 336 - 338, The XML
<exception> on ReplaceFrontierWithSession overstates missing-old-id failures:
update the documentation for the ReplaceFrontierWithSession method (and any
duplicate exception tags for the same method/overload) to only document
guaranteed throw cases — e.g., when a replacement word has a different ProjectId
— and remove or rephrase the claim that missing old IDs always throw, since the
implementation calls createIfNotFound: true and may create missing entries
instead.
🧹 Nitpick comments (1)
Backend.Tests/Repositories/WordRepositoryTests.cs (1)

349-368: Consider extracting repeated “repo unchanged” assertions into a helper.

Several tests repeat the same post-failure state checks; a small helper would reduce duplication and make intent clearer.

♻️ Optional refactor sketch
+        private async Task AssertRepoUnchanged(
+            string expectedOnlyWordId,
+            int expectedFrontierCount = 1)
+        {
+            Assert.That((await _repo.GetAllWords(_projectId)).Select(w => w.Id),
+                Is.EqualTo([expectedOnlyWordId]));
+            Assert.That(await _repo.GetFrontierCount(_projectId), Is.EqualTo(expectedFrontierCount));
+            Assert.That(await _repo.IsInFrontier(_projectId, expectedOnlyWordId), Is.True);
+        }

Also applies to: 432-451, 566-587, 590-602, 680-696

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Backend.Tests/Repositories/WordRepositoryTests.cs` around lines 349 - 368,
Multiple tests repeat the same post-failure assertions checking the repo was
unchanged; extract those assertions into a reusable helper to reduce
duplication. Create a private helper method (e.g.,
AssertRepositoryUnchangedAfterFailedOperation) that takes the project id and
word id (and optionally the original created word) and performs the checks
currently repeated: calling _repo.GetAllWords(_projectId) and comparing Ids,
_repo.GetFrontierCount(_projectId), _repo.GetFrontier(_projectId, createdId) and
asserting frontierWord is not null and frontierWord.Accessibility is not
Status.Deleted; then replace the duplicated assertion blocks in
TestDeleteFrontierModifyActionThrowsLeavesRepoUnchanged and the other indicated
tests with a single call to this helper. Use the existing symbols CreateWord,
_repo.DeleteFrontier, _repo.GetAllWords, GetFrontierCount, and GetFrontier to
implement and verify the helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@Backend/Repositories/WordRepository.cs`:
- Around line 336-338: The XML <exception> on ReplaceFrontierWithSession
overstates missing-old-id failures: update the documentation for the
ReplaceFrontierWithSession method (and any duplicate exception tags for the same
method/overload) to only document guaranteed throw cases — e.g., when a
replacement word has a different ProjectId — and remove or rephrase the claim
that missing old IDs always throw, since the implementation calls
createIfNotFound: true and may create missing entries instead.

---

Nitpick comments:
In `@Backend.Tests/Repositories/WordRepositoryTests.cs`:
- Around line 349-368: Multiple tests repeat the same post-failure assertions
checking the repo was unchanged; extract those assertions into a reusable helper
to reduce duplication. Create a private helper method (e.g.,
AssertRepositoryUnchangedAfterFailedOperation) that takes the project id and
word id (and optionally the original created word) and performs the checks
currently repeated: calling _repo.GetAllWords(_projectId) and comparing Ids,
_repo.GetFrontierCount(_projectId), _repo.GetFrontier(_projectId, createdId) and
asserting frontierWord is not null and frontierWord.Accessibility is not
Status.Deleted; then replace the duplicated assertion blocks in
TestDeleteFrontierModifyActionThrowsLeavesRepoUnchanged and the other indicated
tests with a single call to this helper. Use the existing symbols CreateWord,
_repo.DeleteFrontier, _repo.GetAllWords, GetFrontierCount, and GetFrontier to
implement and verify the helper.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c3549509-20db-44df-95cc-fa7d5b09ac2d

📥 Commits

Reviewing files that changed from the base of the PR and between 7f1fabe and 7e705ee.

📒 Files selected for processing (2)
  • Backend.Tests/Repositories/WordRepositoryTests.cs
  • Backend/Repositories/WordRepository.cs

@imnasnainaec imnasnainaec merged commit 6932b0d into master Mar 19, 2026
22 of 23 checks passed
@imnasnainaec imnasnainaec deleted the mongodb-replica-set-and-transactions branch March 19, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

audio backend dependencies Pull requests that update a dependency file deployment docker documentation Improvements or additions to documentation frontend goal: MergeDup import/export 🟨Medium Medium-priority PR project test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[mongodb] Convert from standalone to a replica set

3 participants