chore(#10455): update CouchDB version to 3.5.1 by shashi-sah2003 · Pull Request #10501 · medic/cht-core

shashi-sah2003 · 2025-11-28T16:24:31Z

fixes #10455

Code review checklist

UI/UX backwards compatible: Test it works for the new design (enabled by default). And test it works in the old design, enable can_view_old_navigation permission to see the old design. Test it has appropriate design for RTL languages.
Readable: Concise, well named, follows the style guide, documented if necessary.
Documented: Configuration and user documentation on cht-docs
Tested: Unit and/or e2e where appropriate
Internationalised: All user facing text
Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

shashi-sah2003 · 2025-11-28T16:25:21Z

@dianabarsan this PR is ready for review

github-actions · 2025-12-29T00:07:26Z

This PR is now marked "stale" after 30 days without activity. It will be closed automatically in 10 days unless you add a comment, push new changes or remove the "stale" label.

dianabarsan

As stated in this comment, we should make an assessment at least on Nouveau performance changes in the new version.

andrablaj · 2026-01-12T12:45:29Z

@shashi-sah2003, would you be available to work on the requirements mentioned by @dianabarsan above? We would love to include this issue in the upcoming CHT release.

… search improvements

shashi-sah2003 · 2026-01-19T16:54:38Z

@dianabarsan I have added benchmarking setup, didn't get desirable results (maybe I am missing something). Go through the README file for setting up the setup. lmk what do you think about this? If there is anything wrong in my setup do lmk

dianabarsan · 2026-01-20T07:03:26Z

All the benchmarking tests already had a framework to be run: https://github.com/medic/cht-core/blob/master/tests/scalability/run_suite.sh#L111 . They are run through the scalability suite, so there was no need to setup a test data generator or anything like that.
So all was needed was adding the Nouveau benchmark tests, and only for the running version. Then we would just run the tests for master and for your updated branch and compare results.

If it's easier, I can setup the Nouveau benchmark on master so there is already the framework in place.

In terms of favorable results, can you share what was unfavorable?

shashi-sah2003 · 2026-01-21T17:32:34Z

@dianabarsan for the results what I saw locally with my current setup is below, is it possible that I am using less data or something else?

[key:_all] queries: 3.5.0 = 11-16ms → 3.5.1 = 46-48ms (3-4x slower)
[type:person] large queries: 3.5.0 = 82ms → 3.5.1 = 176ms (2x slower)
[type:data_record] medium: 3.5.0 = 107ms → 3.5.1 = 89ms (18% faster)

also is the Nouveau benchmark merged on master?

The scalability suite (run_suite.sh) already has framework for: - Setting up CHT environment - Using test-data-generator for population - Running benchmarks via node index.js - Uploading results Removed custom Docker infrastructure as benchmarks only need to run against the actual CHT deployment, not standalone instances.

shashi-sah2003 · 2026-01-28T04:14:06Z

@dianabarsan

  await printResults('nouveau_index', await testNouveauIndex());

this is taking too long, I have left it for around 40 minutes still it didn't complete, is it expected to happen? Except this nouveau_index all other keys are returning benchmark results. refer below:

CouchDb Performance benchmark

Database Info

CouchDb version: 3.5.1

Database doc count: 903030

_changes benchmark

limit	since	include_docs	seq_interval	keys	Duration (ms)
100					12
1000					70
10000					621
100		true			20
1000		true			104
10000		true			1046
100			100		25
1000			1000		15
10000			10000		106
100	Mid Sequence				12
1000	Mid Sequence				65
10000	Mid Sequence				603
100	Mid Sequence	true			15
1000	Mid Sequence	true			102
10000	Mid Sequence	true			922
100	Mid Sequence		100		28
1000	Mid Sequence		1000		15
10000	Mid Sequence		10000		112
10000				true	1379
10000		true		true	2029
10000			10000	true	968

_all_docs benchmark

limit	include_docs	keys	Duration (ms)
100			16
1000			23
10000			198
100	true		12
1000	true		84
10000	true		627
10000		true	1130
10000	true	true	1262

_bulk_get benchmark

count	Duration (ms)
100	10
1000	41
10000	632

view benchmark

limit	include_docs	keys	Duration (ms)
100			226
1000			27
10000			217
100	true		86
1000	true		742
10000	true		6112
100		true	60
1000		true	386
10000		true	4199
10000	true	true	5501

dianabarsan · 2026-01-28T08:06:03Z

this is taking too long, I have left it for around 40 minutes still it didn't complete, is it expected to happen? Except this nouveau_index all other keys are returning benchmark results. refer below:

No, that should not happen, but to be honest I don't know what your setup is.
besides, the values that you shared are not necessarily relevant, because we want to compare them with past benchmarks. The whole point of those is that they are always run in the same type of aws machine, so we make sure that we are comparing apples to apples.

What I believe the next steps should be:

removing all the scalability/couchdb benchmark code, that is not necessary as we already have the architecture in place to generate data and run the tests in CI
merge master branch in your branch so you get the latest scalability tests
fix the build, only a green build can run scalability tests (the docker images need to get published, and that is last step of a successful build)

Once the build is green, I will create a new branch off of your branch and run scalability tests on it - these tests need to be manually triggered on a branch that is in the medic repo.
Once we compare the scalability results and everything is good, we merge your PR into master.

Thanks a lot for your patience!

dianabarsan

per above.

…rvices

shashi-sah2003 · 2026-01-29T02:56:21Z

@dianabarsan I have pushed the setup which I am trying to do fyr: https://github.com/medic/cht-core/pull/10501/changes

Steps that I am following:

docker compose -f docker-compose-test.yml up -d (at root location)
export COUCH_URL=http://admin:password@localhost:5984/not-medic
curl -X PUT http://admin:password@localhost:5984/not-medic
cd /tmp/test-data-generator
npm run generate cht-core/tests/scalability/couchdb-benchmark/tdg-design.js
cd cht-core/tests/scalability/couchdb-benchmark
node index.js

removing all the scalability/couchdb benchmark code, that is not necessary as we already have the architecture in place to generate data and run the tests in CI
merge master branch in your branch so you get the latest scalability tests

was already running with the above mentioned steps locally

fix the build, only a green build can run scalability tests (the docker images need to get published, and that is last step of a successful build)

I will see what is making the build to fail tho it's just a version upgrade.

EDIT: Hmm interesting the build are failing due to couchdb containers aren't able to spin up with version 3.5.1, I believe this is happening due to the support of HTTP/2 in this version? Maybe we have to reconfigure it to use a nouveau configuration yaml file (attached in this pr) for this to spin up properly, what do you think?

dianabarsan · 2026-01-29T10:20:46Z

Maybe we have to reconfigure it to use a nouveau configuration yaml file (attached in this pr) for this to spin up properly, what do you think?

The is the reported error:

[error] 2026-01-29T03:16:14.184935Z couchdb@couchdb-3.local <0.502.0> -------- nouveau_index_manager: db:medic-test ddoc:_design/medic index:contacts_by_freetext failed with: {error,{connection_error,{protocol_error,'Invalid connection preface received. Appears to be an HTTP/1 response? (RFC7540 3.5)'}}}

If you believe additional nouveau config will help, this is a quick way to add it (see the configs section): https://docs.couchdb.org/en/stable/install/nouveau.html#compose

The files to change would be:
https://github.com/medic/cht-core/blob/master/scripts/build/cht-couchdb-cluster.yml.template#L72
https://github.com/medic/cht-core/blob/master/scripts/build/cht-couchdb-single-node.yml.template#L25

Let's give it a go to see if tests pass and then we can polish it.

dianabarsan

per above

shashi-sah2003 · 2026-01-29T16:30:22Z

The is the reported error:
[error] 2026-01-29T03:16:14.184935Z couchdb@couchdb-3.local <0.502.0> -------- nouveau_index_manager: db:medic-test ddoc:_design/medic index:contacts_by_freetext failed with: {error,{connection_error,{protocol_error,'Invalid connection preface received. Appears to be an HTTP/1 response? (RFC7540 3.5)'}}}

yes because of these errors only I have to add this config https://github.com/medic/cht-core/pull/10501/changes#diff-8b77ae47b921cddcb6cad23386ce670c0bf1caa17a7e37df394c21ce756a5aed

If you believe additional nouveau config will help, this is a quick way to add it (see the configs section): https://docs.couchdb.org/en/stable/install/nouveau.html#compose

will add these now, and let you know

EDIT: have pushed the changes using noveau config

…e cluster and single-node templates to include new config structure

…de new config structure

…e obsolete config file

shashi-sah2003 · 2026-02-02T18:02:45Z

@dianabarsan have added more detailed config to avoid Invalid connection preface received. Appears to be an HTTP/1 response. This should pass now, also verify once if all configs look correct

dianabarsan · 2026-02-04T12:56:13Z

I made a copy of your branch and started scalability tests and I don't have super good news. The scalability tests are timing out: they start an ec2 instance that is scheduled to shut down automatically within an hour. running the tests on this branch exceeds one hour. I am now running the tests a second time and noticing this behavior. It seems that the indexing of the nouveau views is taking much longer (it has been over 35 minutes of indexing and it's still not done, so indexing is really much much much slower)

I;m wondering whether your nouveau config is messing up with the performance. I'm referring specifically to:

  nouveau-config:
    content: |
      maxIndexesOpen: 3000
      commitIntervalSeconds: 30
      idleSeconds: 60

Where did you get these values from? Can they be removed?

dianabarsan

per above

shashi-sah2003 · 2026-02-04T17:40:41Z

Where did you get these values from?

from here: https://docs.couchdb.org/en/stable/install/nouveau.html, I thought increasing the value of maxIndexesOpen might help with large test dataset

Can they be removed?

Yeah we could use the default values here as well, idk if that would help but let's give it a try.

PS: what is this UI you are using? looks good for visualization :)

shashi-sah2003 · 2026-02-04T19:37:46Z

didn't see any effect locally, checked the logs seems like indexing is being done correctly

172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F00000000-7fffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/b741404d-60b2-44d9-b971-50ea92dcdca4 HTTP/2.0" 200 11 "-" "-" 0
172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/4009b50f-8169-4080-a7d1-f836fbd4dd57 HTTP/2.0" 200 11 "-" "-" 41
172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F00000000-7fffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/b7b31ab3-5334-4181-a5d0-f2e9849edc77 HTTP/2.0" 200 11 "-" "-" 41
172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/400b0d58-cf79-4ea8-b8e8-4efe63fb96e3 HTTP/2.0" 200 11 "-" "-" 41
172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/4040e90a-3830-46dc-9dec-d7eef8db719e HTTP/2.0" 200 11 "-" "-" 43
172.20.0.3 - - [04/Feb/2026:19:33:11 +0000] "PUT /index/nonode%40nohost%2Fshards%2F00000000-7fffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/b7f5fdf6-41c4-4cb8-aafc-a029e69f11ef HTTP/2.0" 200 11 "-" "-" 43
172.20.0.3 - - [04/Feb/2026:19:33:12 +0000] "PUT /index/nonode%40nohost%2Fshards%2F00000000-7fffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/b8bc1b4a-3149-47bc-8bdd-199e5288fc27 HTTP/2.0" 200 11 "-" "-" 2
172.20.0.3 - - [04/Feb/2026:19:33:12 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/41e731f5-7aff-45c4-9d54-c810c35718b0 HTTP/2.0" 200 11 "-" "-" 43
172.20.0.3 - - [04/Feb/2026:19:33:12 +0000] "PUT /index/nonode%40nohost%2Fshards%2F00000000-7fffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/b926506b-96a4-42fa-a9e9-f33db62a27b5 HTTP/2.0" 200 11 "-" "-" 41
172.20.0.3 - - [04/Feb/2026:19:33:12 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/422f69b1-49e4-457e-896c-9fe1bd5a84b9 HTTP/2.0" 200 11 "-" "-" 43
172.20.0.3 - - [04/Feb/2026:19:33:12 +0000] "PUT /index/nonode%40nohost%2Fshards%2F80000000-ffffffff%2Fnot-medic.1770228770%2F55eeac8224bbdb2cf87f3208d50f3acb86d83116c913716dc377753dd5b61375/doc/42f0ffbf-2283-4587-9d90-a7661f5c9541 HTTP/2.0" 200 11 "-" "-" 42

Database doc count: 903030 is it because of this many docs that it is taking time?

dianabarsan · 2026-02-05T05:04:49Z

Database doc count: 903030 is it because of this many docs that it is taking time?

The doc count is the same as when I ran the tests multiple times. Indexing 1kk docs should not take so long, there's something definitely wrong happening here.
I'll rerun the tests once the build passes again,

shashi-sah2003 · 2026-02-05T14:10:23Z

I'll rerun the tests once the build passes again

I'm wondering how removing that optional config is leading to ci failure 🤔. Also they all are failing with different issues.

dianabarsan · 2026-02-05T14:21:30Z

I kept restarting it. 🤞🏻
after it's green I'll try to run the scalability tests again

shashi-sah2003 · 2026-02-05T14:56:14Z

I don't think they are flaky, failing everytime with same error. ;( Btw did you try running scalability tests on master? Just wanted to see if this issue is related to this feature branch only

dianabarsan · 2026-02-05T15:02:56Z

Btw did you try running scalability tests on master?

yes. medic/scalability-results@085a1f4

shashi-sah2003 · 2026-02-05T16:01:43Z

Hmm this is tricky, according to this it shouldn't take much time for 3.5.1 version, there is something very wrong with our configs

github-actions · 2026-03-08T00:08:19Z

This PR is now marked "stale" after 30 days without activity. It will be closed automatically in 10 days unless you add a comment, push new changes or remove the "stale" label.

dianabarsan · 2026-03-08T06:24:49Z

closing due to #10455 (comment)

Update CouchDB version to 3.5.1

d922388

shashi-sah2003 changed the title ~~chore(#10455): Update CouchDB version to 3.5.1~~ chore(#10455): update CouchDB version to 3.5.1 Nov 28, 2025

github-actions Bot added the Stale label Dec 29, 2025

andrablaj removed the Stale label Dec 29, 2025

andrablaj requested a review from dianabarsan January 7, 2026 12:11

dianabarsan reviewed Jan 12, 2026

View reviewed changes

shashi-sah2003 and others added 3 commits January 16, 2026 11:53

Merge branch 'master' into 10455-upgrade-couchdb

b931b7f

feat: add benchmarking setup for CouchDB 3.5.0 and 3.5.1 with Nouveau…

54fa8c0

… search improvements

Merge branch 'master' into 10455-upgrade-couchdb

a3492eb

github-advanced-security AI found potential problems Jan 19, 2026

View reviewed changes

Comment thread tests/scalability/couchdb-benchmark/populate_test_data.js Fixed

shashi-sah2003 mentioned this pull request Jan 19, 2026

Upgrade to CouchDb 3.5.1 #10455

Closed

shashi-sah2003 and others added 3 commits January 21, 2026 23:04

Merge branch 'master' into 10455-upgrade-couchdb

e5ec2b6

Merge branch 'master' into 10455-upgrade-couchdb

757b89b

shashi-sah2003 requested a review from dianabarsan January 28, 2026 04:19

dianabarsan reviewed Jan 28, 2026

View reviewed changes

feat: add docker-compose and configuration for CouchDB and Nouveau se…

8ab7867

…rvices

shashi-sah2003 requested a review from dianabarsan January 29, 2026 02:57

dianabarsan requested changes Jan 29, 2026

View reviewed changes

Comment thread docker-compose-test.yml Outdated

shashi-sah2003 and others added 4 commits January 29, 2026 22:20

chore: remove obsolete CouchDB and Nouveau configuration files; updat…

e35abee

…e cluster and single-node templates to include new config structure

Merge branch 'master' into 10455-upgrade-couchdb

555a198

feat: add nouveau configuration and update CouchDB templates to inclu…

f45c558

…de new config structure

feat: consolidate nouveau configuration into YAML templates and remov…

8a21a8b

…e obsolete config file

shashi-sah2003 requested a review from dianabarsan February 3, 2026 03:18

dianabarsan requested changes Feb 4, 2026

View reviewed changes

feat: remove obsolete configuration parameters from CouchDB templates

c9445a5

github-actions Bot added the Stale label Mar 8, 2026

github-actions Bot removed the Stale label Mar 9, 2026

dianabarsan closed this Apr 1, 2026

Uh oh!

Conversation

shashi-sah2003 commented Nov 28, 2025

Code review checklist

License

Uh oh!

shashi-sah2003 commented Nov 28, 2025

Uh oh!

github-actions Bot commented Dec 29, 2025

Uh oh!

dianabarsan left a comment

Choose a reason for hiding this comment

Uh oh!

andrablaj commented Jan 12, 2026

Uh oh!

shashi-sah2003 commented Jan 19, 2026

Uh oh!

Uh oh!

dianabarsan commented Jan 20, 2026

Uh oh!

shashi-sah2003 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shashi-sah2003 commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CouchDb Performance benchmark

Database Info

_changes benchmark

_all_docs benchmark

_bulk_get benchmark

view benchmark

Uh oh!

dianabarsan commented Jan 28, 2026

Uh oh!

dianabarsan left a comment

Choose a reason for hiding this comment

Uh oh!

shashi-sah2003 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dianabarsan commented Jan 29, 2026

Uh oh!

dianabarsan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shashi-sah2003 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shashi-sah2003 commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dianabarsan commented Feb 4, 2026

Uh oh!

dianabarsan left a comment

Choose a reason for hiding this comment

Uh oh!

shashi-sah2003 commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shashi-sah2003 commented Feb 4, 2026

Uh oh!

dianabarsan commented Feb 5, 2026

Uh oh!

shashi-sah2003 commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dianabarsan commented Feb 5, 2026

Uh oh!

shashi-sah2003 commented Feb 5, 2026

Uh oh!

dianabarsan commented Feb 5, 2026

Uh oh!

shashi-sah2003 commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 8, 2026

Uh oh!

dianabarsan commented Mar 8, 2026

Uh oh!

shashi-sah2003 commented Jan 21, 2026 •

edited

Loading

shashi-sah2003 commented Jan 28, 2026 •

edited

Loading

shashi-sah2003 commented Jan 29, 2026 •

edited

Loading

shashi-sah2003 commented Jan 29, 2026 •

edited

Loading

shashi-sah2003 commented Feb 2, 2026 •

edited

Loading

shashi-sah2003 commented Feb 4, 2026 •

edited

Loading

shashi-sah2003 commented Feb 5, 2026 •

edited

Loading

shashi-sah2003 commented Feb 5, 2026 •

edited

Loading