perf: rsa sign on slacc#17322
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #17322 +/- ##
===========================================
+ Coverage 15.22% 24.88% +9.66%
===========================================
Files 247 1153 +906
Lines 12322 39279 +26957
Branches 4170 10935 +6765
===========================================
+ Hits 1876 9775 +7899
- Misses 8183 23640 +15457
- Partials 2263 5864 +3601 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Backend memory usage comparisonBefore GC
After GC
After Request
|
|
このPRによるapi.jsonの差分 |
There was a problem hiding this comment.
Pull request overview
This PR migrates RSA signing in the backend to slacc@0.1.5 and introduces a configurable “extra” thread pool to offload CPU-bound signing work away from the Node.js event loop / libuv thread pool.
Changes:
- Upgrade
slacc(and platform-specific packages) to0.1.5and adjust pnpm supply-chain delay exclusions. - Replace RSA signing in ActivityPub/JSON-LD signing paths to use
slacc’s RSA implementation. - Add
threadPoolSizeconfig (default1) and initialize the extra thread pool during backend boot.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
pnpm-workspace.yaml |
Excludes slacc packages (and related) from minimum release age delay. |
pnpm-lock.yaml |
Locks slacc upgrade to 0.1.5 across platforms. |
packages/backend/package.json |
Bumps slacc and platform-specific optional deps to 0.1.5. |
packages/backend/src/core/activitypub/JsonLdService.ts |
Switches RSA signing to slacc (RsaKeyPair.sign). |
packages/backend/src/core/activitypub/ApRequestService.ts |
Switches HTTP signature signing to slacc (but introduces an async/signature mismatch to fix). |
packages/backend/src/config.ts |
Adds threadPoolSize config with defaulting in loadConfig(). |
packages/backend/src/boot/common.ts |
Introduces initExtraThreadPool() that calls slacc.init(threadPoolSize). |
packages/backend/src/boot/worker.ts |
Initializes extra thread pool on worker startup. |
packages/backend/src/boot/master.ts |
Initializes extra thread pool on master startup. |
chart/files/default.yml |
Documents new threadPoolSize config option. |
.devcontainer/devcontainer.yml |
Documents new threadPoolSize config option. |
.config/example.yml |
Documents new threadPoolSize config option. |
.config/docker_example.yml |
Documents new threadPoolSize config option. |
.config/cypress-devcontainer.yml |
Documents new threadPoolSize config option. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR migrates RSA signing in the backend from node:crypto synchronous APIs to slacc@0.1.5, offloading CPU-bound signing work onto a managed thread pool, and introduces a new threadPoolSize config option to control that pool size.
Changes:
- Replace RSA signing implementation in ActivityPub request signing and JSON-LD signatures to use
slacc(async/offloaded). - Add and initialize
threadPoolSizeconfiguration (default1) and wire thread-pool initialization into boot + unit test runtime. - Update dependency manifests/lockfile and document the new config option in examples + changelog.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pnpm-workspace.yaml |
Excludes slacc packages (and related) from minimum release age delay. |
pnpm-lock.yaml |
Updates slacc and platform packages to 0.1.5 with new engine constraints. |
packages/backend/package.json |
Bumps slacc and platform-specific optional deps to 0.1.5. |
packages/backend/src/core/activitypub/ApRequestService.ts |
Makes request signing async and routes RSA signing through slacc (RsaKeyPair.sign). |
packages/backend/src/core/activitypub/JsonLdService.ts |
Routes RsaSignature2017 signing through slacc while keeping verification via node:crypto. |
packages/backend/src/config.ts |
Adds threadPoolSize to config types and loads it with a default of 1. |
packages/backend/src/boot/common.ts |
Adds initExtraThreadPool() wrapper around slacc.init() with a one-time guard. |
packages/backend/src/boot/worker.ts |
Initializes the extra thread pool early in worker startup. |
packages/backend/src/boot/master.ts |
Initializes the extra thread pool in master startup as well. |
packages/backend/jest.config.unit.cjs |
Uses a custom Jest test environment for unit tests. |
packages/backend/test/jest.environment.unit.cjs |
Initializes slacc thread pool for unit tests (init(1)). |
packages/backend/test/unit/ap-request.ts |
Updates unit tests to await the now-async signing helpers. |
chart/files/default.yml |
Documents threadPoolSize in the chart default config. |
.devcontainer/devcontainer.yml |
Documents threadPoolSize in devcontainer config. |
.config/example.yml |
Documents threadPoolSize in the example config. |
.config/docker_example.yml |
Documents threadPoolSize in the docker example config. |
.config/cypress-devcontainer.yml |
Documents threadPoolSize in the cypress devcontainer config. |
CHANGELOG.md |
Notes the new threadPoolSize option and mentions RSA signing offload. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
fded18f to
cd95e1e
Compare
|
/preview |
|
👍 |
RSA 署名処理に slacc@0.1.5 を使用する
What
slacc に rayon ベースのスレッドプールを用意し、aws_lc_rs を使用して RSA 署名処理をそちらに退避させる。また、このスレッドプールのスレッド数の設定を config に
threadPoolSizeとして追加。この設定値はデフォルトで 1 で、ワーカーごとに指定した数のスレッドプールが作られる。Why
従前の RSA 署名処理は
node:cryptoを使用していて、これは同期処理なのでイベントループをブロックする。Web Crypto API に移行して非同期処理にしても、内部的には libuv のスレッドでブロッキングされ、他の libuv のスレッドに待避する処理が詰まる。そもそもこの libuv のスレッド数はデフォルトで 4 であり、何も考えずにこちらに退避させると最悪の場合 clusterLimit * 4 スレッドで CPU バウンドな署名処理が走り、スパイクを重篤化させる可能性がある(皮肉にも同期処理のおかげで回避されてきたともいえる)。
slacc に managed なスレッドプールを作り、こちらにこの CPU バウンドな処理を積むことで、libuv のスレッドで I/O バウンドな処理が詰まることなく、イベントスレッドがブロックされないようにできる。
Additional info (optional)
threadPoolSizeについては基本的に1で良いです。特に、同一環境上に PostgreSQL などが同居していたりする状況でみだりにスレッドプール数を上げすぎてしまうと、スパイク時に頻繁なコンテキストスイッチで OOM などを引き起こして問題を発生させるリスクがあります。大規模なサーバでclusterLimitを CPU のコア数より減らして運用している場合などで初めてthreadPoolSizeの調整が効いてくるようになるかと思います。Checklist