Skip to content

Fix/3246 url memory leaks#3282

Open
nagisa-kunhah wants to merge 14 commits intoapache:developfrom
nagisa-kunhah:fix/3246-url-memory-leaks
Open

Fix/3246 url memory leaks#3282
nagisa-kunhah wants to merge 14 commits intoapache:developfrom
nagisa-kunhah:fix/3246-url-memory-leaks

Conversation

@nagisa-kunhah
Copy link
Copy Markdown

@nagisa-kunhah nagisa-kunhah commented Mar 27, 2026

Description

Fixes #3246

Checklist

  • I confirm the target branch is develop
  • Code has passed local testing
  • I have added tests that prove my fix is effective or that my feature works

@Alanxtl
Copy link
Copy Markdown
Contributor

Alanxtl commented Mar 28, 2026

our project use import-formatter to format import blocks, that's the reason why ur CI fails. For you, u should

  1. run go install github.com/dubbogo/tools/cmd/imports-formatter@latest
  2. cd to the root dir of dubbo-go
  3. run imports-formatter

@Alanxtl Alanxtl requested a review from No-SilverBullet March 28, 2026 02:51
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 28, 2026

Codecov Report

❌ Patch coverage is 85.39326% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.55%. Comparing base (60d1c2a) to head (c3b5c45).
⚠️ Report is 767 commits behind head on develop.

Files with missing lines Patch % Lines
registry/mock_registry.go 0.00% 9 Missing ⚠️
common/url.go 92.59% 1 Missing and 1 partial ⚠️
registry/protocol/protocol.go 88.23% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3282      +/-   ##
===========================================
+ Coverage    46.76%   49.55%   +2.78%     
===========================================
  Files          295      469     +174     
  Lines        17172    34670   +17498     
===========================================
+ Hits          8031    17181    +9150     
- Misses        8287    16108    +7821     
- Partials       854     1381     +527     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nagisa-kunhah
Copy link
Copy Markdown
Author

here are some confusement about this issue

  1. For P1–5, my understanding is that the original behavior may have been to append the new config and apply it together with the existing one. However, based on BaseConfigurationListener.Process, it seems the behavior there is to replace the existing config entirely. Which behavior is actually expected here?
  2. For P2-2, the original behavior in isMatched is to append. Replacing it with SetParam may change that behavior. Is that expected?
    @Alanxtl @AlexStocks

@nagisa-kunhah nagisa-kunhah requested a review from Alanxtl March 28, 2026 09:31
@nagisa-kunhah nagisa-kunhah marked this pull request as ready for review March 28, 2026 09:32
@@ -250,34 +250,49 @@ func (dir *RegistryDirectory) refreshInvokers(event *registry.ServiceEvent) {
// not in the incoming list can be removed. The Action of serviceEvent should be EventTypeUpdate or EventTypeAdd.
func (dir *RegistryDirectory) refreshAllInvokers(events []*registry.ServiceEvent, callback func()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In refreshAllInvokers, a batch containing only configurator URLs now returns early after replaceConfigurators(...). This updates dir.configurators, but seems to skip rebuilding/reapplying overrides to the existing cached invokers.
Could this delay configurator-only updates until the next provider event? It may be worth adding a test that starts with cached provider invokers, sends a configurator-only NotifyAll, and verifies the effective invoker URLs are refreshed immediately.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are two possible behaviors for configurator-only NotifyAll, and I’d like to confirm which one we want before changing the rebuild logic.

Example:

  • original provider URL:
    dubbo://10.0.0.8:20880/com.foo.UserService?group=g1&version=1.0.0&serialization=fastjson&warmup=100
  • consumer reference URL:
    dubbo://127.0.0.1/com.foo.UserService?group=g1&version=1.0.0&timeout=3000&cluster=failover

Then suppose we receive one configurator-only batch:

  • first batch:
    override://0.0.0.0/com.foo.UserService?timeout=2s&serialization=hessian2

and later another configurator-only batch:

  • next batch:
    override://0.0.0.0/com.foo.UserService?cluster=mock3

If we rebuild from the original provider URL each time, the final effective URL would become:

dubbo://10.0.0.8:20880/com.foo.UserService?group=g1&version=1.0.0&serialization=fastjson&warmup=100&timeout=3000&cluster=mock3

If we instead re-apply the new configurator batch on top of the current effective invoker URL, the final URL would become:

dubbo://10.0.0.8:20880/com.foo.UserService?group=g1&version=1.0.0&serialization=hessian2&warmup=100&timeout=2s&cluster=mock3

Since this patch changed configurator handling from append to replace-by-batch, I’m leaning toward the first behavior, but I’d like to confirm the expected semantics first.

@Alanxtl
Copy link
Copy Markdown
Contributor

Alanxtl commented Mar 29, 2026

here are some confusement about this issue

  1. For P1–5, my understanding is that the original behavior may have been to append the new config and apply it together with the existing one. However, based on BaseConfigurationListener.Process, it seems the behavior there is to replace the existing config entirely. Which behavior is actually expected here?
  2. For P2-2, the original behavior in isMatched is to append. Replacing it with SetParam may change that behavior. Is that expected?
    @Alanxtl @AlexStocks

as for 2, do you mean P2-3?
If the goal is only to avoid duplicate growth while preserving append semantics, another option would be to append only when configurators is not already present.

Copy link
Copy Markdown
Contributor

@AlexStocks AlexStocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: fix/3246 url memory leaks

评价

内存泄漏修复方案合理,通过引入 primitiveTS 字段避免重复解析 URL,并通过 CopyParams() 返回深拷贝解决并发访问问题。

[P1] 规范级问题

1. common/url.go:307-320 - deprecated 方法设计问题

GetParams() 标记为 deprecated 但实现已改为返回深拷贝,这可能导致:

  • 调用方误以为返回的是原始 map(修改不会影响原数据)
  • 现有代码无需修改即可安全使用

建议:考虑直接移除 deprecated 标记,因为当前实现是安全的。或者明确文档说明:

// GetParams returns a deep copy of params.
// Deprecated: Use CopyParams() for clarity. This method now also returns a copy.
func (c *URL) GetParams() url.Values {
    return c.CopyParams()
}

2. registry/directory/directory.go:372-388 - 重复代码

refreshAllInvokers 中的 configuratorURLs 处理逻辑与 convertUrl 存在重复,建议抽取公共方法:

func (dir *RegistryDirectory) processServiceEvent(event *registry.ServiceEvent) (*common.URL, bool) {
    // 返回 URL 和是否为 configurator 标志
}

3. common/url_test.go - 并发测试验证

新增的 TestMergeURLAttributeAccessRace 测试验证了并发访问,但仅运行 50ms,建议延长测试时间或增加迭代次数。


总体评价:修复方向正确,代码质量良好,建议优化 deprecated 方法的文档说明。

Copy link
Copy Markdown
Member

@No-SilverBullet No-SilverBullet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :)

@AlexStocks
Copy link
Copy Markdown
Contributor

文件都冲突了

@Alanxtl
Copy link
Copy Markdown
Contributor

Alanxtl commented Apr 5, 2026

pls resolve file confilct

@AlexStocks
Copy link
Copy Markdown
Contributor

please fix the file confliction.

@nagisa-kunhah
Copy link
Copy Markdown
Author

please fix the file confliction.

Thanks for the reminder. I’ve been a bit busy these past few days, and I’ll resolve the file conflict shortly.

@nagisa-kunhah nagisa-kunhah force-pushed the fix/3246-url-memory-leaks branch from 7de710d to c3b5c45 Compare April 6, 2026 12:53
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 6, 2026

@nagisa-kunhah nagisa-kunhah requested a review from Alanxtl April 6, 2026 15:00
}
defer callback()
if len(providerEvents) == 0 && len(events) > 0 {
return
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] 这里在 configurator-only 的 NotifyAll 场景下直接 return,只替换了 dir.configurators,但没有把新规则重新应用到已经缓存的 invoker。结果就是这批 override 要等下一次 provider 事件或配置中心事件触发后才会真正生效,Alan 之前提的那个语义问题还在。建议这里要么遍历 cacheInvokersMap 按原始 provider URL 重新 overrideUrl + doCacheInvoker,要么至少补一个覆盖现有 cached invoker 立即刷新行为的测试,确保纯 configurator 批次不会静默延迟。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.3.2 version 3.3.2 ☢️ Bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] URL struct memory leaks: attributes never deleted, listeners never cleaned, blacklist key mismatch

5 participants