Skip to content

[#10272] Improvement : nsure JobManager.close() always shuts down background executors when jobExecutor.close() fails#10440

Merged
justinmclean merged 2 commits intoapache:mainfrom
AmitaWhite:improvement
Apr 8, 2026
Merged

[#10272] Improvement : nsure JobManager.close() always shuts down background executors when jobExecutor.close() fails#10440
justinmclean merged 2 commits intoapache:mainfrom
AmitaWhite:improvement

Conversation

@AmitaWhite
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR fixes a potential thread leak in JobManager.close(). Previously, if jobExecutor.close() threw an IOException, the method would terminate early, skipping the shutdown calls for statusPullExecutor and cleanUpExecutor.
The fix wraps the jobExecutor.close() call in a try block and ensures that both background executors are shut down in the finally block.

Why are the changes needed?

To guarantee that background resources are properly cleaned up even when the main job executor fails to close, preventing resource leaks.

Improvement : #10272

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added a new unit test TestJobManager to simulate jobExecutor.close() failure and verify that background executors are still shut down.
Ran the specific unit test:
./gradlew :core:test --tests "org.apache.gravitino.job.TestJobManager"

AmitaWhite and others added 2 commits March 16, 2026 00:47
… executors when jobExecutor.close() fails

This commit ensures that statusPullExecutor and cleanUpExecutor are
always shut down in a finally block, even if jobExecutor.close() throws
an IOException. This prevents potential background thread leaks during
JobManager shutdown.
@github-actions
Copy link
Copy Markdown

Code Coverage Report

Overall Project 64.9% +0.14% 🟢
Files changed 85.94% 🟢

Module Coverage
aliyun 1.73% 🔴
api 47.14% 🟢
authorization-common 85.96% 🟢
aws 1.1% 🔴
azure 2.6% 🔴
catalog-common 10.0% 🔴
catalog-fileset 80.02% 🟢
catalog-hive 80.98% 🟢
catalog-jdbc-clickhouse 78.25% 🟢
catalog-jdbc-common 42.98% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.05% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.07% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.25% 🟢
catalog-lakehouse-paimon 77.71% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.83% 🟢
common 49.36% 🟢
core 81.25% +0.11% 🟢
filesystem-hadoop3 76.97% 🟢
flink 38.86% 🔴
flink-runtime 0.0% 🔴
gcp 14.2% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 45.82% 🟢
iceberg-common 50.21% 🟢
iceberg-rest-server 66.24% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 23.78% 🔴
lance-rest-server 57.84% 🟢
lineage 53.02% 🟢
optimizer 82.95% 🟢
optimizer-api 21.95% 🔴
server 85.65% 🟢
server-common 69.72% 🟢
spark 32.79% 🔴
spark-common 39.6% 🔴
trino-connector 31.62% 🔴
Files
Module File Coverage
core JobManager.java 85.94% 🟢

Copy link
Copy Markdown
Member

@justinmclean justinmclean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this

@justinmclean justinmclean merged commit 526b320 into apache:main Apr 8, 2026
28 of 31 checks passed
@AmitaWhite AmitaWhite deleted the improvement branch April 8, 2026 09:04
babumahesh pushed a commit to babumahesh/gravitino that referenced this pull request Apr 9, 2026
…wn background executors when jobExecutor.close() fails (apache#10440)

### What changes were proposed in this pull request?

This PR fixes a potential thread leak in JobManager.close(). Previously,
if jobExecutor.close() threw an IOException, the method would terminate
early, skipping the shutdown calls for statusPullExecutor and
cleanUpExecutor.
The fix wraps the jobExecutor.close() call in a try block and ensures
that both background executors are shut down in the finally block.


### Why are the changes needed?

To guarantee that background resources are properly cleaned up even when
the main job executor fails to close, preventing resource leaks.

Improvement : apache#10272 


### Does this PR introduce _any_ user-facing change?

No


### How was this patch tested?

Added a new unit test TestJobManager to simulate jobExecutor.close()
failure and verify that background executors are still shut down.
    Ran the specific unit test:

```sh
./gradlew :core:test --tests "org.apache.gravitino.job.TestJobManager"
```

Co-authored-by: AmitaWhite <amitawhite@AmitaWhites-MacBook-Pro-13.local>
babumahesh added a commit to babumahesh/gravitino that referenced this pull request Apr 9, 2026
…shuts down background executors when jobExecutor.close() fails (apache#10440)"

This reverts commit 09d0d86.
babumahesh added a commit to babumahesh/gravitino that referenced this pull request Apr 9, 2026
…shuts down background executors when jobExecutor.close() fails (apache#10440)"

This reverts commit 09d0d86.
yuqi1129 pushed a commit to yuqi1129/gravitino that referenced this pull request Apr 15, 2026
…wn background executors when jobExecutor.close() fails (apache#10440)

### What changes were proposed in this pull request?

This PR fixes a potential thread leak in JobManager.close(). Previously,
if jobExecutor.close() threw an IOException, the method would terminate
early, skipping the shutdown calls for statusPullExecutor and
cleanUpExecutor.
The fix wraps the jobExecutor.close() call in a try block and ensures
that both background executors are shut down in the finally block.


### Why are the changes needed?

To guarantee that background resources are properly cleaned up even when
the main job executor fails to close, preventing resource leaks.

Improvement : apache#10272 


### Does this PR introduce _any_ user-facing change?

No


### How was this patch tested?

Added a new unit test TestJobManager to simulate jobExecutor.close()
failure and verify that background executors are still shut down.
    Ran the specific unit test:

```sh
./gradlew :core:test --tests "org.apache.gravitino.job.TestJobManager"
```

Co-authored-by: AmitaWhite <amitawhite@AmitaWhites-MacBook-Pro-13.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants