Revisit the early decision regarding "Remove unused RPCs and disable reconfiguration"

[chenlica]

In #2970 , we decided to "Remove unused RPCs and disable reconfiguration."

@Yicong-Huang @shengquan-ni Can we revisit this decision?

[Yicong-Huang]

I agree we need to reenable them. It was temporarily removed as we are changing our rpc to grpc.

[chenlica]

Thank you @shengquan-ni .

@Yicong-Huang I would like to ask for a meeting to discuss these details. Let's schedule it when you have time. We will include @aglinxinyuan in the meeting.

[Yicong-Huang]

Sure please go ahead.

[chenlica]

@aglinxinyuan Please schedule a meeting and report the finding here.

[aglinxinyuan]

We need @shengquan-ni in the meeting—I’m not familiar with the reconfiguration details. Would 9:00 PM on 1/6 work for everyone?

[chenlica]

@aglinxinyuan Please schedule the meeting offline (using email or Slack). We will report the finding here.

[chenlica]

Thanks. @shengquan-ni Can you chime in?

[chenlica]

@shengquan-ni Can you chime in?

[shengquan-ni]

We have decided to re-enable the reconfiguration feature, and I will lead this effort. This feature highlights our system’s ability to support highly customized interactions, and it is a central focus of two of our papers (Fries and Udon). We believe this capability represents a key strength of our system.

[chenlica]

@shengquan-ni Do you have any update on this effort?

[shengquan-ni]

Had been busy recently. Will try to finish it before the end of Feb.

[Yicong-Huang]

This task is done. we did not create an issue. So I want to explicitly mention the status of the task.

[chenlica]

A general question: should we close those discussions that are done?

[Yicong-Huang]

For this discussion, we reached consensus on a concrete action item: enable the reconfiguration. Then we should have created an issue (a concrete todo), with this button.

Later when we finish the PR (done with the todo), we should close the issue.

The discussion life cycle can be completely detached from the issue and PR. It can be archived right after we reached consensus, or remain open for as long as we want just to give future readers information. So back to your question, either way is fine.

I think this discussion may not necessarily benefit future readers, so we could just archive it IMO.

[chenlica]

Please go ahead to close it.

[Yicong-Huang]

Follow-up: PR #4220 re-enabled the engine-side reconfiguration handlers, but the web-service entrypoint ExecutionReconfigurationService.performReconfigurationOnResume was still in its throw new RuntimeException("reconfiguration is tentatively disabled.") state (the body that actually issues the controller RPC was commented out back in #2970 and was not restored alongside #4220). Result: pause → modify operator property → resume kills the workflow with that exception even though the engine path is now ready.

PR #4531 wires the entrypoint back to controllerInterface.reconfigureWorkflow, adapted to the current UpdateExecutorRequest(targetOpId, newExecInitInfo) proto shape (no more proto-Any boxing of OpExecInitInfo). One scope cut: StateTransferFunc is no longer threaded through, since the new UpdateExecutorRequest schema does not carry it — flagged in the PR description in case anyone wants to bring it back.