Add mypy type hints for utils

[emmuhamm]

Description

Fixes issue #852

Update utils folder to follow mypy standards

This is the first step in introducing mypy to all of drunc. This PR only concerns itself with the utils folder.

The mypy and ruff settings that were used is in this commit: 2097249

It is highly recommended that those who want to add mypy to drunc also follows these settings.

Explanation of the ruff and mymy settings

Meaning of the ruff codes

• D201 means “no blank line before class docstring.”
• D202 means “no blank line after class docstring.”
• D212 means “multi-line docstring summary should start at the first line.”
• D205 means “blank line required between summary line and description.”

D + convention = "google" tells ruff to check docstrings against Google style

Meaning of the mypy codes

General ones

• disallow_untyped_defs = true
  • Every function and method must have type annotations.
• disallow_incomplete_defs = true
  • Partial annotations are not allowed. If you annotate one part of a function signature, the rest must be fully typed too.
• check_untyped_defs = true
  • Even functions without full type annotations still get type-checked internally.
• no_implicit_optional = true
  • A parameter is not allowed to silently mean Optional unless you spell it out.
• warn_return_any = true
  • mypy warns if a function returns Any.
• warn_unused_ignores = true
  • If a type: ignore is no longer needed, mypy will complain.
• warn_redundant_casts = true
  • mypy warns when a cast does nothing useful.
• strict_equality = true
  • mypy checks equality comparisons more carefully and flags suspicious ones.
• disallow_any_generics = true
  • You cannot use generic types without specifying their type arguments, like bare list or dict where a more specific type is expected.
• disallow_subclassing_any = true
  • You cannot subclass something mypy thinks is Any, because that usually means the base type is unknown.

Any-specific rules

• disallow_any_explicit = true
  • You cannot write Any directly in annotations unless you explicitly relax it somewhere else.
• disallow_any_expr = false
  • mypy is allowed to accept expressions that evaluate to Any. This is the important escape hatch here. It means you are not forcing the whole codebase to be clean of inferred Any yet.
• disallow_any_decorated = true
  • mypy rejects functions whose types become Any because of decorators.
• disallow_any_unimported = true
  • mypy rejects Any that comes from imports it could not properly follow or type-check.

It should be noted that we avoid changing the TOMLs in this PR. Therefore, if you run mypy right now, then these are the errors you might encounter

Details

With no changes in the toml, the output is as follows

src/drunc/utils/grpc_utils.py:15: error: Skipping analyzing "google.rpc": module is installed, but missing library stubs or py.typed marker  [import-untyped]

src/drunc/utils/configuration.py:8: error: Skipping analyzing "conffwk": module is installed, but missing library stubs or py.typed marker  [import-untyped]

src/drunc/utils/configuration.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 2 errors in 2 files (checked 6 source files)

Type of change

• New feature / enhancement
• Optimization
• Bug fix
• Breaking change
• Documentation

List of required branches from other repositories

N/A

Suggested manual testing checklist

To check if its safe to merge in develop

The usual

To check if the mypy checks work

1. revert 2981ad4. This reverts the revert of the toml, so you should be able to see the new and improved toml files.
2. Make sure you install mypy. Do this with pip install -e .[dev,test]
3. Run mypy src/drunc/utils/. All checks should pass
4. Run ruff check src/drunc/utils/. All checks should pass

Developer checklist

Prior to marking this as "Ready for Review"

Tests ran on: np04-srv-028 from release Nightly from 16 April.

Unit tests - some tests can't be ran on the CI. This is documented. If this PR checks a feature that can't be tested with CI, this has been marked appropriately.

Integration tests - the daqsystemtest_integtest_bundle requires a lot of resources, and connections to the EHN1 infrastructure. Check the cross referenced list if you can't run these. The developer needs to run at least the .

• Unit tests (pytest --marker) passed
  • With relevant marker
  • Without marker
• Integration tests passed
  • Only daqsystemtest_integtest_bundle.sh -k minimal_system_quick_test.py
  • Full daqsystemtest_integtest_bundle.sh
• Testing skipped as there are no core code changes in this PR, this only relates to documentation/CI workflows

Final checklist prior to marking this as "Ready for Review"

• Code is clearly commented.
• New unit tests have been added, or is documented in # ISSUE NUMBER
• A suitable reviewer has been chosen from this list.

Reviewer checklist

• This branch has been rebased with develop prior to testing.
• Suggested manual tests show changes.
• CI workflows fails documented (if present)
• Integration tests passed
  • Only concern yourself if failures related to drunc are in the log files
  • If non-drunc failure appears:
    • Validate failure in fresh working area
    • Contact Pawel if unsure

Once the features are validated and both the unit and integration tests pass, the PRs is ready to be merged.

Prior to merging

Choose one of the following an complete all substeps

• Changes only affect the Run Control, are in a single repository, and do not affect the end user.
  • Changes are documented in docstrings and code comments
  • Wiki has been updated if architectural or endpoint changes
• Otherwise
  • Workflow changes demonstrated in the Change Log (if necessary)
  • Wiki has been updated (if necessary)
  • #daq-sw-librarians Slack channel notified (see below)

Once completed, the reviewer can merge the PR.

Notification message for #daq-sw-librarians Slack channel

For an single merge that changes the user workflow

The CCM WG has an isolated PR ready to merge that affects user workflows. The PR is:

_URL_

I will leave time for any comments, otherwise will merge these at the end of the work day _Insert your time zone_.

For co-ordinated merge

The CCM WG has a set of co-ordinated merges ready to merge. The PRs are:

_URL_

_URL_


I will leave time for any comments, otherwise will merge these at the end of the day.

[emmuhamm]

Note: when developing this it was discovered that ruff has two settings in drunc, one in .ruff.toml and another in the pyproject.toml.

We should probably choose which one we want to keep, and also probably propagate to mypy.

Basically

• do we want settings to be all encompassed in pyproject? or do we want each thing to have its own file?

To be discussed

[PawelPlesniak]

This is great work, thanks @emmuhamm . I left some comments. Once they are addressed I will review, but this looks like the exact changes we need towards working with mypy.

Note: when developing this it was discovered that ruff has two settings in drunc, one in .ruff.toml and another in the pyproject.toml.

We should probably choose which one we want to keep, and also probably propagate to mypy.

Basically

• do we want settings to be all encompassed in pyproject? or do we want each thing to have its own file?

To be discussed

These should be consolidated, if it can be done with the pre-commit workflows. I would expect that this can be done. The pyproject.toml contains more rigorous linting rules, I think we should go with those

[emmuhamm]

Hi @PawelPlesniak this is now ready for review. I've updated the description so hopefully thats all clear. lmk if somethings not clear :)

[PawelPlesniak]

Note no changes were introduced from the merge of develop in the utils dir

[PawelPlesniak]

Nice work on this. Integ tests are currently running. 2 questions

• Was RichErrorServerInterceptor(grpc.ServerInterceptor): to RichErrorServerInterceptor: necessary for mypy?
• You use the variable TYPE_CHECKING to control when some of the stubs are defined. Where is this defined, and how come it is not used in all the files?

[PawelPlesniak]

Manual tests passed
Integ tests passed on np04-srv-029

+++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++ SUMMARY ++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++

Tue Apr 28 05:30:19 PM CEST 2026
Log file is: /tmp/pytest-of-pplesnia/dunedaq_integtest_bundle_20260428164007.log

⮕ Running daqsystemtest/3ru_1df_multirun_test.py ⬅
=============================== 1 error 🚨 in 0.32s ===============================
⮕ Running daqsystemtest/3ru_3df_multirun_test.py ⬅
======================== 6 passed ✅ in 343.30s (0:05:43) =========================
⮕ Running daqsystemtest/example_system_test.py ⬅
======================== 12 passed ✅ in 365.53s (0:06:05) ========================
⮕ Running daqsystemtest/fake_data_producer_test.py ⬅
======================== 6 passed ✅ in 294.63s (0:04:54) =========================
⮕ Running daqsystemtest/long_window_readout_test.py ⬅
============================== 1 skipped 🟡 in 1.76s ==============================
⮕ Running daqsystemtest/minimal_system_quick_test.py ⬅
========================= 4 passed ✅ in 77.44s (0:01:17) =========================
⮕ Running daqsystemtest/readout_type_scan_test.py ⬅
======================== 33 passed ✅ in 874.98s (0:14:34) ========================
⮕ Running daqsystemtest/sample_ehn1_multihost_test.py ⬅
============================= 4 skipped 🟡 in 52.81s ==============================
⮕ Running daqsystemtest/small_footprint_quick_test.py ⬅
========================= 3 passed ✅ in 80.16s (0:01:20) =========================
⮕ Running daqsystemtest/tpg_state_collection_test.py ⬅
======================== 5 passed ✅ in 146.50s (0:02:26) =========================
⮕ Running daqsystemtest/tpreplay_test.py ⬅
======================== 6 passed ✅ in 172.60s (0:02:52) =========================
⮕ Running daqsystemtest/tpstream_writing_test.py ⬅
======================== 4 passed ✅ in 143.68s (0:02:23) =========================
⮕ Running daqsystemtest/trigger_bitwords_test.py ⬅
======================== 18 passed ✅ in 440.48s (0:07:20) ========================

[jamesturner246]

Likely should remain a subclass of grpc.ServerInterceptor. Pinging @miruuna, as she's the one to talk to about that stuff. Maybe hold fire until she confirms.

Also is the type checker fine without those NoReturns?

[PawelPlesniak]

Agreed, should confirm with Miruna before making this change
I also think we should clear the NoReturns, I am a little more free now so can help with this is wanted/needed

[PawelPlesniak]

Marked back as In Progress until we resolve the above discussions

[emmuhamm]

Hi All, apologies for the delay in replying.

Before I jump back in the discussion, let me just preface that I'm not an expert of the utils folder, and this is basically coming from discussions with copilot and some reading of the mypy docs. Please let me know if I've misunderstood something >.< .

NoReturn

From my understanding, noreturn means the function should always raise or exit. When we have a function that does nothing, it basically does an implicit 'return None', which is why mypy complains with this

flask_manager.py:267: error: Implicit return in function which does not return [misc]

Thats why I swapped the NoReturns to the Nones. There might be a better way?

Can be disabled with disable_error_code = ["misc"]. Although I prefer the solution in the PR.

Was RichErrorServerInterceptor(grpc.ServerInterceptor): to RichErrorServerInterceptor: necessary for mypy

Without this, mypy throws an error for:

grpc_utils.py:438: error: Return type "RpcMethodHandler[object, object] | None" of "intercept_service" incompatible with return type "RpcMethodHandler[_TRequest, _TResponse] | None" in supertype "grpc.ServerInterceptor"  [override]
grpc_utils.py:440: error: Argument 1 of "intercept_service" is incompatible with supertype "grpc.ServerInterceptor"; supertype defines the argument type as "Callable[[HandlerCallDetails], RpcMethodHandler[_TRequest, _TResponse] | None]"  [override]
grpc_utils.py:440: note: This violates the Liskov substitution principle
grpc_utils.py:440: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides

You can probably override this with disable_error_code = ["override"]

This one I will need help in fixing as i don't know what the purpose is.

Typechecking

This is used as a flag used for mypy, that it uses this when static analysis is performed vs the else block when the code actually runs.

This block is used in flaskmanager because these packages dont have any stubs installed or in the typeshed.

So the thing in the blocks are basically standinds for mypy.

As for why they aren't used elsewhere, its because they have types existing from other imported packages. Saying that, there are cases in drunc which I added that does type_checking already. See here

[emmuhamm]

Re Typechecking, this is a reminder to myself to talk to Pawel about how type_checking is done with stubs in protobuf etc

[miruuna]

While not necessary for the class to work, we'd want to keep the class inheritance in RichErrorServerInterceptor(grpc.ServerInterceptor) not just for code readability but also for compatibility, in case any more logic will be added in the base class in the future. I am not sure if that's the best option but we could just type ignore using # type: ignore?