Skip to content

Johnfreeman/daqdataformats issue79 code refactoring#81

Merged
jcfreeman2 merged 12 commits into
developfrom
johnfreeman/daqdataformats_issue79_code_refactoring
Apr 21, 2026
Merged

Johnfreeman/daqdataformats issue79 code refactoring#81
jcfreeman2 merged 12 commits into
developfrom
johnfreeman/daqdataformats_issue79_code_refactoring

Conversation

@jcfreeman2
Copy link
Copy Markdown
Contributor

@jcfreeman2 jcfreeman2 commented Apr 15, 2026

Description

This PR is meant to address parts of Issue #79 (a general code review of daqdataformats) with a strong focus merely on refactoring (so, not changing the interfaces in ways that will require changes in other packages).

The only exception to this that I dropped ComponentRequest::operator[] since it had been range checking the index passed to it but in the standard library the at function in containers is range checked and operator[] isn't. This means I needed to make very minor changes to dfmodules and dfmessages replacing operator[] with at:
DUNE-DAQ/dfmodules#488
DUNE-DAQ/dfmessages#79
Those PRs should be merged along with this PR.

Testing the changes should begin with a test build which uses this feature branch and the feature branches of the same name in dfmodules and dfmessages. Basically it's regression - we want to make sure everything behaves the same. So, standard integration tests and extended integration tests should give us the same results as usual.

Please note that once this PR is approved we should run the code through clang-format before merging.

Type of change

  • Documentation (non-breaking change that adds or improves the documentation)
  • New feature or enhancement (non-breaking change which adds functionality)
  • Optimization (non-breaking change that improves code/performance)
  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (whatever its nature)
    n.b. while there are changes to in-code comments, the contents of docs/ will be addressed in a separate PR

Testing checklist

  • Unit tests pass (e.g. dbt-build --unittest)
  • Minimal system quicktest passes (pytest -s minimal_system_quick_test.py)
  • Full set of integration tests pass (dunedaq_integtest_bundle.sh)
  • Python tests pass if applicable (e.g. python -m pytest)
  • Pre-commit hooks run successfully if applicable (e.g. pre-commit run --all-files)

Comments here on the testing

Further checks

  • Code is commented where needed, particularly in hard-to-understand areas
  • Code style is correct (dbt-build --lint, and/or see https://dune-daq-sw.readthedocs.io/en/latest/packages/styleguide/)
  • If applicable, new tests have been added or an issue has been opened to tackle that in the future.
    (Indicate issue here: # (issue))

Your Name added 9 commits April 13, 2026 10:23
…nto detail/*.hxx file; also add static_asserts enforcing simple layouts given how these structs are used
By "unused", I'm not counting usage in unit tests or Python bindings

This includes:
Fragment::BufferAdoptionMode::kReadOnlyMode
TriggerRecord::set_header
TriggerRecord::set_fragments
TriggerRecord::get_sum_of_fragment_payload_sizes
TriggerRecordHeader::operator[] (*)

(*) This actually *is* used, but it's exactly replicated by
TriggerRecordHeader::at, so a couple of lines in other package will
need to switch over to TriggerRecordHeader::at. Note that I'm
preferring "at" over "operator[]" because in std:: containers' at
functions perform bounds checking and their operator[] functions don't
… they're not used in our codebase; also fix an incorrectly placed detail/ComponentRequest.hxx include
…e - move inline keywords from point-of-declaration to point-of-definition and some function definitions into *.hxx files
-free up memory	in the Fragment	constructor before it throws an	exception
-Remove	an overflow check which	only checks a tiny sliver of overflow phase space
-Change TypeDefaults from a class with everything public to a struct
-Take advantage of the memory continuity of an std::vector of ComponentRequests
-Refactor Fragment::set_header_fields to be robust against the addition of new header fields
… coding standards (declaration order, etc.). Starting to think unit tests should be explicitly given looser standards.
-Add a couple of brief descriptions at the tops	of files
-Simplify Doxygen syntax in places
-Remove comments which simply restate what's clear from the code
Your Name added 3 commits April 17, 2026 15:34
…and the the at functions in std:: by having it return-by-const-reference rather than by copy
…ings since the addition of Python bindings to the == and != operators mean that Python now expects to be able to put SourceID in a set
…the unsigned int in the hashing function
@jcfreeman2
Copy link
Copy Markdown
Contributor Author

Further details can be found in the commit comments (and of course, the code changes themselves, but here's a summary of what I did:

  • Removed some dead API surface. By "dead API surface", I mean functions that were only used in unit tests and/or Python bindings, if at all. These include TriggerRecord::set_header,
    TriggerRecord::set_fragments, Fragment::BufferAdoptionMode::kReadOnlyMode, TriggerRecord::get_sum_of_fragment_payload_sizes, as well as the string-to-struct operators (operator>>)

  • Some miscellaneous refactors. Fixed a memory leak in the Fragment constructor, removed virtual declarations where they're not needed (e.g., TimeSlice is a concrete class which we don't treat polymorphically), added information to an error message, etc.

  • For those structs which get reinterpret_casted, and more generally, treated as an organized blob-of-bytes, added static_assert checks for is_trivially_copyable and is_standard_layout; also added sizeof checks so the compiler didn't insert unknown padding and a check for little-endianness so that if someday someone tries to build this code on big-endian hardware they'll know at compile time the challenge they're up against.

  • Got TriggerRecordHeader::operator[] / TriggerRecordHeader::at to comply with the conventions of containers in the standard library. Note that outside of unit tests, the former was only used in dfmodule's TRBModule and the latter wasn't used at all. However: since both of the TriggerRecordHeader functions had range checking, whereas in the standard library only at does, I replaced operator[] with at() in a PR for dfmodules, and eliminated TriggerRecordHeader::operator[] entirely.

  • Updated in-code documentation. E.g., I added a brief explanation of what a FragmentHeader is and removed comments which didn't add information not already clear from the code.

  • Improved compliance with our coding standards - e.g., copy/move functions for a class are moved below the declaration of their normal member functions

  • Updated Python bindings, adding __str__ and __repr__ for structs, comparison operators for SourceID, a previously-missing FragmentType::kMPD, etc.

@eflumerf
Copy link
Copy Markdown
Member

eflumerf commented Apr 21, 2026

Looking at the usage in TRBModule, shouldn't it really be component_at()? Should we also add iterators (e.g. component_begin() and component_end()) so that the loop in TRBModule can be simplified to something like

for(auto& component_it = header.component_begin(); component_it != header.component_end(); ++component_it) {
  if (component_it->component == temp_fragment->get_element_id()) {
    requested = true;
    break;
  }
}

@eflumerf eflumerf marked this pull request as ready for review April 21, 2026 15:13
Copy link
Copy Markdown
Member

@eflumerf eflumerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build, unit tests, and integration tests all look normal. Can be merged as-is, or after another round of review if further method renames are performed.

@jcfreeman2 jcfreeman2 merged commit 9a5e6b0 into develop Apr 21, 2026
4 checks passed
@jcfreeman2 jcfreeman2 deleted the johnfreeman/daqdataformats_issue79_code_refactoring branch April 21, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants