Skip to content

Add continuous flow support to mxl-fabrics#504

Open
mlefebvre1 wants to merge 13 commits into
dmf-mxl:mainfrom
jonasohland:feature/fabrics/jonas/protocols-audio
Open

Add continuous flow support to mxl-fabrics#504
mlefebvre1 wants to merge 13 commits into
dmf-mxl:mainfrom
jonasohland:feature/fabrics/jonas/protocols-audio

Conversation

@mlefebvre1
Copy link
Copy Markdown
Contributor

@mlefebvre1 mlefebvre1 commented May 6, 2026

This pull request introduces support for the "continuous flows" to Fabrics.

This will close the following issues: #178 and #181.

To visualize what was implemented https://github.com/dmf-mxl/mxl/blob/main/docs/fabrics/img/mxl-fabrics-cont-flow-rma.png

mlefebvre1 and others added 10 commits May 6, 2026 09:12
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
…st creation. Rename DataLayout substypes to Discrete/Continuous instead of Video/Audio. Change some variable declarations to the 'modern' way.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
…ed 'count' parameter validations to ensure the user doesn't pass a value that would exceed the size of the bounce buffer entry.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
…g audio entry header size of the entrySizeRequired.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
…for a given protocol.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
@mlefebvre1 mlefebvre1 requested review from a team May 6, 2026 13:30
@jonasohland jonasohland changed the title Feature/fabrics/jonas/protocols audio Add continuous flow support to mxl-fabrics May 7, 2026
@felixpou felixpou requested a review from KimonHoffmann May 7, 2026 15:11
@mlefebvre1 mlefebvre1 added this to the v1.1 milestone May 8, 2026
Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
Comment thread lib/fabrics/ofi/src/internal/AudioBounceBuffer.cpp
Comment thread lib/fabrics/ofi/src/internal/LocalRegion.cpp Outdated
Comment thread tools/mxl-fabrics-demo/demo.cpp Outdated

if (status != MXL_ERR_NOT_READY && status != MXL_STATUS_OK)
{
return status;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this skip the cleanup after the while loop?

Copy link
Copy Markdown
Contributor Author

@mlefebvre1 mlefebvre1 May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I've moved the clean-up to the dtor. Also found and fixed a bug in the removeTarget function now that this is called during clean-up.

Comment thread lib/tests/fabrics/test_basics.cpp Outdated
if (status == MXL_STATUS_OK)
{
// transfer complete
return;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return here will skip the cleanup

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also took the opportunity to clean-up the tests by creating abstractions for waiting for connection, waiting for transfer and waiting for completions.

// entry count is a parameter of the protocol instance.
domain->registerRegion(_region, FI_WRITE);

_localRegion = domain->localRegions().front();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of multi-targets, wouldn't this be no longer correct? I think we should do similar thing here as with RMAGrainEgressProtocoLTemplate and get the size of localRegions before calling registerRegion

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should and is called exclusively during initiator setup to register the audio region containing the samples that will be transferred to all targets. This is unlike RMASampleEgressProtocol::registerMemory() which is called on addTarget() and registers a list of AudioEntryHeader entries which are different for each target.

So, what happens during the transfer setup is the following: the scatter-gather list is built of one of those AudioEntryHeader buffer and the samples. The AudioEntryHeader is different per targets, but the samples are the same for each targets.

mlefebvre1 and others added 2 commits May 14, 2026 13:41
…e wrong id was used for searching the target.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
…ansfers and poll for completions.

Co-authored-by: Michael Lefebvre <michael.lefebvre.31@gmail.com>
Signed-off-by: Michael Lefebvre <michael.lefebvre@riedel.net>
@mlefebvre1 mlefebvre1 force-pushed the feature/fabrics/jonas/protocols-audio branch from 43e0ca3 to ad4178c Compare May 14, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fabrics: PR Step 2: API implementation Fabrics: Continuous flow support

2 participants