Skip to content

change attention logic to use tridao flash attention in some cases#1

Open
bmedishe wants to merge 1 commit intomainfrom
custom_rocm_flash_dit
Open

change attention logic to use tridao flash attention in some cases#1
bmedishe wants to merge 1 commit intomainfrom
custom_rocm_flash_dit

Conversation

@bmedishe
Copy link
Owner

@bmedishe bmedishe commented Aug 6, 2024

ref : https://github.com/Beinsezii/quickdif/blob/50cd01147f315f70625dc42bc0a1b5479e66a840/attn_custom/rocm_flash.py

What does this PR do?

Fixes # (issue)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@bmedishe bmedishe self-assigned this Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant