Detect Replit trailers in commit messages, use confidence levels#26
Merged
Conversation
Closes chaoss#23. Currently Replit has 2 main AI products producing git commits - Agent for from scratch, and Assistant for extending existing projects. More info see: https://blog.replit.com/new-ai-assistant-announcement Replit adds Replit-Commit-Author trailer for commits generated by both Assistant or Agent, where we have following cases: - Agent commits always have the trailer as all code in commit is gen and owned by agent - Assistant commits have the trailer when: (a) asst was used to summarize commit message only or (b) asst was used to write all code in commit or (c) asst wrote or changed only some code in commit - Assistant commits don't have the trailer when entire code and commit message is manual and committed using Replit ui. So we can check and use Replit-Commit-Author trailer as follows: - if its Agent then medium confidence - if its Assistant then low confidence Additionally we increase (+1) confidence if Replit-Commit-Session-Id, which denotes that commit was generated as part of an AI conversation or workflow. Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
github-project-automation
Bot
moved this to Backlog
in AI Detection GitHub Action Roadmap
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
andrew
requested changes
Apr 9, 2026
Contributor
andrew
left a comment
andrew
left a comment
There was a problem hiding this comment.
Would be good to get a mention of this in the readme as well
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Contributor
Author
Done |
github-project-automation
Bot
moved this from Backlog
to Done
in AI Detection GitHub Action Roadmap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #23.
Currently Replit has 2 main AI products producing git commits - Agent for from scratch, and Assistant for extending existing projects. More info see: https://blog.replit.com/new-ai-assistant-announcement
Replit adds Replit-Commit-Author trailer for commits generated by both Assistant or Agent, where we have following cases:
So we can check and use Replit-Commit-Author trailer as follows:
Additionally we increase (+1) confidence if Replit-Commit-Session-Id, which denotes that commit was generated as part of an AI conversation or workflow.