Skip to content

[NEW_PATTERN]_{20}_{SAME_PATCH_DIFFERENT_TOOL}.#2

Open
buabeng2 wants to merge 1 commit intoIntelligent-CAT-Lab:mainfrom
buabeng2:main
Open

[NEW_PATTERN]_{20}_{SAME_PATCH_DIFFERENT_TOOL}.#2
buabeng2 wants to merge 1 commit intoIntelligent-CAT-Lab:mainfrom
buabeng2:main

Conversation

@buabeng2
Copy link
Copy Markdown

@buabeng2 buabeng2 commented Apr 1, 2026

[NEW_PATTERN]{20}{SAME_PATCH_DIFFERENT_TOOL}.

The pattern is when a patch is attempted to be made, and it is unsuccessful. The unsuccessful patch is then followed up by attempting the same thing again but with a different tool. The LLM considers the failure to have to do specifically with the tool, and just retries the same patch. Usuaully this results in an LLM in a stuck state where it continues to attempt and do the same thing to fix issues. This can be found in gpt-5-mini model with the instanceID mwaskom__seaborn-3187. Here is a representation shown here.
{
"label": "str_replace_editor: str_replace",
"args": {
"path": "/testbed/seaborn/_core/scales.py",
"file_text": "",
"view_range": [],
"old_str": " if prop.legend:\n axis.set_view_interval(vmin, vmax)\n locs = axis.major.locator()\n locs = locs[(vmin <= locs) & (locs <= vmax)]\n labels = axis.major.formatter.format_ticks(locs)\n new._legend = list(locs), list(labels)\n\n return new\n",
"new_str": " if prop.legend:\n axis.set_view_interval(vmin, vmax)\n locs = axis.major.locator()\n locs = locs[(vmin <= locs) & (locs <= vmax)]\n formatter = axis.major.formatter\n labels = formatter.format_ticks(locs)\n # Attempt to capture any multiplicative offset used by the formatter\n offset = None\n try:\n # Many formatters (e.g. ScalarFormatter) provide a get_offset() method\n off = formatter.get_offset()\n except Exception:\n off = None\n if off:\n offset = str(off)\n new._legend = list(locs), list(labels) if offset is None else (list(locs), list(labels), offset)\n\n return new\n",
"insert_line": 0,
"edit_status": "failure: unknown"
},
"flags": {},
"phases": [
"patch"
],
"step_indices": [
33
],
"tool": "str_replace_editor",
"command": "",
"subcommand": "str_replace",
"id": "33:str_replace_editor: str_replace"
},
{
"label": "python",
"args": [
"-",
"from pathlib import Path\np=Path('/testbed/seaborn/_core/scales.py')\ns=p.read_text()\nold=''' if prop.legend:\n axis.set_view_interval(vmin, vmax)\n locs = axis.major.locator()\n locs = locs[(vmin <= locs) & (locs <= vmax)]\n labels = axis.major.formatter.format_ticks(locs)\n new._legend = list(locs), list(labels)\n\n return new\n'''\nif old in s:\n new=''' if prop.legend:\n axis.set_view_interval(vmin, vmax)\n locs = axis.major.locator()\n locs = locs[(vmin <= locs) & (locs <= vmax)]\n formatter = axis.major.formatter\n labels = formatter.format_ticks(locs)\n # Attempt to capture any multiplicative offset used by the formatter\n offset = None\n try:\n # Many formatters (e.g. ScalarFormatter) provide a get_offset() method\n off = formatter.get_offset()\n except Exception:\n off = None\n if off:\n offset = str(off)\n new._legend = list(locs), list(labels) if offset is None else (list(locs), list(labels), offset)\n\n return new\n'''\n s=s.replace(old,new)\n p.write_text(s)\n print('patched')\nelse:\n print('pattern not found')"
],
"flags": {
"heredoc": true
},
"phases": [
"validation"
],
"step_indices": [
34
],
"tool": "",
"command": "python",
"subcommand": "",
"id": "34:python"
},
{
"label": "sed",
"args": [
"/testbed/seaborn/_core/scales.py"
],
"flags": {
"n": "1,440p"
},
"phases": [
"patch"
],
"step_indices": [
35
],
"tool": "",
"command": "sed",
"subcommand": "",
"id": "35:sed"
},

Right here the json shows the process of the replacement of the str_replace method with manual patching to make everything work.
In here this is used instead of the Graphtrajectory due to the large size of the image.
gpt-5-mini-mwaskom__seaborn-3187.pdf

@ReyhansLab ReyhansLab requested a review from Shuyang-Liu April 6, 2026 17:23
@ReyhansLab
Copy link
Copy Markdown
Member

Thanks for opening the PR. We're in the process of reviewing this and will get back to you soon.

@Shuyang-Liu
Copy link
Copy Markdown
Collaborator

Shuyang-Liu commented Apr 6, 2026

Thanks for opening the PR! The proposed inefficiency pattern makes sense, but it does not work on the provided example. Unlike str_replace, which explicitly specifies old_str and new_str during tool calls, extracting the file path and the old/new strings from a python bash command that performs the edit is non-trivial.
In mapPhase.py (def _extract_edited_files_from_python_code) we use lightweight heuristics to parse the code and extract the edited file path. This can be extended to also extract the old/new strings to support the new pattern.
Tip: please pull the latest version of Graphectory. There is a phase classification issue in your example: the python node should be labeled as patching and sed as localization. This has already been fixed in recent commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants