`<Leader>bs` shows the wrong buffer… sometimes

[mscheper]

I've been using and loving Bufexplorer for well over 15 years, but in recent months, it's sometimes opened a buffer other than the one I selected. This happens maybe once or twice a week, and I've always just quit gVim and started a new session to work around it.

I haven't filed an issue before now, because, unfortunately, I don't have a reliable set of steps for making it happen. But I can reproduce the issue in my gVim session right now. I'm hoping that, if I tell you about my setup and what's changed, you could suggest things to try, to see whether I can reproduce it more reliably.

I started this gVim session a few hours ago, using a Bash script that loads 26 files. Honestly, I wrote the script because of this bug; I always made heavy use of :mksessions, but having a clean starting point of frequently needed files for this project has been pretty useful, too. Anyhow, I've been using Bufexplorer and other tools during this session without issue, until now. Now, when I type \bs (\ being my Leader), I see 27 buffers, the highest number being 33, because I've loaded a few other files and deleted them (d) in Bufexplorer. That's all good, except that, consistently, when I move the cursor to buffer 33, a file called 'home.html', and type Enter, it instead loads buffer 4, one of the 'urls.py' files that I've also been editing.

When I first noticed this issue today, I had my session split into four windows, and I was trying to use \bs to open a fifth. I closed most of the windows with :q, and used \be to load various other files, all of which loaded correctly. Still, when I try to load the file I want, 'home.html', it loads that 'urls.py' instead. Strangely, 'home.html' is listed with the %a attributes, but I'm pretty sure I never actually had it open. The relevant 'urls.py' is also listed as active, i.e. with a.

One thing I did that might have confused Bufexplorer this time is that previously, for 'home.html', file, I used :file and :w to create a copy of the file under a new name. I've had this issue come up without me having done that, though, so this may be a red herring. I've also done this many times over the years without Bufexplorer acting up, but maybe this is a new bug, or something related to a newly installed plug-in.

When I've previously noticed this issue, once or twice I found that the buffer I wanted was actually being loaded into another tab. I've never used tabs in gVim, so it may have happened more often than that without me noticing. There are no tabs visible now, though. I recently disabled the menu bar in my .vimrc, but even when I make it appear again now, with set guioptions+=m, still no tabs appear.

I'm using bufexplorer 7.5.0 (so #117, which seemed vaguely related, should be fixed?) under gVim 9.1. I was also experiencing this issue under gVim 8.2. My OS is Linux Mint 21.3 Cinnamon.

These are the other plug-ins I have installed:

Plug 'editorconfig/editorconfig-vim'
Plug 'jlanzarotta/bufexplorer'
Plug 'tmhedberg/SimpylFold'
Plug 'vim-scripts/indentpython.vim'
Plug 'Rigellute/shades-of-purple.vim'
Plug 'Valloric/YouCompleteMe'
Plug 'dense-analysis/ale'
Plug 'nvie/vim-flake8'
Plug 'sbdchd/neoformat'
Plug 'tpope/vim-surround'
Plug 'mtdl9/vim-log-highlighting'
Plug 'stevearc/vim-arduino'
Plug 'tpope/vim-abolish'
Plug 'chrisbra/csv.vim'
Plug 'madox2/vim-ai'

I've installed the bottom three pretty recently, but after I started noticing this issue. I've used most of the rest since well before I started noticing this issue. However, I've made a lot of changes to my ALE setup (dense-analysis/ale) and YouCompleteMe in recent months. I honestly can't remember whether this issue started happening before I started making those changes, and since this issue occurs fairly seldomly, even if I revert those changes today, it'd take a week or two of this issue not occurring, to convince myself that they're related. And then it would open the whole can of worms for which changes are causing the problem.

But if you can think of things to try that might reproduce the issue more quickly, I could give it a go and let you know.

Sorry this is so vague.

[drmikehenry]

I'm sorry to hear about this frustrating intermittent problem.

You mention running 7.5.0 now, where you are seeing the problem. Do you have an idea which version of BufExplorer you were using in the past before you first encountered this issue? You might be able to narrow things down by running that older version of BufExplorer for a while to verify that the issue doesn't occur, rather than trying to revert your ALE and YouCompleteMe changes. There still could be some interaction between the plugins, but having a smaller range of BufExplorer versions to consider would be helpful.

Visiting the wrong buffer when pressing <Enter> is a surprising behavior. BufExplorer's s:SelectBuffer() function chooses a buffer to open:

bufexplorer/plugin/bufexplorer.vim

Line 802 in 13a64f2

function! s:SelectBuffer(...)

When pressing <Enter>, the buffer number is determined on line 831 via:

let _bufNbr = str2nr(getline('.'))

The current line in the BufExplorer window is retrieved and converted to a number. Vim's str2nr() function will skip over leading spaces, convert any digits it finds, and stop at the first non-digit in the string, e.g.:

:echo str2nr(' 33u      home.html  /path/....')

In the usual case, the flow picks up with line 882. In your scenario with only a single tab open and with home.html not open in any existing windows when you place your cursor on home.html and press <Enter>, tabNbr should be zero (since home.html would not be found in a window in any tab). In that case, line 889 should result in executing keepjumps keepalt silent b!33 for your example. Even if you have more tabs open and home.html were open in another tab, the expected behavior (when "Locate buffer" is enabled) is to switch to that tab and window where home.html is already open. So I don't immediately see how the wrong buffer could be selected, assuming your cursor is indeed on the home.html line in the BufExplorer window when you press <Enter>.

One way to verify how may tabs are open is to echo the last tab number (which is also the number of open tabs):

:echo tabpagenr('$')

But I'm assuming you indeed had only the one tab open during your tests.

For the case of opening into another tab, I wonder whether you might have pressed shift-<Enter> instead of just <Enter>. Shift-<Enter> is an alias for t, which opens the buffer in a new tab.

When you next encounter the issue, it might help to ensure you have only one tab open, then close all but one window and use \be to launch BufExplorer. That would narrow the logic in s:SelectBuffer(). If you are trying to open buffer 33 (for example) but are ending up elsewhere, maybe you could exit BufExplorer via q and then manually try to switch to the buffer via keepjumps keepalt silent b!33 (as BufExplorer does). I'd expect that to work; if it does, then perhaps switch back to another buffer, launch via \be and try again.

[mscheper]

@drmikehenry: I appreciate your thoughtful reply.

My best guess for which version of Bufexplorer I was using, before I first noticed this issue, would be either 7.4.23 or 7.4.26. (I started working on a new project early-to-mid 2023, using tech I hadn't used in a while, and that's when I usually update my tools and scripts.) Looking at the release notes since then, and from your description of s:SelectBuffer(), it's indeed hard to see what could be causing this issue. But next time it happens, I'll come back to your comment, and see if I can get more information.

OTOH, it's quite conceivable that I accidentally pressed Shift+Enter at some point, given that the right Shift key sits right below Enter on my keyboard. I might even be able to blame the cat for that. 🙀 Thanks for that insight.

BTW, can you please tell me how you link to a line of code in a GitHub comment like that? I often want to do it but have never found a way, and your comment is somehow the first where I've seen it done.

[drmikehenry]

One idea for getting a little more information is to use the below script to watch Vim's autocommands.

Save this script as bufx.vim:

function! Bufx_log(msg)
    call writefile([a:msg], '/tmp/bufx.log', 'a')
endfunction


function! Bufx_record(cmd)
    let bufNbr = expand('<abuf>')
    let winNbr = winnr()
    let tabNbr = tabpagenr()
    let name = expand('<afile>')
    let numWindows = winnr('$')
    call Bufx_log(printf('%-12s %3s, %3s, %3s, #win=%2d, «%s»',
            \ a:cmd, bufNbr, winNbr, tabNbr, numWindows, name))
endfunction


augroup Bufx
    autocmd!
    autocmd BufNew * call Bufx_record('BufNew')
    autocmd BufEnter * call Bufx_record('BufEnter')
    autocmd BufDelete * call Bufx_record('BufDelete')
    autocmd BufWipeout * call Bufx_record('BufWipeout')
    autocmd WinEnter * call Bufx_record('WinEnter')
    autocmd WinLeave * call Bufx_log('') | call Bufx_record('WinLeave')
    autocmd TabNew * call Bufx_record('TabNew')
    autocmd TabEnter * call Bufx_record('TabEnter')
    autocmd TabLeave * call Bufx_record('TabLeave')
    autocmd TabClosed * call Bufx_record('TabClosed')
augroup END

This script may be sourced at any time in a running Vim instance via:

:source bufx.vim

It creates a log file recording the comings and goings of buffers, windows, and tabs. The columns in the log file are:

Event  bufferNumber  windowNumber  tabNumber numberOfWindows  filename

It's got a hard-coded path to the log file (/tmp/bufx.log at present); tailor this path as you see fit.

After sourcing, below is the output from launching BufExplorer via \be and choosing file1.txt which is loaded into buffer 1:

BufNew       152,   1,   1, #win= 1, «[BufExplorer]»
BufEnter     152,   1,   1, #win= 1, «[BufExplorer]»
BufDelete    152,   1,   1, #win= 1, «[BufExplorer]»
BufWipeout   152,   1,   1, #win= 1, «[BufExplorer]»
BufEnter       1,   1,   1, #win= 1, «file1.txt»

When you next encounter the issue, you can source this script and then watch the log file as you trigger the issue. If there's some weird interaction with other plugins, perhaps you'll be able to see something interesting in the logs (e.g., a BufEnter for your actual desired file followed by another BufEnter of the wrong file).

You could also patch the Bufx_log() function into BufExplorer itself such that you could add tracing to the logic in BufExplorer to show what's going on (perhaps logging the chosen buffer number in s:SelectBuffer() or something).

I endorse the time-honored approach of blaming the cat! :-)

Github has a "permalink" feature for linking into code. To make a permalink:

• Visit the repository: https://github.com/jlanzarotta/bufexplorer

• Visit a file of interest: https://github.com/jlanzarotta/bufexplorer/blob/master/README.md

• Choose the "Code" tab showing line numbers and the raw text (necessary for file types such as Markdown that have a "Preview" feature).

• Click on a line number of interest (say, line 5); triple dots will appear beside the line number.

• Click on the triple dots and choose "Copy permalink":

  bufexplorer/README.md

  Line 5 in 93ad8af

  BufExplorer Plugin for Vim

You can optionally choose a different branch or tag before navigating to the file and line number.

Only after typing the above did I think to hunt up a reference: https://github.com/github/docs/blob/main/content/get-started/writing-on-github/working-with-advanced-formatting/creating-a-permanent-link-to-a-code-snippet.md

[mscheper]

So I just noticed it happening again, as soon as I typed :source bufx.vim, the problem went away, and it started loading the file I selected again. Does that make any sense at all?!

Could loading bufx.vim be somehow overwriting another autocmd, that itself could be causing the problem?

Actually, I have my own autocmd BufEnter in my .vimrc, which calls a function that sets colours depending on the directory the file is in. That's still working, even after loading bufx.vim, but I could comment it out and see whether that stops the problem from happening.

I suppose I should just source bufx.vim in my .vimrc, too, even though this would create a very long file.

[drmikehenry]

In one way, I am indeed surprised that sourcing bufx.vim would change BufExplorer's behavior; but on the other hand, the only idea I've got at present about this issue is that somehow there's an interaction between BufExplorer and another part of your configuration (another plugin, or your own BufEnter autocommand, etc.), so maybe there's some weird timing thing interacting with the autocommands from bufx.vim. The autocommands in bufx.vim shouldn't do anything other than change the timing of things. While it's technically possible for bufx.vim to be overwriting other autocommands, it would happen only if other code were using the same named autocommand group, Bufx. The augroup Bufx / augroup END delimiters place the contained autocmd statements into a named group. The autocmd! line deletes everything in that named group before the autocommands are defined. This is done to make it easier to re-:source a script without restarting Vim, as any previously installed autocommands would be replaced with their new definitions. I doubt you're using an autocommand group named Bufx, but you could rename that group to an arbitrarily long and unlikely name to reduce the probability as low as you'd like.

If it doesn't cause you performance issues, sourcing bufx.vim in your ~/.vimrc might get you some more information. If you could ever trigger the issue while the autocommands were being logged, maybe the log would be enlightening.

In general, autocommands are fiddly things that run behind the scenes. It's not always obvious (to users and plugin authors alike) that operations like :edit file.txt can trigger actions in autocommands that occur while the file is in various stages of being "ready". It's easy to inadvertently create a buggy autocommand that changes some assumption made by other parts of the Vim configuration. It's safest for autocommands to do the least amount possible to get their job done. Making changes to plugin-local variables is pretty safe; changing local vim options, changing global vim options, or changing the user interface (e.g., making or deleting windows) are increasingly riskier actions for an autocommand.

There have been a few significantly-sized changes in BufExplorer recently. You're already testing against 7.6.0. There's one more pull request (#121) in the works that, if accepted, would change some autocommand-related behaviors in BufExplorer and possibly impact this hard-to-reproduce issue. You could try running the branch from that pull request now, or you could wait to see if it gets accepted (with possible changes resulting from review).

Additionally, I've made a branch from BufExplorer 7.6.0 that does some logging to /tmp/bufx.log when a buffer is selected. You could try running that branch to see what buffer number (and possibly tab number) BufExplorer is trying to switch to:
https://github.com/drmikehenry/bufexplorer/tree/log-select-buffer

[drmikehenry]

@mscheper Pull request #121 was finalized and merged to become BufExplorer 7.7.0. BufExplorer 7.8.0 was also released, but it is functionally equivalent to 7.7.0. BufExplorer now has different logic in the handling of autocommands, so you may want to try out version 7.8.0. I've also updated my log-select-buffer branch to be based on 7.8.0 in case you want to try 7.8.0 with some logging.

[drmikehenry]

@mscheper It's been a while, so I thought I'd check back to see if you're still having issues with the wrong buffer being selected.

[mscheper]

@drmikehenry: It has been a while! Thanks for checking in, and my apologies for not getting back to you sooner.

The answer is yes, sort of, and it's become more clear what's actually going on. But I don't know why, and I'm not sure it's actually BufExplorer's fault.

This has become a bit of an essay, but here's what I've noticed:

• The problem seems mostly to occur when the file I select in BufExplorer is already being edited, and probably visible. By 'probably visible', I mean that I'm able to reproduce the issue pretty reliably when a window with the file I'm selecting is already open. I suspect that a lot of the time, I had so many buffers open that I didn't notice that the file I was switching to was already squished into a 1- or 2-line window somewhere. But that hasn't always been true: There have been cases where I've made sure the file I selected wasn't already in a visible window.
• I suspect that, in those cases, the file was in some state I don't understand, between visible and closed. One thing I've learnt through all this is that even when a buffer with unsaved changes disappears from view, the changes aren't lost. I don't understand how this happens, but whew! And it seems likely that when a file I select in BufExplorer doesn't open, it's in that weird state. I'm not sure it always has unsaved changes, though.
• <Leader>bs always splits the window at the top of GVim to show BufExplorer, instead of splitting the window my cursor is in. I'm pretty sure that's not how it used to behave, so quite possibly, another plug-in is interfering somehow. Maybe it's trying to do something else when I type <Leader>bs, and it mucks everything up.
• When I select a file that's already open, GVim may be trying to move the cursor to that file, but since I have mousefocus set,* my attention doesn't get drawn to it. And then, when the BufExplorer window closes, the file in the top window that got split is replaced with the file I was already editing—which is also still open in the window where the cursor originally was. (And quite possibly, if the file in the window that got split had unsaved changes, it ends up in that state that confused me, invisible but not lost. But I'm not sure that's always the way files end up in that state.)

* I should say, there's some additional problem with my GVim, that's preventing the mouse pointer, and therefore the focus, from following the cursor. So when the BufExplorer window closes, the focus actually goes to some random window. This is an issue I've been putting up with for a long time (over 5½ years, it turns out!), which almost certainly has nothing to do with BufExplorer, but it might be contributing to the problem I'm having. I've recently speculated that the font I have set for GVim, which is supposed to have fancy code ligatures but that doesn't really work in GVim, might be stuffing up the maths that either GVim or my window manager uses to work out where the mouse cursor should go. [Update: Nope, it's not the fonts. But when I load a buncho' files into gvim -u NONE and turn mousefocus on, it still has trouble putting the mouse in the right place when I type Ctrl+Wj or Ctrl+Wk (AKA, in vim help, CTRL-W_j or CTRL-W_k), so that doesn't seem to be the fault of any plugin.]

I suppose what I ought to do is create a whole new vimrc file, with nothing but BufExplorer, and add pieces of my existing file, including different plugins, until the issues I experience start happening again. That might take an afternoon, but I suspect the time I've wasted fighting with these issues over the last 5½+ years has added up to more time than that, so it should be worth the time in the long run. I'll let you know what I find, when I get to it.

But in the meantime, if any of this prods any suggestions in your mind, please do let me know. Thanks!

[drmikehenry]

I see that I've inserted a regression in BufExplorer's splitting logic in version 7.10.0; I'll be making a pull request to fix that.

I run Ubuntu 24.04 with KDE/Plasma. I don't normally use :set mousefocus, but it seems to work, both with my full configuration and with an empty configuration. On my system, when I switch windows manually via CTRL-w j and friends, the mouse cursor moves to the new window. I setup a Mint 21.3 Cinnamon VM for testing; from a quick initial glance, it doesn't look like Gvim on Mint moves the mouse cursor. Switching windows via CTRL-w j does work, but if I bump the mouse (which is still in the original window), then focus jumps back to the original Vim window. More testing might be needed here to see what's going on.

I'll look further into the splitting regression and the mousefocus issue and let you know what I find out.

[drmikehenry]

I've submitted a pull request to fix the splitting regression: #135

In addition, it ensures that the window-at-launch will be activated when BufExplorer quits via q.

I'm still considering the mousefocus issue.

[drmikehenry]

Vim uses the function gui_mch_setmouse(int x, int y) in gui_gtk_x11.c to move the mouse pointer. This is implemented using XWarpPointer():
https://github.com/vim/vim/blob/20eb68a8f29008fe9041e60b920246f64ee29335/src/gui_gtk_x11.c#L7168

I patched this area of the Vim source to verify that it's being called and that the coordinates are reasonable. It works on my Ubuntu 24.04 machine, but not on my Mint Cinnamon 21.3 VM.

I then wrote a small C program demonstrating that I can toggle between (0,0) and (50, 50) using XWarpPointer(); as before, this works on Ubuntu but not Mint.

It turns out that xdotool mousemove is also implemented using XWarpPointer(), so you can demonstrate this easily yourself.

Install xdotool (apt install xdotool), then try:

xdotool mousemove 0 0

This moves the pointer to the top-left of my screen on Ubuntu; it doesn't appear to move the pointer on Mint. However, I think it actually moves in some sense, though the visible mouse cursor stays put. For example, on Ubuntu:

xdotool mousemove 0 0 getmouselocation

The mouse visibly moves, and the output confirms the new location:

x:0 y:0 screen:0 window:27263110

Trying the above on Mint:

xdotool mousemove 0 0 getmouselocation

The mouse visibly seems to stay put, but the output confirms the new location is actually (0, 0):

x:0 y:0 screen:0 window:58724127

However, if you then barely move the mouse, the mouse cursor moves slightly from its original position, and xdotool confirms the pointer has moved back:

xdotool mousemove 0 0 getmouselocation

with output:

x:942 y:740 screen:0 window:58724127

You can also confirm that the mouse has really moved by simulating a click at (0, 0); if you have a window there, it will be focused:

xdotool mousemove 0 0 click 1

I think this suggests that this is a problem with Mint, not with Vim. I saw some evidence of others having issues with xdotool mousemove on Mint, with no resolution. There could perhaps be something funny going on with VirtualBox and my Mint VM, but if you see these results on your machine as well with xdotool, I think that's pretty good evidence that Mint is doing something non-standard such that XWarpPointer() doesn't work correctly.