Key Chords

[firngrod]

It’s been talked about before, but recently, while figuring out a way to cram mouse buttons into the “comfort zone” of my layout, I turned to chording, and I find that I really like the concept. It is kind of a pain to set up with macros though, and even more so to maintain. I have four chords between three buttons, so each macro has to align with two others about three chords each. That is already rather intricate.

So I was wondering if it could become a feature of the keyboard. I think it has been previously suggested and considered, but ultimately turned down? I do have an idea on how it might be done without too much impact on the rest of the code.

Basically, a list of chords would live somewhere. On initial parsing of that list, buttons would be marked as chord-members in their actions. Then if a chorded key is pressed, ifShortcut-style logic is applied, with a configurable timeout, evaluated with regards to whatever chords the key are member of. Then, if a chord is detected, that chord applies to those keys, if not, normal key action applies. If more chords are valid, delay until (one of) the largest is achieved, or until timeout, then apply largest available, breaking ties with first completed. If any key is pressed which is not part of any registered chord with the initiating key, chord is broken and normal key logic resumes. Macros could be run, their values would be set to the values of the first key pressed in the chord. Possibly, an ifChord conditional could be implemented. Chord release would be on release of the first pressed key, in alignment with macro variables.

Chords would be configured independently of keys, but on a per-layer basis as "custom keys", as a list of chords, each containing a list of buttons, and an action which can be configured like keys can. Right now, I don't see any technical reasons why any of the actions available to the user in UI should be impossible, although secondary roles might be a little tricky. Possibly a limit on chord count and keys per chord to manage RAM, possibly living in the same memory-space as macros, which I don't think live in RAM, and can have arbitrary sizes?

It seems like a fun project, and I am probably going to look into making a proof of concept sometime in the not too distant future. I would appreciate any input. I will likely need some guidance from maintainers on some things, such as where the list of chords could live and so on if this is to have any chance of becoming merge-worthy.

EDIT: Added some clarifications

[firngrod]

There are still unknowns of course, such as how to handle overlays and so on, but that can be postponed for now.

[mhantsch]

Have you read my ideas at https://forum.uhk.io/t/chorded-escape-q-w-example-configuration-feature-brainstorming/1429 regarding chords?

I think having a list of chords that the firmware can check against automatically would be great.

Ideally, a matched chord would trigger an action, just like a regular matched keypress.

You are asking where that chords list would live. I spent a bit of time with the macro engine source code in the past weeks, and had some discussions with @kareltucek on design restrictions. One of the facts is that RAM memory is quite limited on the UHK, so the macro engine does not parse / cache statements into RAM, but keeps parsing the actual macro command strings directly from the user configuration, at the cost of re-parsing text multiple times. It's a CPU cycles vs. RAM usage tradeoff. There is no dynamic memory allocation (no malloc/free), all RAM is allocated statically.

For example, the ifShortcut command gets re-parsed, and its list of keyIds gets read and processed multiple times each time a new key is pressed while the ifShortcut is active. If you had a general chords detection implemented in a similar way, the list of chords would be parsed and processed each time any key is pressed. I'm not sure that is feasible (but maybe it is?).

[firngrod]

Yes, I read that some time ago. That was where I had seen it discussed and not pursued yet.

The limits of RAM is exactly why I would like to store the list of chords in user configuration. Whenever the list of available chords changes, on map/layer change, we do not care particularly much about performance in that moment, so we read the chords and store in RAM for each key only the fact of whether the key is part of any chord or not. Then only trigger the chord logic on press of those keys, based on that one RAM'ed bit. No lookups required, very limited RAM cost.

Once a chorded key is hit, we go into postponing and once again don't care particularly about performance since we're already in a state where no output is expected until a timeout, key release or chord completion or break, so I don't think any performance hit of referencing user memory is of any real concern.

For my POC, I would allocate room for a static list of 20 chords of up to 4 keys and create a macro command to set them, which can be done in the macro event of your choosing.

[mhantsch]

If you want to do it similar to the ifShortcut command, then you would not allocate the 4*20 list, but instead parse the chord statements each time the trigger bit for a key is seen. High CPU cost, low RAM use. That's one of the design principles that @kareltucek pursued. You don't parse config into a RAM data structure, you just re-parse again whatever you need into local variables.

Personally, I think it should be parsed into a trie (at least for ifGesture? But that's not chords...), and a certain amount of memory needs to be statically allocated for this.

I guess for your POC you can work with the static array.

Looking forward to your implementation!

[firngrod]

Actually, I think I may go for a macro-based POC in a different way. More hacky, more flexible. Look for special macros defined with names like $chords, possibly also $chords MAP and $chords MAP layer in which chords can be defined using some syntax, probably just //01 02 03 holdKey delete or //01 02 exec SomeMacro. Then those are used to deal with the chords. Then again, I may not, because of code complexity as opposed to just introducing a setChord command.

[mhantsch]

For a start, I like this approach of a $chords macro. For RAM concerns, it could be parsed once at initialisation, and used to mark chord starting keys with the "this key can start a chord" flag. Afterwards, whenever such a key is hit, the macro would be parsed again to see if any chord matches. No need to store chord tables in memory.

For an approach that is less RAM constrained, the macro could be parsed only once at initialisation, and construct the necessary RAM tables for chord detection during runtime. No need to parse the chord macro again and again.

[mhantsch]

BTW, have you seen how Kanata configures chords?

(defchordsv2
  (q w) esc 100 all-released (maxtend)
  (, .) (macro bspc ,) 100 all-released (maxtend)
  (. /) (macro bspc .) 100 all-released (maxtend)
  (grv 1) (macro grv grv grv ret ret grv grv grv ret up up) 100 all-released (maxtend)
)

First, a list of all keys that make up the chord.
Second, action to perform when chord matches.
Third, timeout for waiting for more chord keys.
Fourth, when should chord action release? First key released? All keys released?
Fifth, list of layers where this chord should be excluded (or was it included? I can't remember).

[mhantsch]

Your $chords macro would contain something like this:

ifChord q w final holdKey escape
ifChord commaAndLessThan periodAndGreaterThan final exec bspc_comma
ifChord periodAndGreaterThan slashAndQuestionMark final tapKeySeq backspace periodAndGreaterThan
...

On initialisation, it would be read once, and all the keys in all the ifChord statements would be marked as "chord starting key".

Later, the macro would be processed again each time a "chord starting key" has been hit, which would also trigger postponing. Some general chord timeout is needed too, so we can end postponing if no chord ever matches.

ifChord would never wait, it would just check once for those keys in the postponer queue and continue with the next statement if it doesn't match.

Does that make sense? I'm not an expert in the postponer; I don't dig how it works (yet).

[firngrod]

I have not tried any other keyboardamajiggery than UHK.

I think I just might use the approach of $chords macro and an ifChord command. That will allow me to provide feedback to the user on syntax and so on.

[mhantsch]

Sounds good.

For the RAM conservative path: Each time you are in postponing due to a chord starting key, you will have to run through those commands for each subsequent keypress.

I think the $chord macro should not allow any other statements than ifChord, or else Pandora's box opens on every keypress. Heck, but it needs to allow other commands, because it should execute them if the condition is true. 🤔

The implementation of ifChord is similar to ifShortcut but much simpler. It doesn't have to wait and restart, and wait for more keys on its own. It just checks once and if it doesn't match, continue with the next statement.

[kareltucek]

On initial parsing of that list, buttons would be marked as chord-members in their actions.

Sounds good.

Chords would be configured independently of keys, but on a per-layer basis as "custom keys", as a list of chords, each containing a list of buttons, and an action which can be configured like keys can. Right now, I don't see any technical reasons why any of the actions available to the user in UI should be impossible, although secondary roles might be a little tricky.

Sounds good.

Possibly a limit on chord count and keys per chord to manage RAM, possibly living in the same memory-space as macros, which I don't think live in RAM, and can have arbitrary sizes?

I am not sure how to evaluate that efficiently.

For my POC, I would allocate room for a static list of 20 chords of up to 4 keys and create a macro command to set them, which can be done in the macro event of your choosing.

Sounds good.

That's one of the design principles that @kareltucek pursued.

Depends.

Personally, I think it should be parsed into a trie (at least for ifGesture? But that's not chords...), and a certain amount of memory needs to be statically allocated for this.

Maybe a sorted list?

For a start, I like this approach of a $chords macro. For RAM concerns, it could be parsed once at initialisation, and used to mark chord starting keys with the "this key can start a chord" flag. Afterwards, whenever such a key is hit, the macro would be parsed again to see if any chord matches. No need to store chord tables in memory.

No.

Macro parsing is expensive. Having to evaluate a macro of hundreds of lines on every keypress is definitely not acceptable.

BTW, have you seen how Kanata configures chords?

No. Thanks for this!

On initialisation, it would be read once, and all the keys in all the ifChord statements would be marked as "chord starting key".

Please, no special handling.

---

I very much prefer a native represention - macros would be used just to register the chords.

[mhantsch]

Macro parsing is expensive.

What confuses me is that for ifShortcut and ifGesture it's OK? Their list of keyIds get parsed multiple times also...

That said, I would love a native chords implementation.

Why not provide already binary structure in ValidatedUserConfig that would not need additional RAM for chords? User could configure chords until they run out of user config space. At the end it is similar to having extra keys that do some action. Firmware would just directly search chord keyIds from ValidatedUserConfig, and actions to be done for each chord.

[kareltucek]

What confuses me is that for ifShortcut and ifGesture it's OK?

They have different asymptotic complexity.

Why not provide already binary structure in ValidatedUserConfig that would not need additional RAM for chords?

That sounds like it lacks flexibility.

I really don't want to have to deal with agent and config migrations each time we want to change the behavior.

PoC implementation should ideally not depend on Agent.

[firngrod]

Macro parsing is expensive. Having to evaluate a macro of hundreds of lines on every keypress is definitely not acceptable.

I want chords to be as much like keys as possible. In the ideal world, they would be defined in Agent using a slick UI where all the options for normal keys are available.
In a POC, I currently lean towards a $chords MAP layer macro which is parsed on layer activation. I would strive to run macros as rarely as possible. Keys are marked somewhere in RAM as chorded. When a chorded key is pressed, postponing starts. If no other keys are pressed before release or chord timeout, postponing ends and the key progresses as usual. If another key is pressed, if it's not a chorded key, postponing ends and the keys are handled normally. Only if that other key pressed is another chorded key is the presumably relatively short macro $chords ... is parsed. It is presumed to be relatively short because it would only contain one line for each chord within one layer. I would also add other optimizations, such as if no chord exists which contains all the chorded keys, we abandon ship, and so on.

Would this still be unacceptable do you think, Karel?

[mhantsch]

I know @kareltucek is the definite authority. I'll just add my two cents of thoughts:

$chords macro would provide a reasonable configuration method for chords for a POC. The question whether you parse it only once on keymap/layer switch into internal structures that can be handled by core logic in the firmware, or you mark keys on the initial parse and re-parse the macro on every keypress of a marked key -- that question is secondary; it's an implementation detail, and it can be changed later.

BUT: The weirdest part about such a macro is that it should only contain chord statements on its top level, shouldn't it? So it would be a different kind of macro from all other kinds of macros. What if you tried something like:

$chords QTY base:

chord q w keystroke escape
if ($mySpecialCondition > 0) chord e r t keystroke enter

Should these things work? What if user adds other commands to the chords macro?

Would the chord command bind an action to that chord, or should it run any macro statements? e.g. which of these:

chord q w keystroke escape
chord q w final tapKey escape
chord q w final {
    tapKey semicolonAndColon
    delayUntil 100
    tapKey enter
}