Shortcuts API

[ocornut]

Writing down notes about implementing a shortcut system. I have started using menus more extensively in my own applications, and the lack of support for shortcuts has become a little annoying.

Without shortcuts the user is required to duplicate code to handle menu items and shortcuts for a same action. Consider that MenuItem() can takes both a "checked" and "enabled" parameter which may need to be fetch/computed from your application state, duplicating that code can be pretty annoying and ImGui strive to reduce redundancy so it's quite a flaw to have to do that.

In the old Menus API #126 thread I said we'd need;

Support local keyboard shortcuts, We can use Windows syntax of using & (e.g. "&Save") for convenience.

And

Support general "global" shortcuts (e.g. "CTRL+S"). As a design goal of ImGui we want to avoid code and state duplication, so I'd like the ImGui system to handle shortcuts for the user. It will be optional but likely available by default. So the program can have a single entry point for "Save" whether it is activated via clicking in the menu or via pressing the shortcut. The way it would work is that when a global shortcut scheme is activated, the menu functions always notify the user code to develop its content so ImGui can parse and execute the shortcuts as they are declared, but the actual menu is not layed out nor rendered. The shortcut scheme can be disabled on a per-menu/window/global basis. In particular, procedurally generated menus that may have infinite depth will need to be able to disable the global shortcut scheme. In its "closed" state, the system has to be as lightweight as if the user were testing a bunch of shortcuts themselves. The scope of shortcuts can be dependent on factor such as if the parent window is focused so they aren't always "global". The user should also be able to display the label for a shortcut in the menu without letting ImGui handle the shortcut itself.

So here is a rough list of what I think we need. Unfortunately some of those will need the user to update their ImGui integration to provide the necessary inputs, but there won't be any breakage.

PART A, for regular shortcuts (typically global shortcuts, but they can be local to a window)

EDIT Not needed!
- [ ] We are going to need translated characters, aka the "A" in "CTRL+A" or "ALT+A" is a translated character. So the end-user application needs to feed those inputs probably via io.AddInputCharacter() and this isn't really a problem for ImGui to solve. However, and that's very surprising, retrieve this information under Windows is NOT straightforward. The WM_CHAR message isn't sent when ALT or CTRL are pressed. WM_SYSCHAR is only sent when ALT is pressed. No CHAR messages are sent when CTRL is pressed. By adding this in a Windows message handler I seem to be able to retrieve those characters.

    case WM_KEYDOWN:
    case WM_SYSKEYDOWN:
            if (GetKeyState(VK_CONTROL) & 0x8000)
            {
                BYTE keys[256];
                memset(keys, 0, sizeof(keys));
                keys[VK_SHIFT] = (GetKeyState(VK_SHIFT) & 0x8000) ? 0x80 : 0;
                WCHAR buf[4];
                const int scancode = (lParam >> 16) & 0x1ff;
                const int buf_sz = ToUnicodeEx(wParam, scancode, keys, buf, 4, 0, NULL);
                for (int n = 0; n < buf_sz; n++)
                    io.AddInputCharacters(buf[n]);
            }
            return 1;

It is a little scary but appears to work. End-user would need a similar mechanism which is rather annoying. For Windows messages user can copy demo code if they are pointed to it. GLFW has been patched in the 3.1 branch to support the ALT key but not the CTRL key yet (would be nice if it does so), what that means is that support for those inputs won't be widespread in most applications soon and it is only likely to be widely available in GLFW when 3.2 ships (also means that support for CTRL key would better be pushed in GLFW before they release 3.2!). Or user can freely do their own cheap conversions if they don't care about funky localisation things.

I'd be curious to know whether GLFW 3.1 gives you character inputs in ImGui_ImplGlfw_CharCallback when ALT or CTRL are pressed on MacOS, Linux, etc. See following rely for instructions on how to perform the test. If you can do a test let me know which version of GLFW you are using. Thanks!

• Need to add all non-printable to the ImGuiKey_ enum: functions keys, insert, keypad stuff.. Unfortunately I didn't add those initially, my bad :( (Been thinking about adding a "Tester" to the demo code that helps you verify your ImGui integration, test things like clipping rectangle, texture changes, all sort of inputs. So this tester could list the keys and make sure you can input them.)
• We are going to need to parse shortcuts string efficiently. Encoding should fit in 32-bits (e.g. 16-bit value, 1 bit to select translated character or key, 3 bits modifiers, 2 bits scope: window, window-and-parent, global). Maybe just parsing on the spot (hardcoded stricmp for prefixes such as "CTRL+") is simpler and faster but we need some sort of fast handling of all the named keys that aren't character (e..g F5, HOME, etc.) so this may make the encoding slower. Also figure out a syntax to specify shortcut scope, or this may be not in the string but rather specified in the callee function. If parsing is slow, perhaps hash/FNV1a the string and cache parsed result as a single u32 stored in a ImGuiStorage which is a contiguous sorted map, touching O(N log N) 8-bytes entries. Either way both are rather easy to try.
• Function that checks if an encoded shortcut is pressed. May or not be merged with the parsing function. Trivial.
• Menus need to be able to run in a mode where nothing renders but the global shortcuts are still processed. As per my old comment above, this may be optional and not the default, and the user needs to be able to disable the feature to allow for recursive menu. Perhaps the feature not spreading to sub-menu automatically would make more sense?

PART B (for & ALT-style local shortcuts, lesser priority)

• Menus needs to process the & ALT-style local shortcuts. The behavior little different from the other shortcuts and may imply keyboard controllable menu, at least support for the Enter key.
• We are going to need be able to parse and render the "&Save" syntax for & ALT-style local shortcuts. The text size calculation and text renderer will need a flag to support this feature. For size calculation, we can treat & as zero-width. For rendering, we need to query the width of the next character and draw an underscore below the baseline. We might or not need to handle inhibition with the \ character to be able to render regular & in this mode. All that would be normally easy BUT one problem is that CalcTextSizeA() is often a major bottleneck in very large UI and thus we can't afford to make it slower so those feature would have to be designed accordingly and will need to alter the low-level text rendering API. (Unrelated to this we want to allow centered and right text alignment on a per-line basis and this would have an effect on the text rendering API as well).
• How about adding support for shortcuts to regular widgets like Buttons? Problem with & ALT-style is that it may have a small overhead with text-processing. But may be a nice default and thus we may design the feature expecting it to be always on. For CTRL-style shortcut, probably best to avoid encoding them in a label and just accept that the user can do their own shortcut checking aka
  if (ImGui::Button("Refresh") || ImGui::IsShortcutPressed("F5") { .. }.

That's it for now. Those are merely notes for myself. If there are

[emoon]

I tested this (in the opengl_example) on Mac and added a printf() in KeyCallback function and I got there everytime when pressing some key and holding ctrl, alt or command.

[ocornut]

Sorry my request wasn't clear, the problem is with the CharCallback not the KeyCallback.

To enable CharCallback with modifiers in GLFW 3.1 you need change
glfwSetCharCallback(window, ImGui_ImplGlfw_CharCallback);
to
glfwSetCharModsCallback(window, ImGui_ImplGlfw_CharCallback);
And add the extra integer to the ImGui_ImplGlfw_CharCallback function.
It isn't done in the default ImGui example because this function was introduced in 3.1 which is not in some package repos.

[emoon]

Did the changes and Alt seems to work just fine. Ctrl seems to work in some cases but not in others.

[ocornut]

Any specific pattern with CTRL working vs not working?
Do simple things like Ctrl+A, Ctrl+Shift+A, Ctrl+Alt+A works? Are the broken cases on specifc keys?

Thanks for looking, this is really helpful. This stupid app-side input problem has been the number one barrier for adding shortcuts so I am eager to solve it. I submitted a GLFW tentative patch for Windows but have no way to look into Mac right now.

For this specific use of shortcuts frankly I don't mind too much if some weird keys don't work seeing it is an application-side problem that can be fully solved later, I just would like basic support to work for most users.

[emoon]

So it seems Ctrl+A-Z doesn't work but 0-9 and other non A-Z keys (like /= etc) works fine.

Alt seems to work on all keys. Ctrl+Alt-.... behaves the same as Ctrl- only

[ocornut]

Thank you, very useful. Taking it to glfw.

[emoon]

np :)

[ocornut]

The first point in the list appears to be the most problematic. It looks like I may just need to redesign the IO API, which was desirable anyway, to switch to an event-registration system so e.g. have a AddKeyPress the same way we have a AddInputCharacter and the user may pass a optional stringified localized key name along with keypress.

May or not need to break compatibility with the user-glue code, very obviously if I can avoid it I'll try very hard to avoid it. I always liked how ImGui took this nice shortcut of taking a sample of input state because it makes initial integration with all sort of libraries easier, but we also need a way out of that.

[emoon]

Yeah that sounds. Good also something I have been thinking about for a while (sorry for being a bit of topic) is the versioning of the lib. Something I have started to come to like is "Semantic Versioning" http://semver.org I kinda like this approach as it's clear when breaking changes comes that may cause issue. This would of course break the current versioning and it's of course up to you to use or not but may be something to consider.

[zhiayang]

Sorry to jump on like this, but would it be possible to allow for more flexibility in the shortcut modifier? On OS X it's quite annoying whenever an application doesn't implement Cmd+A/S/V/N/X shortcuts, and chooses to go with Ctrl+whatever.

I already maintain a local patchjob of ImGui::InputTextEx that handles the OS X shortcut style and text editing behaviour (rather, I just #ifdef it away and include a copied version from a header file). It would be great if the new global shortcut system could allow a little leeway in not hardcoding for Ctrl so much?

Just a suggestion, of course.

[ocornut]

How do you think it would work? I don't know enough about Mac experience to design that. My idea is that the user would pass in strings to the api, e.g. "CTRL+S". Then it could be the responsibility of user code to pass in "CMD+S" there? Or an optional/default behavior would be to allow to remap CTRL to CMD by default? (both for key testing and display).

Is the Command key a wholly different key? Does typical third API feed it as an modifier? GLFW has GLFW_KEY_LEFT_SUPER/GLFW_KEY_RIGHT_SUPER keys for the Command key and a keyboard modifier GLFW_MOD_SUPER so that's all ok. Is Super a good terminology that can adapt to both Command and Windows key?

EDIT SDL2 has KMOD_GUI/KMOD_LGUI/KMOD_RGUI so I'll assume other API are supporting this as a modifiers keys as well.

Also - you could post your InputTextEx() mod for reference?

[emoon]

So the way I have done that in the past is like this:

https://github.com/emoon/rocket/blob/master/ogl_editor/src/Menu.c?ts=4#L26

EMGUI_KEY_COMMAND, EMGUI_KEY_CTRL

First part is the modifier on Mac and the other is on Windows/Linux. The thing is that while the Windows key isn't used that much on Windows actually for shortcuts inside programs that is certainly the case on Mac.

[zhiayang]

Ah, thanks for responding. I've changed a number of things actually:

1. Command key for shortcuts. In theory it should work on windows (if ENABLE_OSX_TEXT_EDITING is defined to be 0)
2. Double click selects a word instead of everything (this is the behaviour on OS X)
3. Movement by word in OS X is done with the ALT key instead of the control key, I put a couple of #if guards in there to handle it as well.

(3) is done a bit hackily, since I'm not too familiar with the code, I did a move left word, select word right.

I know that these changes work on OS X, but since I don't have a windows machine to test on (and I'm not sure the rest of my code would work), I can only guarantee my changes to work... "in theory". It's all compile time checks too, so no performance hit either.

I also added a KeyCmd member in the ImGuiIO struct, which is basically the same as KeyAlt and the others. It's updated in the backend input event function as well.

Here's the diff: https://gist.github.com/zhiayang/5764aacd621ebbed6f0b
(um... i have a habit of changing the whitespace, so in case the diff is too unreadable there's the whole function below. terribly sorry)

Here's my edited version: https://gist.github.com/zhiayang/27b3f032a054e503484a

[ocornut]

On InputTextEx( ) for Mac: regardless of the shortcut API above we should probably merge those three into master, add the KeyCmd member. They could be a runtime settings (for slightly better build coverage) and set the default differently based on __APPLE__ define. If someone wants to try making a clean version of that I'll merge else I'll give it a go sometimes.

For shortcuts, we could require the user to pass in different strings per os for apps caring about portability in this manner (not all imgui apps will do), but that would make user code more bulky. Or support a syntax like "CTRL|CMD+S", simple but a little weird. Or have a system where by default "CTRL+S" gets translated to "CMD+S" on Mac ("Mac" defined via a runtime flag again) BUT with a mechanism to enforce actual CTRL when desired, aka a different identifier for CTRL on Mac, e.g. "XCTRL". Or Control key stays "CTRL" and we have a different short identifier meaning "CTRL|CMD"., I don't know how to name that.

CTRL = Ctrl (Windows), Cmd (Mac)
XCTRL = Ctrl (Windows), Ctrl (Mac)
CMD = Cmd (Windows, nearly never used), Cmd (mac)

With R/L variants "LCTRL", "RCTRL".
Looking at Daniel's table it looks like the majority of occurrences of CTRL are to be CMD on Mac, but not all of them.