VST with second window not receiving keystrokes

[hanoixan]

I'm writing a VST using the JUCE framework on Win32. This particular VST spawns a second window and requires receiving keystrokes for editing text. In Bidule (Standalone build 9722), the keystrokes (WM_CHAR messages) never arrive, even though the window is returning DLGC_WANTALLKEYS. It does receive system keys, though, such as cursor arrow keys. The correct expected behavior happens in Reaper, VSTHost, and Cubase. Some hosts have settings that let you receive all keys if you want them (e.g., Cubase), but I don't see anything like this in Bidule.

Any idea what the issue is and if there's a workaround? Searching for "VST key" didn't bring anything useful up in the forum search.

Thank you,
Sean

[seb@plogue]

I received a similar issue from the SIR/butterfly dev, my debugging showed that my handlers were receiving all keystrokes (and not processing them so they can go to the next handler as well as using the effEditKey opcodes) up until the window pops up and I can't seem to get anything once the window is shown. I wanted to get more details about how the handling is set up inside JUCE with that case but did not received any answers.

I'll have another look later this week to try to see if the keystroke goes anywhere in Bidule's code or not. (besides somehow ending up in the menu/key shortcuts I'm not sure where it could go)

[hanoixan]

Seb,

Fortunately, the JUCE source is available to compile and step through. If you'd like assistance with creating a VST test case that exhibits this behavior, I'd be happy to send you one, or even try some tests for you and report back if that would be easier.

Thanks,
Sean

[seb@plogue]

I know that the JUCE code is available, I just don't want to compile+setup+debug unless I get a rough idea of what is really happening (code/run-time wise) beforehand, over the years I've done that kind of thing way too often to end up pointing bugs that aren't mine (whether it's in the ui framework or something I can see the actual code or what I suspect the plug-in dev has done without seeing any code).

[hanoixan]

Totally understandable. As far as getting a better idea of what's happening, I'm offering to dig into things as far as you'd like. Just tell me what you need to know about JUCE's framework, and how it interacts with Win32, and I'll do what I can.

Sean

[seb@plogue]

You already mention that the arrow keys are working, first thing would be to check if arrow keys are coming in from a different code path than "regular" keystrokes in another host and then see if that code at the lowest level to catch keystrokes in JUCE is actually called or not when running inside Bidule. Are the keys going to the first window and not the second? That kind of thing...

[seb@plogue]

With butterfly which also pops a window to enter a preset name, if I set a breakpoint at main windowproc in Bidule it never gets key down/up messages when the window has focus, if the window is closed or focus is switched to first JUCE window or back to Bidule's main, the breakpoint is hit on any key down/up.

[hanoixan]

Hopefully I can make sense with my findings...

Below is a table of tests and results. Also included are the call stacks showing the chain of command for each of WM_CHAR and WM_KEYDOWN.

It would seem that the new window doesn't receive WM_CHAR to follow up the WM_KEYDOWN. WM_CHAR only happens when the key corresponds to an ASCII character. It would be wrong of JUCE to rely on this alone, and it doesn't. During the WM_KEYDOWN, JUCE calls PeekMessage to see if there's a WM_CHAR in the queue. If there is, it assumes it will handle the key press during the WM_CHAR message. If not, it maps the key to a character and registers the keypress.

Code: Select all

                    if (! PeekMessage (&msg, hwnd, WM_CHAR, WM_DEADCHAR, PM_NOREMOVE))
                    {
                        // if there isn't a WM_CHAR or WM_DEADCHAR message pending, we need to
                        // manually generate the key-press event that matches this key-down.

                        const UINT keyChar = MapVirtualKey ((UINT) key, 2);
                        used = handleKeyPress ((int) LOWORD (keyChar), 0) || used;
                    }


The problem is that although PeekMessage returns true in every test, in the Bidule cases it never receives the WM_CHAR that is promised.

Tests

Code: Select all

Bidule

   VST window
      'a' key: WM_KEYDOWN(value 65), WM_CHAR(value 97, param 1966081)
      left arrow key: WM_KEYDOWN(value 37)

   New window
      'a' key: WM_KEYDOWN(value 65)
      left arrow key: WM_KEYDOWN(value 37)

Reaper

   VST window
      'a' key: WM_KEYDOWN(value 65), WM_CHAR(value 97, param 1966081)
      left arrow key: WM_KEYDOWN(value 37)

   New window
      'a' key: WM_KEYDOWN(value 65), WM_CHAR(value 97, param 1966081)
      left arrow key: WM_KEYDOWN(value 37)


Call Stacks

Code: Select all

WM_KEYDOWN

>   JuceDemoPlugin.dll!juce::TextEditor::keyStateChanged(const bool isKeyDown=true)  Line 2091   C++
   JuceDemoPlugin.dll!juce::ComponentPeer::handleKeyUpOrDown(const bool isKeyDown=true)  Line 228 + 0x12 bytes   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::doKeyDown(const unsigned int key=51)  Line 1849 + 0xa bytes   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::peerWindowProc(HWND__ * h=0x000407ba, unsigned int message=256, unsigned int wParam=51, long lParam=262145)  Line 2221 + 0xc bytes   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::windowProc(HWND__ * h=0x000407ba, unsigned int message=256, unsigned int wParam=51, long lParam=262145)  Line 2102 + 0x18 bytes   C++


WM_CHAR

>   JuceDemoPlugin.dll!juce::TextEditor::keyPressed(const juce::KeyPress & key={...})  Line 2086   C++
   JuceDemoPlugin.dll!juce::ComponentPeer::handleKeyPress(const int keyCode=65, const unsigned int textCharacter=97)  Line 180 + 0x11 bytes   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::doKeyChar(int key=65, const long flags=1966081)  Line 1915   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::peerWindowProc(HWND__ * h=0x000407ba, unsigned int message=258, unsigned int wParam=97, long lParam=1966081)  Line 2236 + 0x10 bytes   C++
   JuceDemoPlugin.dll!juce::HWNDComponentPeer::windowProc(HWND__ * h=0x000407ba, unsigned int message=258, unsigned int wParam=97, long lParam=1966081)  Line 2102 + 0x18 bytes   C++


[seb@plogue]

I'll check if for some reason WM_CHAR is removed from the queue because no WM_KEYDOWN is received but that would mean that the JUCE windowproc either removes it from the queue or has a return value preventing it to go to another windowproc.

[hanoixan]

When I get the chance tonight I'll log all the messages it's receiving, do a more in depth code review, and see if I can find some evidence of that. Thanks for helping investigate this.

Sean

[seb@plogue]

OK, I think I've found the issue. If I dig deeper into what we have left of wxWindows I can see that some messages are discarded when they are from a dialog from an handle that is unknown to wx... That doesn't make much sense to me and the comment next to that smells of hacky fix, removing that I can see the characters appear in the dialog box for new preset in ButterFly. I obviously need to check if this doesn't have any other side effects but the fix would normally be in the next minor revision of Bidule.

[hanoixan]

That sounds promising! For what it's worth, this was what I got for messages in Bidule tonight. And I couldn't find any part of the JUCE code that was stripping out messages from the event queue.

VST Window

Code: Select all

MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_CHAR WPARAM: 97 LPARAM: 1966081
MESSAGE: WM_NCHITTEST WPARAM: 0 LPARAM: 36701418
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_CHAR WPARAM: 97 LPARAM: 1966081
MESSAGE: WM_NCHITTEST WPARAM: 0 LPARAM: 36701418
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_CHAR WPARAM: 97 LPARAM: 1966081
MESSAGE: WM_NCHITTEST WPARAM: 0 LPARAM: 36701418
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_CHAR WPARAM: 97 LPARAM: 1966081
MESSAGE: WM_NCHITTEST WPARAM: 0 LPARAM: 36701418
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743


Spawned Window

Code: Select all

MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_GETDLGCODE WPARAM: 65 LPARAM: 13760084
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_NCHITTEST WPARAM: 0 LPARAM: 9633940
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_KEYUP WPARAM: 65 LPARAM: -1071775743
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0
MESSAGE: WM_GETDLGCODE WPARAM: 65 LPARAM: 13760084
MESSAGE: WM_KEYDOWN WPARAM: 65 LPARAM: 1966081
MESSAGE: WM_GETDLGCODE WPARAM: 0 LPARAM: 0


[valsolim]

I am glad to see this bug reported since it affects MeldaProduction plugins as well. Hopefully, it's going to be fixed in the next update

Best regards
--
Miloslav