MCE Controller Plugin for Keyboard and Mouse Interface

This is a plugin for MCE Controller by Charlie Kindel (MCE Controller | Robust remote control of Windows PCs over the network.). In addition, I am including a helper server (MouseHelper) I wrote that provides a trackpad-like mouse interface that sends mouse commands to MCEC. In combination, this allows you to have a fully-functional, network-based keyboard and mouse for a HTPC available in THR.

This plugin has 6 Settings that can be configured:

  • Host - the hostname/IP address of the computer running MCEC
  • Port - the port MCEC is listening on (default is 5150)
  • MAC - the MAC address of the computer running MCEC (for Wake-on-LAN messages)
  • UseIPforWOL - if left empty, only the MAC address will be used; if not empty, Host must be an IP address (default is TRUE)
  • WOLPort - Wake-on-LAN port; if left empty, port is not sent (default is 9)
  • MatchModifiersOnConnect - TRUE or FALSE depending on whether you want the modifiers (shift, ctrl, alt, etc) to match the current pluginā€™s internal state; this is probably not necessary, but in theory those could get out of sync and this will allow fixing that easily

I know others have written plugins that interface with MCEC for fairly specific use cases. This plugin is different in that it is intended to be general-purpose and could, in theory, be used (hopefully without any modification) for any situation. It supports a very large number of MediaCommands, most of which are self-explanatory. The general groupings are: standard THR MediaCommands (power, basic navigation, and simple numbers); modifier commands (for controlling shift/control/alt/win keys), key commands (corresponding to keys on the keyboard, which can be impacted by the modifier keys, such as upper casing and non-alphanumeric symbols), and mouse commands (just various mouse button inputsā€“these are useful in combination with the MouseHelper server). The modifier and key commands are prefixed with "Letter", such as LetterShiftOn and LetterA. The mouse commands are prefixed with "Mouse", such as MouseLeftClick.

In addition, there is a special MediaCommand that will pass any command through to MCEC. You send this by prefixing the MCEC command with "MCEC:". So, for example, sending the MediaCommand MCEC:chars:Test! would send MCEC the chars:Test! command.

One thing to be aware of that could be a little tricky is that my initial use case was to simulate a keyboard. Thus, the commands are structured to send what the key-presses are, and not what would be typed on the screen. To give an example, to type an exclamation point ("!"), you would, as on an actual keyboard, first turn on the shift with LetterShiftOn and then hit the ā€œ1ā€ key with Letter1. Youā€™d probably then want to turn off the shift with LetterShiftOff. I ended up taking this approach for two reasons: (1) it translates pretty well to my use case and (2) more critically, it is actually very difficult to, in a general purpose way, send specific values to be typed rather than the corresponding key to be pressed due to the way MCEC works. That said, I did originally intend to make commands such as LetterCapitalA and LetterExclamationPoint that would always type a capital ā€œAā€ and ā€œ!ā€, for example. So, there is the beginnings of a framework to support such commands in the plugin, and if there is interest, I would be willing to extend it further to support that behavior as well, to the extent possible. Lastly, the plugin is written for key presses on a standard US keyboard layout. It is possible there are some key mappings that would have to change for other layouts. Iā€™d be happy to share what I know if someone has a need along those lines.

Regarding the MouseHelper server interface, this is something I wrote back when I was using iRule. It is basically unchanged since then and still works great for me. I wanted to preserve it here and provide it in case it would be helpful to others. It is basically a very simple web server that serves up a single web page. The web page uses a library to detect various touch interactions and sends those to the server via a websocket. The server opens a connection to MCEC to pass through the commands from the web page to MCEC. You would run it on the same machine as MCEC is running on. In THR, you just use a WebBrowser control to load the web page and then use it as a touch pad to control the mouse on the HTPC via MCEC's mouse support. Assuming the Home Remote app and your device support it, the webpage supports single/double left click with a single/double tap and single/double right click with a single/double two-finger tap (and maybe a few other things, itā€™s been a long time since Iā€™ve dug through the code). In addition, a scroll region is provided which lets you scroll a page up or down. I find it fairly intuitive, but please let me know if you have questions. In the interest of full disclosure, this concept was based on something someone named ā€œSteveā€ posted to the iRule forum. I ended up re-writing the whole thing to get it to work on my setup.

The instructions for setting up the MouseHelper server are in a post further down due to space limitations here.

Here is a screenshot of the keyboard and mouse interface I use for my theater remote:

When combined, you get a handy network-based keyboard and mouse which is nice for doing various tasks on your HTPC, including general maintenance and setup, and you donā€™t have the problem of a wireless/Bluetooth keyboard not having enough range to work reliably. One last thing, though: certain Windows prompts (the admin/UAC ones) will not respond to MCEC's virtual mouse, so youā€™ll sadly still need to get out of your chair for a wireless keyboard on those rare occasions they pop up (although in more recent testing, this seems to have really improved).

One additional noteā€“usually in my plugins, I strip out the Advanced Plugin Logging capabilities to keep them ā€œcleanā€. However, I run all of my plugins with that code, so Iā€™m going to leave it in because it makes debugging so much easier when running in the app. If you donā€™t want it, it is pretty easy to strip out.

Change Log:

  • 2023-05-28 - expanded WOL configuration, since it seems Windows prefers to be sent the magic packet using UDP, so have to specify IP address and potentially port (thanks, @Shuggy!)
  • 2023-04-30 - added PowerOn (WOL), PowerOff, and a few other MediaCommands; added matching pluginā€™s internal modifier state to MCEC on connect
  • 2020-08-07 - original version

Last updated: 2023-05-28

Here is the plugin code:
MCEController-2023-05-28.plugin (21.9 KB)

plugin.Name = "MCEController";
plugin.OnChangeRequest = onChangeRequest;
plugin.OnConnect = onConnect;
plugin.OnDisconnect = onDisconnect;
plugin.OnPoll = onPoll;
plugin.OnSynchronizeDevices = onSynchronizeDevices;
plugin.PollingInterval = -1;
plugin.DefaultSettings = {
    "Host"                      : "192.168.1.1",
    "Port"                      : "5150",
    "MAC"                       : "11:22:33:44:55:66",  // for Wake-on-LAN
    "UseIPforWOL"               : "TRUE",               // if left empty, only the MAC address will be used; host must be IP address and not name if used
    "WOLPort"                   : "9",                  // if left empty, the port will not be sent
    "MatchModifiersOnConnect"   : "TRUE",
};

var VERSION = "2023-05-28";

/*
    This is intended to be a flexible, general purpose interface to MCEC (the MCEC Controller
    (https://tig.github.io/mcec/)).  Consequently, it provides a large number of defined
    MediaCommands along with some additional functionality to make using it fairly easy in
    many situations.  It can, of course, be customized further, but the intention is to avoid
    the need for any customization.  The initial use cases this was written for were to provide
    a fairly simple interface to Plex Home Theater/Plex Media Player and a full-featured keyboard.
    However, it should allow for interfacing with MCEC in an intuitive way for (hopefully) most
    situations.

    There are several categories of MediaCommand mappings below, each of which may be treated
    differently, but which are all combined to be available as MediaCommands that can be sent by
    Home Remote user interface elements, as usual.  Here is a brief overview of each grouping:
        mediaCommandStandardMappings - these are standard Home Remote MediaCommands
        mediaCommandModifierMappings - these are commands related to modifier keys (shift, control,
                                    alt, and win).  They provide several different ways to control
                                    the up/down state of the modifier keys.
        mediaCommandKeyMappings - these commands correspond to specific keys on the keyboard.
                                    That is, not necessarily the values of those keys, but the
                                    actual keys themselves.  For example, if you want to press
                                    the "1" key, you would use the MediaCommand "Letter1".  When
                                    you let MCEC handle the modifier keys as well, that can result
                                    in either a "1" (shift is not pressed) or a "!" (shift is
                                    pressed) being typed.  This works very well in the case of
                                    emulating the behavior of typing on a keyboard and removes the
                                    complexity of having to configure buttons to send different codes
                                    in the user interface for shifted keys.  Note that, Letter1-0 are
                                    treated differently from Number1-0.  The former are treated
                                    as number keys on the keyboard and the latter are treated as
                                    keys on the number pad (and are defined in the standard HR
                                    mappings instead).
        mediaCommandMouseMappings - these commands send mouse button commands to MCEC.
    
    In addition to the MediaCommand mappings discussed above, there is a special MediaCommand
    supported.  Any MediaCommand sent to the plugin that begins with "MCEC:" will be sent
    as-is to MCEC (with the "MCEC:" prefix removed, of course).  So, for example, to send
    MCEC the "restart" command, you would use the MediaCommand "MCEC:restart".

    This plugin is written for the standard US keyboard layout and sends the virtual key ("vk_")
    codes that correspond to the appropriate keys in that keyboard layout.  For alphanumeric
    keys, it will probably work fine on many other layouts.  However, for non-alphanumeric characters
    (for example, `, ~, [, ], ;, :, ', ", etc.) other keyboards may use different virtual key codes.
*/

var socket = new TCPClient();

var debug = true;
var LOG_MAX = 5000;  // maximum size of the log in characters; set to 0 for unlimited
var LOG_TRIM = 500;  // when maximum size reached, trim the log plus this much more so not constantly trimming

var MCEC_RAW_CMD_PREFIX = "MCEC:";
var MCEC_RETRY_ATTEMPTS = 3;

// These are used to track the up/down state of modifier keys.
// NOTE: while tracked, these states are not currently used, but would be needed if this plugin
//      was extended to support commands to send literal values rather than keys (a feature
//      that was considered but not (yet) implemented).
var modifierStates = {
    "shift" : false,
    "ctrl"  : false,
    "alt"   : false,
    "lwin"  : false,
    "rwin"  : false,
};

// These are standardized Home Remote MediaCommand mappings.
var mediaCommandStandardMappings = {
    "PowerOn"           : "WOL",
    "PowerOff"          : "standby",

    "DirectionUp"       : "vk_up",
    "DirectionLeft"     : "vk_left",
    "DirectionDown"     : "vk_down",
    "DirectionRight"    : "vk_right",
    "Select"            : "vk_return",
    "Enter"             : "vk_return",
    "Back"              : "vk_back",

    "Rewind"            : "rew",
    "Play"              : "vk_media_play_pause",
    "Pause"             : "vk_media_play_pause",
    "PlayPause"         : "vk_media_play_pause",
    "FastForward"       : "fwd",
    "SkipBackward"      : "vk_media_prev_track",
    "SkipForward"       : "vk_media_next_track",
    "Stop"              : "vk_media_stop",

    "Number1"           : "vk_numpad1",
    "Number2"           : "vk_numpad2",
    "Number3"           : "vk_numpad3",
    "Number4"           : "vk_numpad4",
    "Number5"           : "vk_numpad5",
    "Number6"           : "vk_numpad6",
    "Number7"           : "vk_numpad7",
    "Number8"           : "vk_numpad8",
    "Number9"           : "vk_numpad9",
    "Number0"           : "vk_numpad0",
};

// These commands are for controlling the state of modifier keys.
var mediaCommandModifierMappings = {
    "LetterShift"       : "shift",
    "LetterShiftOn"     : "shift",
    "LetterShiftOff"    : "shift",
    "LetterControl"     : "ctrl",
    "LetterControlOn"   : "ctrl",
    "LetterControlOff"  : "ctrl",
    "LetterAlt"         : "alt",
    "LetterAltOn"       : "alt",
    "LetterAltOff"      : "alt",
    "LetterWin"         : "lwin",
    "LetterWinOn"       : "lwin",
    "LetterWinOff"      : "lwin",
    "LetterLWin"        : "lwin",
    "LetterLWinOn"      : "lwin",
    "LetterLWinOff"     : "lwin",
    "LetterRWin"        : "rwin",
    "LetterRWinOn"      : "rwin",
    "LetterRWinOff"     : "rwin",
};

// These commands correspond to specific keys.  That is, not necessarily
//  the unshift/shifted values of those keys, but the actual keys themselves.
//  When you let MCEC handle the modifier keys as well, using these will have
//  the behavior of typing on the keyboard.  This removes the complexity of
//  having to send different codes in the user interface for shifted keys.
var mediaCommandKeyMappings = {
    "LetterA"  : "vk_a",
    "LetterB"  : "vk_b",
    "LetterC"  : "vk_c",
    "LetterD"  : "vk_d",
    "LetterE"  : "vk_e",
    "LetterF"  : "vk_f",
    "LetterG"  : "vk_g",
    "LetterH"  : "vk_h",
    "LetterI"  : "vk_i",
    "LetterJ"  : "vk_j",
    "LetterK"  : "vk_k",
    "LetterL"  : "vk_l",
    "LetterM"  : "vk_m",
    "LetterN"  : "vk_n",
    "LetterO"  : "vk_o",
    "LetterP"  : "vk_p",
    "LetterQ"  : "vk_q",
    "LetterR"  : "vk_r",
    "LetterS"  : "vk_s",
    "LetterT"  : "vk_t",
    "LetterU"  : "vk_u",
    "LetterV"  : "vk_v",
    "LetterW"  : "vk_w",
    "LetterX"  : "vk_x",
    "LetterY"  : "vk_y",
    "LetterZ"  : "vk_z",

    "Letter1"  : "vk_1",
    "Letter2"  : "vk_2",
    "Letter3"  : "vk_3",
    "Letter4"  : "vk_4",
    "Letter5"  : "vk_5",
    "Letter6"  : "vk_6",
    "Letter7"  : "vk_7",
    "Letter8"  : "vk_8",
    "Letter9"  : "vk_9",
    "Letter0"  : "vk_0",

    "LetterEscape"      : "vk_escape",
    "LetterBackspace"   : "vk_back",
    "LetterTab"         : "vk_tab",
    "LetterSpace"       : "vk_space",

    "LetterBackQuote"   : "vk_oem_3",
    "LetterHyphen"      : "vk_oem_minus",
    "LetterEqual"       : "vk_oem_plus", // despite the name, this really is the equal key

    "LetterLeftSquareBracket"   : "vk_oem_4",
    "LetterRightSquareBracket"  : "vk_oem_6",
    "LetterBackslash"   : "vk_oem_5",

    "LetterSemicolon"   : "vk_oem_1",
    "LetterSingleQuote" : "vk_oem_7",

    "LetterComma"       : "vk_oem_comma",
    "LetterPeriod"      : "vk_oem_period",
    "LetterSlash"       : "vk_oem_2",

    "LetterInsert"      : "vk_insert",
    "LetterDelete"      : "vk_delete",
    "LetterHome"        : "vk_home",
    "LetterEnd"         : "vk_end",
    "LetterPageUp"      : "vk_prior",
    "LetterPageDown"    : "vk_next",

    "LetterF1"  : "vk_f1",
    "LetterF2"  : "vk_f2",
    "LetterF3"  : "vk_f3",
    "LetterF4"  : "vk_f4",
    "LetterF5"  : "vk_f5",
    "LetterF6"  : "vk_f6",
    "LetterF7"  : "vk_f7",
    "LetterF8"  : "vk_f8",
    "LetterF9"  : "vk_f9",
    "LetterF10" : "vk_f10",
    "LetterF11" : "vk_f11",
    "LetterF12" : "vk_f12",
};

// These commands are for mouse buttons.
var mediaCommandMouseMappings = {
    "MouseLeftClick"        : "mouse:lbc",
    "MouseLeftDoubleClick"  : "mouse:lbdc",
    "MouseLeftDown"         : "mouse:lbd",
    "MouseLeftUp"           : "mouse:lbu",

    "MouseRightClick"       : "mouse:rbc",
    "MouseRightDoubleClick" : "mouse:rbdc",
    "MouseRightDown"        : "mouse:rbd",
    "MouseRightUp"          : "mouse:rbu",

    "MouseMiddleClick"      : "mouse:mbc",
    "MouseMiddleDoubleClick": "mouse:mbdc",
    "MouseMiddleDown"       : "mouse:mbd",
    "MouseMiddleUp"         : "mouse:mbu",
};


function debugLog(s) {
    // Logs a string to both the console and the device Log attribute
    //  because the console is not available in the app.  It can also
    //  trim the device Log to keep it from growing too large if
    //  LOG_MAX is not 0.
    if(debug) {
        var device = plugin.Devices[plugin.Name];

        // is this the first time through?
        if(device && !device.Log) {
            // log is empty, so start it off with version info
            var verstr = plugin.Name + ": version " + VERSION;
            console.log(verstr);
            device.Log = verstr + "\n";
        }

        var logstr = plugin.Name + ": " + s;

        // send it to the console
        console.log(logstr);

        // send it to the device log
        if(device && device.Log) {
            device.Log += logstr + "\n";
            
            // trim the log if necessary
            if(LOG_MAX && (device.Log.length > LOG_MAX)) {
                device.Log = device.Log.slice(-(LOG_MAX + LOG_TRIM));
            }
        }
    }
}

// this function can optionally re-throw a connection error or suppress it
//  to allow for retrying at a higher level instead
function mcecConnect(suppressErrorOnConnect) {
    if(!socket.isConnected) {
        var host = plugin.Settings["Host"];
        var port = plugin.Settings["Port"];

        debugLog("  mcecConnect() attempting connection...");
        debugLog("    Host: " + host);
        debugLog("    Port: " + port);

        try {
            socket.connect(host, port);

        } catch(err) {
            // unable to connect, that's not good...
            debugLog("ERROR: unable to connect to MCEC: " + err.message);
            if(suppressErrorOnConnect) {
                return false;
            } else {
                throw err;
            }
        }

        // connected
        debugLog("  mcecConnect() connected...");

        // see if we need to match modifiers
        if((plugin.Settings["MatchModifiersOnConnect"].toUpperCase() == "TRUE")
            || (plugin.Settings["MatchModifiersOnConnect"].toUpperCase() == "YES")
            || (plugin.Settings["MatchModifiersOnConnect"].toUpperCase() == "Y")
        ) {
            debugLog("  mcecConnect() matching modifiers...");
            var modifiers = Object.keys(modifierStates);
            for(var m = 0; m < modifiers.length; m++) {
                var modKey  = modifiers[m]
                var modVal  = modifierStates[modKey]
                var modCmd  = (modVal ? "shiftdown:" : "shiftup:") + modKey;
                debugLog("    setting '" + modKey + "' modifier to: " + modVal);

                // send the command
                mcecSend(modCmd);
            }
        }
    }

    return true;
}

function mcecDisconnect() {
    if(socket.isConnected) {
        try {
            socket.close();

        } catch(err) {
            // we're closing down anyway, so it doesn't matter if the close()
            //  failed--just fail silently
        }
    }
}

// wrapper that tries to make sure we're connected before sending
//  returns true if the send worked, false otherwise, to make
//  having resend logic simple
function mcecSend(c) {
    if(!socket.isConnected) {
        // not connected for some reason, try to connect first, but don't
        //  throw an error because we may retry the send
        mcecConnect(true);
    }

    try {
        // send the command
        socket.send(c + "\r\n");

    } catch(err) {
        // send didn't work for some reason (probably connection closed by other end and we
        //  just found out about it)
        debugLog("ERROR: unable to send to MCEC: " + err.message);

        // let the caller know it didn't work
        return false;
    }

    // seems to have worked okay if we got this far
    return true;
}

function mcecGetCommands(mc) {

    // NOTE: the reason we use an array is because, in the future, it may be
    //      the case that a single MediaCommand requires multiple MCEC commands
    //      to be sent, such as in the case of literal value MediaCommands to
    //      control modifier key states, for example.
    var mceCmds = [ ];

    if((mc.slice(0, MCEC_RAW_CMD_PREFIX.length)) == MCEC_RAW_CMD_PREFIX) {
        debugLog("  MCEC raw command identified: " + mc);

        // extract the MCEC raw command and use it as-specified
        var mceRawCmd = mc.slice(MCEC_RAW_CMD_PREFIX.length);
        if(mceRawCmd) {
            mceCmds.push(mceRawCmd);

        } else {
            debugLog("ERROR: '" + MCEC_RAW_CMD_PREFIX + "' must be followed by a command to send");
        }
    
    } else if(mediaCommandStandardMappings.hasOwnProperty(mc)) {
        debugLog("  standard HR command identified: " + mc);

        // standard commands, aside from PowerOn, have no special processing needed
        mceCmds.push(mediaCommandStandardMappings[mc]);

    } else if(mediaCommandModifierMappings.hasOwnProperty(mc)) {
        debugLog("  modifier command identified: " + mc);

        // handle modifier commands by updating our internal modifier state for
        //  the relevant modifier key and then building command to send to MCEC
        var modKey = mediaCommandModifierMappings[mc];

        if(modifierStates.hasOwnProperty(modKey)) {
            // because apparently we don't have endsWith()...
            if(mc.slice(-2) == "On") {
                modifierStates[modKey] = true;

            } else if(mc.slice(-3) == "Off") {
                modifierStates[modKey] = false;

            } else {
                // toggle relevant shift state
                modifierStates[modKey] = !modifierStates[modKey];
            }

            debugLog("  modifier state for " + modKey + " is now " + modifierStates[modKey]);

            // set up command to send to MCEC
            mceCmds.push(((modifierStates[modKey]) ? "shiftdown:" : "shiftup:") + modKey);

        } else {
            debugLog("ERROR: unknown modifier key: " + modKey);
        }

    } else if(mediaCommandKeyMappings.hasOwnProperty(mc)) {
        debugLog("  key command identified: " + mc);

        // keys do not require special processing
        mceCmds.push(mediaCommandKeyMappings[mc]);

    } else if(mediaCommandMouseMappings.hasOwnProperty(mc)) {
        debugLog("  mouse command identified: " + mc);

        // mouse commands do not require special processing
        mceCmds.push(mediaCommandMouseMappings[mc]);
    
    } else {
        // unknown command
        debugLog("ERROR: unsupported command: " + mc);
    }

    return mceCmds;
}

function onChangeRequest(device, attribute, value) {
    debugLog("onChangeRequest called...");

    switch(attribute) {
        case "MediaCommand":

            var cmds = [];

            // get the appropriate MCEC command(s) for the specified
            //  Home Remote MediaCommand
            cmds = mcecGetCommands(value);

           // now we're ready to send the command(s)
           if(cmds.length) {
                for(var c = 0; c < cmds.length; c++) {
                    var sendSuccess = false;

                    // handle WOL separately since it isn't a MCEC command
                    if(cmds[c] == "WOL") {
                        var host        = plugin.Settings["Host"];
                        var mac         = plugin.Settings["MAC"];
                        var useipforwol = (plugin.Settings["UseIPforWOL"] != "") ? true : false;
                        var wolport     = plugin.Settings["WOLPort"];

                        debugLog("  Host: " + host);
                        debugLog("  MAC: " + mac);
                        debugLog("  Use IP for WOL?: " + useipforwol);
                        debugLog("  WOLPort: " + wolport);

                        if(useipforwol) {
                            var wolOpts = {};

                            wolOpts["address"] = host;
                            if(wolport != "") {
                                wolOpts["port"] = parseInt(wolport);
                            }

                            debugLog("  sending WOL to: " + mac + " with {address : " + wolOpts["address"] + ", port : " + wolOpts["port"] + "}");
                            WOL.wake(mac, wolOpts);

                        } else {
                            debugLog("  sending WOL to: " + mac);
                            WOL.wake(mac);
                        }

                        // no way to know, so assume it was successful...
                        sendSuccess = true;

                    } else {

                        debugLog("  sending MCEC command: " + cmds[c]);
                        // try to send the command; if it doesn't work, retry
                        for(var attempt = 0; attempt < MCEC_RETRY_ATTEMPTS; attempt++) {
                            if(mcecSend(cmds[c])) {
                                // that send worked, break out of the retry loop
                                debugLog("  command sent successfully...");
                                sendSuccess = true;
                                break;

                            } else {
                                // didn't send successfully, so wait briefly and try again
                                debugLog("  send unsuccessful, retrying shortly...");
                                sleep(500);
                            }
                        }
                    }

                    // make sure we eventually sent the command and if not, throw an error
                    if(!sendSuccess) {
                        debugLog("ERROR: unable to successfully send command!");
                        throw new Error(plugin.Name + ": unable to send command to MCEC!");
                    }

                    // done with that command, move on to the next one...
                }
            }

            // all done
            break;

        default:
            debugLog("ERROR: unsupported command type: " + attribute);
            break;
    }
}

function onConnect() {
    debugLog("onConnect called...");

    var device = plugin.Devices[plugin.Name];

    if(device) {
        debugLog("  building supported media commands...");
        
        var mediaCommandMappings = [];

        // build all of our actual MediaCommands
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandStandardMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandModifierMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandKeyMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandMouseMappings));        

        // add the special MCEC raw command prefix
        mediaCommandMappings[MCEC_RAW_CMD_PREFIX];

        debugLog("  setting supported media commands...");
        device.SupportedMediaCommands = mediaCommandMappings;
        debugLog("  set (" + device.SupportedMediaCommands.length + "): " + device.SupportedMediaCommands);
    }

    // connect to the MCEC server
    debugLog("  connecting...");
    mcecConnect();
    debugLog("  connected");
}

function onDisconnect() {
    debugLog("onDisconnect called...");
    mcecDisconnect();
    debugLog("  disconnected");
}

function onPoll() {
    debugLog("onPoll called...");
}

function onSynchronizeDevices() {
    debugLog("onSynchronizeDevices called...");

    var device = new Device();

    device.Name = plugin.Name;  // override default naming conversion so it looks nicer
    device.Id = plugin.Name;
    device.DisplayName = "MCE Controller";
    device.Icon = "Remote";
    device.Capabilities = [ "MediaControl" ];
    device.Attributes = [ "Log" ];

    plugin.Devices[plugin.Name] = device;
    
    debugLog("  done syncing");
}

Here is an .hrp file with a sample Keyboard Control that uses the defined MediaCommands. It is set up to be used in a PageBrowser control. As-is, the shift, control, and alt keys change the background color when selected/unselected. You may want to alter the PropertActions in their respective EventTriggers to fit your theme more.
KeyboardControl.hrp (95.1 KB)

1 Like

HFN:

To be clear, the MCE Controller plugin you wroteā€¦ executes the keystrokes and mouse input on the same computer which is running MCE, right? I mean, lets say you have a computer \Plex which is running Plex Server and has access to your media library. This plugin would let you virtually type (e.g. to search for a title) on \Plex. On the other hand, it would not, for example, let you send keystrokes to another device, such as an Apple TV or a Roku. Is that correct?

If I understand correctly, then I believe the answer to your question is ā€œyesā€. The MCE plugin opens a connection to the MCE Controller (MCEC) server that is running on whichever computer you want the input to be received on. To slightly clarify your example, if you have a HTPC that is running some kind of Plex client (Plex Home Theater, Plex Media Player, Plex for Windows, etc) (which may be the same computer as the one running the Plex Media Server, but it doesnā€™t have to be), then you would run MCEC on the HTPC in server mode and the MCE plugin would connect to it and let you send the keyboard commands as if you were typing on the HTPC directly.

Similarly, if you were to also run the MouseHelper on the the HTPC, it would also connect the the MCEC server and provide a webpage (that you can open via a WebBrowser control in Home Remote) that acts as a trackpad to send mouse commands to the HTPC also as if you were using its mouse directly.

The MCE plugin (or the MCE Controller itself) cannot send keystrokes to another device like an Apple TV or Roku. Instead, you would want to use a plugin that can talk to those devices to send them keystrokes. I have also written a Roku plugin that supports sending keyboard commands (for this very use case): Roku (HFN) Plugin

In order to avoid having to create multiple keyboards to use with these different devices, I made sure that the MediaCommand names for sending the keyboard characters were the same between them. I then created a Keyboard page that I embedded in different pages via a Device Browser control so that the MediaCommands were sent to the correct device without having to customize the keyboards:

So I hope that clarifies things for you, but if not, please let me knowā€“Iā€™m happy to help! :slight_smile:

A couple of quick additional notes: First, the screenshot of the Plex Keyboard above doesnā€™t show the MouseHelper webpage correctly because I brought it up in the simulator which cannot normally talk to my HTPC (on a different network)ā€“it normally looks like the screenshot in the original post.

Second, MCE Controller/Windows have some limitations that I have encountered, but managed to work around most of the time. For example, if a User Account Control (UAC) window pops up, Windows will not accept inputs from MCECā€™s virtual keyboard (Iā€™m guessing for security reasons), so the plugin and MouseHelperā€™s inputs will be ignored while it is on screen. This mostly comes up when software (such as PMP) needs to installed/updated and I have a separate bluetooth keyboard I use on those occasions (with much aggravation). If there are other situations where Windows is ignoring the virtual keyboard inputs of MCEC, the plugin/MouseHelper wonā€™t work, but that is a limitation of MCEC/Windows and not the plugin/MouseHelper. I think the only places Iā€™ve encountered it is in the UAC prompts and possibly when applications running as Administrator have focus.

1 Like

Hello.
Having trouble with the home remote windows client connecting to the MCEC server on the same computer. It works fine in designer, but wont connect with the full client. In the MCEC console window it indicates a client connecting when the designer connects, but no message indicating the full client connects.
Any ideas?
Thanks

I donā€™t use the Windows HR app, but my first guess would be it could be an issue with the windows firewall blocking the loopback connection for the HR app for some reason. Iā€™m not much of a windows networking expert, but I know that every time the HR Designer updates I get prompted by the Windows firewall to allow connections to private/local networks, maybe itā€™s a similar thing you need to allow for the HR app?

Thanks and yes, digging in to it, it seems to be a firewall issue(from the firewall logs its dropping the connection request packet to MCEC). However, Iā€™ve tried turning off the firewall for the private connection and the issue still exists. Now delving down in to deeper firewall filtering rules. I have looked at the app permissions in the firewall configuration, and both designer and the full client seemed to be allowed.

As I noted, Iā€™m not at all proficient with Windowā€™s Firewallā€“is it possible your local network is actually configured as a public one (in poking around, I noticed that is how mine is configured; I had no idea) and that is the kind you have to allow connections for? The only other thing I can think of at the moment is the standard Windows solution: try rebooting :slight_smile: Sorry I canā€™t be more help, but maybe someone with more Windows skills than I will be able to provide some insights.

Thanks again, appreciate the suggestions. Tried turning off firewall on all networks, private, public and domain- same. Tried rebooting, uninstalling and reinstalling the app. No luck. I suspect there is something deep in windows firewall that cant be turned off and seems to be causing the issue.

I had this working on my system some time ago and when I built a replacement system I had the same issue which Martin was having. I was able to resolve the problem by going into the mhserver.js file and updating the address that was being used to connect to the MCE server from ā€˜localhostā€™ to ā€˜127.0.0.1ā€™. I have no idea why localhost was not working, but that modification fixed it for me.

1 Like

Thatā€™s great, Steve, glad you were able to find a fix. I have no idea why localhost would stop working either, but Windows networking has always been so weird over the years, I really shouldnā€™t be surprisedā€¦

Thanks Steve! Where would I find this mhserver.js file? Scoured the computer and canā€™t seem to find it.

It is the mhserver - 2015-09-12-js.txt file attached to the initial post in this thread, which has to be renamed mhserver.js in order to actually use it. I am not looking at it on my computer that actually utilizes the program, but I think it is in line 57 of that file where I needed to make the change to get it to work. You do of course need to install javascript and then as I recall, there are also several dependencies which need to be installed in order to get the whole thing operational. I am not that familiar with JS, but I was able to get it working using information I could find on the internet.

Iā€™m getting my new theater up and running and was looking back through these directions to experiment with a few new HTPC options. I discovered I had not actually included the instructions for how to setup the MouseHelper server referenced in the first post (although they were available on the old google groups THR forum, thankfully, since the iRule forum where they were originally posted seems to be long gone). So, for anyone else that is interested, here are those instructions (which would have been too long to include in the first post anyway):

  1. install MCE Controller (from: MCE Controller | Robust remote control of Windows PCs over the network.) (which youā€™ve probably already done)

    • after installing, setup MCE Controller, including enabling the mouse commands
  2. install node.js (from: https://nodejs.org/)

    • be sure to tell the installer to add it to the path (which seems to be the default now)
  3. create a base directory for the MouseHelper server

    • for example: C:\Users\username\mouse
  4. after installing node.js, from a command prompt run in the base directory:

    • npm install express
    • npm install ws
  5. place the file "mhserver.js" in the base directory (attached below)

  6. create a subdirectory called "MouseHelper" in the base directory

    • for example: C:\Users\username\mouse\MouseHelper
  7. place the file "index.html" in the "MouseHelper" subdirectory (attached below)

  8. download "hammer.min.js" from https://hammerjs.github.io/ and put it in the "MouseHelper" subdirectory

    • you should be able to just do a ā€œSave asā€ on the link on the main page with the latest version (v2.0.8)

That sets everything up. There are some configurable items at the top of "mhserver.js", but you shouldnā€™t need to change any of them unless you want to. There are also some configurable items in the "index.html" file (also near the top). You may want to play with those, as they let you set the background color for the mouse region and whether or not to have a scroll region on the right side of the mouse region.

To manually start the MouseHelper server, just run node mhserver.js from the base directory. You will also need to start the MCE Controller. The start-up order shouldnā€™t matter, but youā€™ll get fewer messages from the MouseHelper server if you start MCE Controller first. Once both servers are running, you can point your WebBrowser control or a regular web browser at the IP address of your HTPC and port 6224 and that will bring up the interface and scroll region. For example, if your HTPC IP address is 192.168.1.100, then use http://192.168.1.100:6224. If youā€™re testing from the HTPC itself, you can just use: http://localhost:6224.

You may get a pop-up from the Windows firewall. You need to allow incoming connections from the local network to whatever ports you have the MCE Controller and MouseHelper server listening on. Iā€™m a little rusty on the right settings to do that, but having been bit by it early on, I wanted to mention that the Windows firewall can cause strange issues if set wrong. Basically, the behavior you may see is that connecting from the same machine works fine but not from another computer on the same network (THR device or just another PC/tablet/whatever).

Lastly, you probably want to have both MCE Controller and MouseHelper server run on startup. Again, Iā€™m a little rusty, but from my notes, for Windows 10:

  • Type WindowsKey-R to bring up the Run prompt and type: shell:common startup
  • Into the folder that opens up, copy in shortcuts for MCE Controller and node.exe
  • Configure in the properties for the node.exe shortcut that it should run in the base directory and take mhserver.js as a command line parameter

If anyone knows an easier/cleaner way to enable these to run on startup, please let me know.


Change Log for "mhserver.js":

  • 2023-05-15 - fixed a bug where would not actually attempt to reconnect if first attempt to connect to MCE Controller failed (something changed in the behavior of node.js/associated libraries, because this used to work)
  • 2023-03-25v2 - fixed a bug related to handling messages (something must have changed from older API)
  • 2023-03-25 - incorporates the fix that @kskok68 identified here
  • 2015-09-12 - original version

Last updated: 2023-05-15

Here is the ā€œmhserver.jsā€ file:
NOTE: because of forum limitations, you need to save this/rename it as ā€œmhserver.jsā€ after downloading (that is, remove the ".txt" extension and change the "-js" to ".js"):
mhserver - 2023-05-15-js.txt (5.1 KB)


Change Log for "index.html":

  • 2023-07-01 - added support to configure minimum press interval and set default to 700ms (as suggested by @Shuggy)
  • 2015-09-12 - original version

Last updated: 2023-07-01

Here is the ā€œindex.htmlā€ file:
index - 2023-07-01.html (9.4 KB)

Made a few small updates to this plugin to add support for the following MediaCommands:

  • PowerOn
  • PowerOff
  • Rewind
  • Play
  • Pause
  • PlayPause
  • FastForward
  • SkipBackward
  • SkipForward
  • Stop

Most of this are straight-forward pass-throughs to MCEC's built-in commands, but PowerOn actually sends a Wake-on-LAN packet (and no command is sent to MCEC itself). This requires a new MAC Setting for the plugin.

In addition, added a feature where, on connection to MCEC, commands will be sent so that MCEC's modifier state (shift, ctrl, alt, etc up/down) matches the pluginā€™s modifier state. This is probably not necessary, but in theory those could get out of sync and this will allow fixing that easily. In addition, if you wanted to change the pluginā€™s defaults so that, for example, shift starts down, this would support that too, although I donā€™t recommend doing that.

1 Like

This looks amazing! Thanks for taking the time and effort to produce this. Iā€™m in the early stages of learning THR, but my setup pretty much revolves around my HTPC so this will be invaluable when I get to the PC end of things.
:clap: :clap:

1 Like

Fixed a bug where the MouseHelper server would not actually attempt to reconnect if it initially failed to connect to the MCE Controller. This could happen, for example, if the MouseHelper server was started before the MCE Controller. This must have been due to some kind of change in node.js, because this used to work correctly, but I recently got bit by this myself, where I had no mouse control on the HTPC.

Iā€™ve updated the post above with the corrected mhserver.js file.

1 Like

I finally got a decent amount of time at the weekend to really start integrating my HTPC with HR (and this MCEC plugin). Iā€™m blown away by how powerful these tools are. At first I struggled to wrap my head around some of the setup but fast forward a day and (almost) everything is working as expected. Iā€™ve even moved onto creating my own MCEC commands to run PC batch scripts. I can now do complex script-driven processes with a single button press (eg: change the display fps > run an app > force fullscreen > change inputs, etc.)

This is such a big leap forward, it already makes my Harmony Hub redundant and very limited by comparison.

But! (thereā€™s always a but!) There are a few things Iā€™m having issue with on the MCEC/plugin side and hoping you can help me out with the followingā€¦

1. The mouse right-click doesnā€™t seem to work on Android (double-finger tap in the mousepad). Iā€™ve tested this on two Samsung devices (tablet and phone) and nothing happens on both.
*Note: The mouse right-click command does work when placed on a button (I also have L/M/R buttons setup at the side, similar to your example layout above)

Any ideas why this doesnā€™t work on Android? Itā€™s no real biggie as I can use the right-click button instead, but it would be nice if the mousepad worked too for speed/ease of use.

2. How do you deal with long-press (or ā€˜holdā€™) functionality in your setup (if you do at all)? The specific example I have come across isā€¦

Left-click mouse button (HOLD) - I tried to set this up to mimic a real mouse whereby you can left-click-hold and drag a selection (icons or text) to highlight them and then copy/paste, etc. I tried this by adding two event triggers onto a button (Pressed = ā€œMouseLeftDownā€ & Released = ā€œMouseLeftUpā€). I then press this button down and drag over some notepad text to highlight it. This all looks good, but when I cut/copy the text it seems the buffer is empty - nothing is pasted. My suspicion is that the Pressed part is fine (putting the left mouse button down) but as soon as the button is released, the button goes back up ā€˜releasingā€™ the selected text (even though it is still highlighted in notepad) meaning there is nothing to cut/copy.

Have you come across this issue and if so, have you solved it?? I may be missing something here (very likely!) so feel free to shoot my method to ribbons and point out the obvious.

3. In a similar vein to #2 above, how can I mimic long-pressing a key and it repeating the keystroke/input?

Example: On a normal keyboard (or mobile device) if you press and hold the left arrow key, it will keep repeating the left input until you release the key. Is there any way of doing this in HR with the MCEC plugin - maybe using a specific repeat trigger or script??

Thanks in advance,
Shuggs

Hi @Shuggy, Iā€™m glad it is (mostly) working well for you! I recently got so frustrated with my Nvidia Shield that I had to control over IR with a Global Cache that I put together a spare box to be my HTPC again and I have really enjoyed the experience so much more. Iā€™m glad others find it useful as well :slight_smile:

Iā€™m stumped as to why the right-click (2-finger tap) doesnā€™t work in Android. My theater remote has always been an iPad, and Iā€™m almost positive it works there (havenā€™t used 2-finger taps in a while, but will try to play with it this weekend). Unfortunately, I donā€™t have a spare Android tablet to try to reproduce your exact situation.

I will be honest, I wrote the webpage portion of the MouseHelper back in 2015 and basically havenā€™t looked at it since. Even the mhserver.js, while I have updated a couple of things over the years, I havenā€™t really studied since then eitherā€“I got it working and have basically left it alone :slight_smile: All of the touch-sensing is handled by the hammer.min.js library, so it seems most likely if there is a bug for Android, it is in that library. Unfortunately, itā€™s not actively supported anymore. I noticed the github page for it does have some suggested alternatives in the some of the comments/bug reports, so maybe I can find some time to look into using a different library for the touch stuff. If you have any skill with javascript, please feel free to take a stab at it, as I hate javascriptā€¦

One other thing to note, in case it was not apparent, is that when you map a THR button to a mouse button, that is actually sent directly to MCEC from THR by the plugin (just like any of the other MediaCommands). When you tap/doubletap/etc in the MouseHelper, those commends get sent to MCEC via mhserver.js. That would explain why the button works when the 2-finger tap doesnā€™t.

Iā€™m not entirely sure I follow what you tried to set up to address this. However, I can tell you there is no long-press (as I understand that term) support for the mouse buttons in the MouseHelper interface. If you hold a single-finger press (vs a tap; so you donā€™t have to hold super-long, but I think itā€™s set to half a second), then the MouseHelper enters ā€œdraggingā€ mode, and so then it will be as if the mouse button is being held down and you can, for example, drag a window around or a highlighting cursor. Then any other input that isnā€™t a drag (tap, doubletap, another press, etc) will ā€œreleaseā€ from dragging mode and return things to normal. So you shouldnā€™t need to do anything complicated in THR to get that behaviorā€“itā€™s handled in the MouseHelper interface.

That said, that isnā€™t a long-press as I understand the term. Thatā€™s just a hold-down-the-mouse-button-so-I-can-drag/drop-in-the-interface. My understanding of long-press is that you get a different behavior when you hold down the button for some period of time vs a shorter click.

I suppose if you were appropriately quick, you could use the dragging mode to simulate a long-press: press on the MouseHelper interface and then after a brief period of time (long enough for a long-press) tap it again and to the computer it would appear as the left mouse button having been help down for that interval.

I think thatā€™s the best you can do, because hammer.min.js doesnā€™t have any long-press support that Iā€™m aware of.

Having re-read this, I think the dragging mode is probably what youā€™re after here. One possibility (and this may tie in to the 2-finger tap issue you mentioned as well) is that the MouseHelper interface (really, hammer.min.js) can be very sensitive to slight movements of your fingers in the actions. So, for example, right click may not work because your 2 fingers donā€™t come down close enough together in time (in hammer.min.js's opinion) or they move slightly, and so it just doesnā€™t register it correctly (thinks it is a rotate instead of a 2-finger tap or something). I donā€™t think thereā€™s anything you can do about that beyond trying to be more careful (which is annoying). You could try repeatedly 2-finger tapping and see if you are ever able to get a right-click sent. Tying back to the select and drag thing, it is possible your release at the end also involves enough movement that Windows drops the highlighting. But thatā€™s just pure speculation.

This is, I believe, both a THR limitation and an MCEC limitation. THR does not, to my knowledge, support long-presses (or if it does, it is platform dependent), which I think in this context would actually be thought of as ā€œcontinuousā€ presses (again, I think long-press has a different, specific meaning). THR also doesnā€™t support continuous presses as far as I know. I havenā€™t looked into it a ton because itā€™s not something Iā€™ve ever needed (except maybe on arrow keys); I donā€™t know that there is any reliable way to simulate it in THR, but Iā€™m sure others have asked this before.

From a quick look through MCEC's documentation, I donā€™t believe it supports long-presses or continuous presses (except of mouse buttons) either. If what youā€™re after is continuous press, as long as you can get THR to repeatedly fire the button, MCEC would ā€œdo the right thingā€ at least.

Sorry I canā€™t be more help with some of these issues. But please let me know if you find solutions, as I would love to incorporate them. My intention with this plugin and the MouseHelper was to make it as full-featured/general purpose as possible because sometimes itā€™s just nice to have your own keyboard/mouse on your HTPC :slight_smile:

1 Like

Wow! Thanks for the detailed reply and help @hotelfoxtrotnovember. Way above and beyond what I expected.

Yeah, Iā€™ve had various media boxes - the most recent Iā€™m moving from is the Vero4K+. As amazing as that box is, itā€™s starting to struggle with my Kodi skin and various OSMC-related issues that I canā€™t be bothered to keep fixing. We also want to manage Kodi, TV, web browsing and gaming (Steam & emulation) from a single box - so HTPC is the way.

No problemā€¦donā€™t worry about looking into the Android functionality - as I said Iā€™ll just use the right-click button and forget about the double-fingers on the touchpad.

Unfortunately, zero js skill. Iā€™m from an Art/design/UI background so very much on the aesthetic side. However, I do pick up some technical things quickly once I grasp the method.

Aha! I didnā€™t know there was a built-in ā€˜drag-selectā€™ mode in the mousehelper. Iā€™ve since tested this out and it works really well tbh - moreso on the bigger mousepad obviously so it looks like Iā€™m going to stick to using a 10" tablet rather than a smaller mobile screen.

You are correct - I defer to your terminology. I wasnā€™t thinking too much about the terminology when I wrote the post, so yes - a long-press would be a defined longer period of time (1-2 secs, etc) and I was really talking about a hold/continuous mouse (or button) press.

Yeah Iā€™d thought about the ā€˜granularityā€™ issue and that it was either too sensitive or not judged to be close enough together. So I tested a THR page containing only a full screen mousepad on a 10" android tablet and even that didnā€™t work so I donā€™t think itā€™s that (though it is still very possible). If it is down to the sensitivity/granularity, even on a 10" tablet, then Iā€™d need something the size of my 65" TV. :wink:
Regardless, Iā€™ll just use a ā€˜right-clickā€™ button next to the mousepad.

Iā€™ve got a big list of THR/MCEC stuff to test and setup this weekend so I will look into trying to repeat a button press. If I get anywhere with it I will let you know. Thereā€™s probably a way to do it with a script, but alas Iā€™m no scripter.

Absolutely!..Iā€™d love to get as fully-featured a keyboard/mouse setup as possible too. Iā€™ll report back anything of relevance and any other issues I come across. I really appreciate the help and your efforts in producing this in the first place. :+1:

Shuggs

1 Like

@Shuggy one thing just popped into my head so I did a little poking around. You should be able to use the RepeatInterval property on the button to get it to fire multiple times when continuously held down. I havenā€™t personally tested it, but see here. I donā€™t have it enabled (or I just never held it long enough, 250ms is pretty long) on my buttons so that is why I probably never knew you could do it, but THR is pretty awesome :slight_smile:

1 Like