MCE Controller Plugin for Keyboard and Mouse Interface

This is a plugin for MCE Controller by Charlie Kindel (https://tig.github.io/mcec/). In addition, I am including a helper server I wrote that provides a trackpad-like mouse interface that also sends commands to MCEC. In combination, this allows you to have a fully-functional, network-based keyboard and mouse for a HTPC available in Home Remote.

The MCEC plugin supports two Settings: Host and Port. Host should be the hostname/IP address of the computer running MCEC and Port will default to 5150, MCEC’s default port.

I know others have written plugins that interface with MCEC for fairly specific use cases. This plugin is different in that it is intended to be general-purpose and could, in theory, be used (hopefully without any modification) for any situation. It supports a very large number of MediaCommands, most of which are self-explanatory. The general groupings are: standard Home Remote MediaCommands (basic navigation and simple numbers); modifier commands (for controlling shift/control/alt/win keys), key commands (corresponding to keys on the keyboard, which can be impacted by the modifier keys, such as upper casing and non-alphanumeric symbols), and mouse commands (just various mouse button inputs–these are useful in combination with the mouse helper server discussed below). The modifier and key commands are prefixed with “Letter”, such as LetterShiftOn and LetterA. The mouse commands are prefixed with “Mouse”, such as MouseLeftClick.

In addition, there is a special MediaCommand that will pass any command through to MCEC. You send this by prefixing the MCEC command with “MCEC:”. So, for example, sending the MediaCommand MCEC:chars:Test! would send MCEC the chars:Test! command.

One thing to be aware of that could be a little tricky is that my initial use case was to simulate a keyboard. Thus, the commands are structured to send what the key-presses are, and not what would be typed on the screen. To give an example, to type an exclamation point ("!"), you would, as on an actual keyboard, first turn on the shift with LetterShiftOn and then hit the “1” key with Letter1. You’d probably then want to turn off the shift with LetterShiftOff. I ended up taking this approach for two reasons: (1) it translates pretty well to my use case and (2) more critically, it is actually very difficult to, in a general purpose way, send specific values to be typed rather than the corresponding key to be pressed due to the way MCEC works. That said, I did originally intend to make commands such as LetterCapitalA and LetterExclamationPoint that would always type a capital “A” and “!”, for example. So, there is the beginnings of a framework to support such commands in the plugin, and if there is interest, I would be willing to extend it further to support that behavior as well, to the extent possible. Lastly, the plugin is written for key presses on a standard US keyboard layout. It is possible there are some key mappings that would have to change for other layouts. I’d be happy to share what I know if someone has a need along those lines.

Regarding the mouse helper interface, this is something I wrote back when I was using iRule. It is unchanged since then and still works great for me. I wanted to preserve it here and provide it in case it would be helpful to others. It is basically a very simple web server that serves up a single web page. The web page uses a library to detect various touch interactions and sends those to the server via a websocket. The server opens a connection to MCEC to pass through the commands from the web page to MCEC. You would run it on the same machine as MCEC is running on. In Home Remote, you just use a WebBrowser control to load the web page and then use it as a touch pad to control the mouse on the HTPC via MCEC’s mouse support. Assuming the Home Remote app and your device support it, the webpage supports single/double left click with a single/double tap and single/double right click with a single/double two-finger tap (and maybe a few other things, it’s been a long time since I’ve dug through the code). In addition, a scroll region is provided which lets you scroll a page up or down. I find it fairly intuitive, but please let me know if you have questions. In the interest of full disclosure, this concept was based on something someone named “Steve” posted to the iRule forum. I ended up re-writing the whole thing to get it to work on my setup.

Here is a screenshot of the keyboard and mouse interface I use for my theater remote:

When combined, you get a handy network-based keyboard and mouse which is nice for doing various tasks on your HTPC, including general maintenance and setup, and you don’t have the problem of a wireless/Bluetooth keyboard not having enough range to work reliably. One last thing, though: certain Windows prompts (the admin/UAC ones) will not respond to MCEC’s virtual mouse, so you’ll sadly still need to get out of your chair for a wireless keyboard on those rare occasions they pop up.

Here is the plugin code, last updated: 2020-04-28:
MCEController-2020-04-28.plugin (16.7 KB)

plugin.Name = "MCEController";
plugin.OnChangeRequest = onChangeRequest;
plugin.OnConnect = onConnect;
plugin.OnDisconnect = onDisconnect;
plugin.OnPoll = onPoll;
plugin.OnSynchronizeDevices = onSynchronizeDevices;
plugin.PollingInterval = -1;
plugin.DefaultSettings = {
    "Host"  : "192.168.1.1",
    "Port"  : "5150",
};

var VERSION = "2020-04-28";

/*
    This is intended to be a flexible, general purpose interface to MCEC (the MCEC Controller
    (https://tig.github.io/mcec/)).  Consequently, it provides a large number of defined
    MediaCommands along with some additional functionality to make using it fairly easy in
    many situations.  It can, of course, be customized further, but the intention is to avoid
    the need for any customization.  The initial use cases this was written for were to provide
    a fairly simple interface to Plex Home Theater/Plex Media Player and a full-featured keyboard.
    However, it should allow for interfacing with MCEC in an intuitive way for (hopefully) most
    situations.

    There are several categories of MediaCommand mappings below, each of which may be treated
    differently, but which are all combined to be available as MediaCommands that can be sent by
    Home Remote user interface elements, as usual.  Here is a brief overview of each grouping:
        mediaCommandStandardMappings - these are standard Home Remote MediaCommands
        mediaCommandModifierMappings - these are commands related to modifier keys (shift, control,
                                    alt, and win).  They provide several different ways to control
                                    the up/down state of the modifier keys.
        mediaCommandKeyMappings - these commands correspond to specific keys on the keyboard.
                                    That is, not necessarily the values of those keys, but the
                                    actual keys themselves.  For example, if you want to press
                                    the "1" key, you would use the MediaCommand "Letter1".  When
                                    you let MCEC handle the modifier keys as well, that can result
                                    in either a "1" (shift is not pressed) or a "!" (shift is
                                    pressed) being typed.  This works very well in the case of
                                    emulating the behavior of typing on a keyboard and removes the
                                    complexity of having to configure buttons to send different codes
                                    in the user interface for shifted keys.  Note that, Letter1-0 are
                                    treated differently from Number1-0.  The former are treated
                                    as number keys on the keyboard and the latter are treated as
                                    keys on the number pad (and are defined in the standard HR
                                    mappings instead).
        mediaCommandMouseMappings - these commands send mouse button commands to MCEC.
    
    In addition to the MediaCommand mappings discussed above, there is a special MediaCommand
    supported.  Any MediaCommand sent to the plugin that begins with "MCEC:" will be sent
    as-is to MCEC (with the "MCEC:" prefix removed, of course).  So, for example, to send
    MCEC the "restart" command, you would use the MediaCommand "MCEC:restart".

    This plugin is written for the standard US keyboard layout and sends the virtual key ("vk_")
    codes that correspond to the appropriate keys in that keyboard layout.  For alphanumeric
    keys, it will probably work fine on many other layouts.  However, for non-alphanumeric characters
    (for example, `, ~, [, ], ;, :, ', ", etc.) other keyboards may use different virtual key codes.
*/

var socket = new TCPClient();

var MCEC_RAW_CMD_PREFIX = "MCEC:";
var MCEC_RETRY_ATTEMPTS = 3;

// These are used to track the up/down state of modifier keys.
// NOTE: while tracked, these states are not currently used, but would be needed if this plugin
//      was extended to support commands to send literal values rather than keys (a feature
//      that was considered but not (yet) implemented).
var modifierStates = {
    "shift" : false,
    "ctrl"  : false,
    "alt"   : false,
    "lwin"  : false,
    "rwin"  : false,
};

// These are standardized Home Remote MediaCommand mappings.
var mediaCommandStandardMappings = {
    "DirectionUp"       : "vk_up",
    "DirectionLeft"     : "vk_left",
    "DirectionDown"     : "vk_down",
    "DirectionRight"    : "vk_right",
    "Select"            : "vk_return",
    "Enter"             : "vk_return",
    "Back"              : "vk_back",

    "Number1" : "vk_numpad1",
    "Number2" : "vk_numpad2",
    "Number3" : "vk_numpad3",
    "Number4" : "vk_numpad4",
    "Number5" : "vk_numpad5",
    "Number6" : "vk_numpad6",
    "Number7" : "vk_numpad7",
    "Number8" : "vk_numpad8",
    "Number9" : "vk_numpad9",
    "Number0" : "vk_numpad0",
};

// These commands are for controlling the state of modifier keys.
var mediaCommandModifierMappings = {
    "LetterShift"       : "shift",
    "LetterShiftOn"     : "shift",
    "LetterShiftOff"    : "shift",
    "LetterControl"     : "ctrl",
    "LetterControlOn"   : "ctrl",
    "LetterControlOff"  : "ctrl",
    "LetterAlt"         : "alt",
    "LetterAltOn"       : "alt",
    "LetterAltOff"      : "alt",
    "LetterWin"         : "lwin",
    "LetterWinOn"       : "lwin",
    "LetterWinOff"      : "lwin",
    "LetterLWin"        : "lwin",
    "LetterLWinOn"      : "lwin",
    "LetterLWinOff"     : "lwin",
    "LetterRWin"        : "rwin",
    "LetterRWinOn"      : "rwin",
    "LetterRWinOff"     : "rwin",
};

// These commands correspond to specific keys.  That is, not necessarily
//  the unshift/shifted values of those keys, but the actual keys themselves.
//  When you let MCEC handle the modifier keys as well, using these will have
//  the behavior of typing on the keyboard.  This removes the complexity of
//  having to send different codes in the user interface for shifted keys.
var mediaCommandKeyMappings = {
    "LetterA"  : "vk_a",
    "LetterB"  : "vk_b",
    "LetterC"  : "vk_c",
    "LetterD"  : "vk_d",
    "LetterE"  : "vk_e",
    "LetterF"  : "vk_f",
    "LetterG"  : "vk_g",
    "LetterH"  : "vk_h",
    "LetterI"  : "vk_i",
    "LetterJ"  : "vk_j",
    "LetterK"  : "vk_k",
    "LetterL"  : "vk_l",
    "LetterM"  : "vk_m",
    "LetterN"  : "vk_n",
    "LetterO"  : "vk_o",
    "LetterP"  : "vk_p",
    "LetterQ"  : "vk_q",
    "LetterR"  : "vk_r",
    "LetterS"  : "vk_s",
    "LetterT"  : "vk_t",
    "LetterU"  : "vk_u",
    "LetterV"  : "vk_v",
    "LetterW"  : "vk_w",
    "LetterX"  : "vk_x",
    "LetterY"  : "vk_y",
    "LetterZ"  : "vk_z",

    "Letter1"  : "vk_1",
    "Letter2"  : "vk_2",
    "Letter3"  : "vk_3",
    "Letter4"  : "vk_4",
    "Letter5"  : "vk_5",
    "Letter6"  : "vk_6",
    "Letter7"  : "vk_7",
    "Letter8"  : "vk_8",
    "Letter9"  : "vk_9",
    "Letter0"  : "vk_0",

    "LetterEscape"      : "vk_escape",
    "LetterBackspace"   : "vk_back",
    "LetterTab"         : "vk_tab",
    "LetterSpace"       : "vk_space",

    "LetterBackQuote"   : "vk_oem_3",
    "LetterHyphen"      : "vk_oem_minus",
    "LetterEqual"       : "vk_oem_plus", // despite the name, this really is the equal key

    "LetterLeftSquareBracket"   : "vk_oem_4",
    "LetterRightSquareBracket"  : "vk_oem_6",
    "LetterBackslash"   : "vk_oem_5",

    "LetterSemicolon"   : "vk_oem_1",
    "LetterSingleQuote" : "vk_oem_7",

    "LetterComma"       : "vk_oem_comma",
    "LetterPeriod"      : "vk_oem_period",
    "LetterSlash"       : "vk_oem_2",

    "LetterInsert"      : "vk_insert",
    "LetterDelete"      : "vk_delete",
    "LetterHome"        : "vk_home",
    "LetterEnd"         : "vk_end",
    "LetterPageUp"      : "vk_prior",
    "LetterPageDown"    : "vk_next",

    "LetterF1"  : "vk_f1",
    "LetterF2"  : "vk_f2",
    "LetterF3"  : "vk_f3",
    "LetterF4"  : "vk_f4",
    "LetterF5"  : "vk_f5",
    "LetterF6"  : "vk_f6",
    "LetterF7"  : "vk_f7",
    "LetterF8"  : "vk_f8",
    "LetterF9"  : "vk_f9",
    "LetterF10" : "vk_f10",
    "LetterF11" : "vk_f11",
    "LetterF12" : "vk_f12",
};

// These commands are for mouse buttons.
var mediaCommandMouseMappings = {
    "MouseLeftClick"        : "mouse:lbc",
    "MouseLeftDoubleClick"  : "mouse:lbdc",
    "MouseLeftDown"         : "mouse:lbd",
    "MouseLeftUp"           : "mouse:lbu",

    "MouseRightClick"       : "mouse:rbc",
    "MouseRightDoubleClick" : "mouse:rbdc",
    "MouseRightDown"        : "mouse:rbd",
    "MouseRightUp"          : "mouse:rbu",

    "MouseMiddleClick"      : "mouse:mbc",
    "MouseMiddleDoubleClick": "mouse:mbdc",
    "MouseMiddleDown"       : "mouse:mbd",
    "MouseMiddleUp"         : "mouse:mxbu",
};


function mcecConnect() {
    if(!socket.isConnected) {
        var host = plugin.Settings["Host"];
        var port = plugin.Settings["Port"];
    
        console.log("  Host: " + host);
        console.log("  Port: " + port);
    
        try {
            socket.connect(host, port);

        } catch(err) {
            // unable to connect, that's not good...
            console.log("ERROR: unable to connect to MCEC: " + err.message);
            throw err;
        }
    }
}

function mcecDisconnect() {
    if(socket.isConnected) {
        try {
            socket.close();

        } catch(err) {
            // we're closing down anyway, so it doesn't matter if the close()
            //  failed--just fail silently
        }
    }
}

// wrapper that tries to make sure we're connected before sending
//  returns true if the send worked, false otherwise, to make
//  having resend logic simple
function mcecSend(c) {
    if(!socket.isConnected) {
        // not connected for some reason, try to connect first
        mcecConnect();
    }

    try {
        // send the command
        // NOTE: currently at least some exceptions from TCPClient.send() cannot be caught, which means
        //  this try/catch still won't gracefully handle a failed send.  This is definitely the case
        //  when the socket appears to be connected but the other end isn't there anymore.  The send()
        //  call will throw an uncatchable exception.  Hopefully this will be fixed in a later version
        //  of Home Remote/TCPClient.
        socket.send(c);

    } catch(err) {
        // send didn't work for some reason (probably connection closed by other end and we
        //  just found out about it)
        console.log("ERROR: unable to send to MCEC: " + err.message);

        // let the caller know it didn't work
        return false;
    }

    // seems to have worked okay if we got this far
    return true;
}

function mcecGetCommands(mc) {

    // NOTE: the reason we use an array is because, in the future, it may be
    //      the case that a single MediaCommand requires multiple MCEC commands
    //      to be sent, such as in the case of literal value MediaCommands to
    //      control modifier key states, for example.
    var mceCmds = [ ];

    if((mc.slice(0, MCEC_RAW_CMD_PREFIX.length)) == MCEC_RAW_CMD_PREFIX) {
        console.log("  MCEC raw command identified: " + mc);

        // extract the MCEC raw command and use it as-specified
        var mceRawCmd = mc.slice(MCEC_RAW_CMD_PREFIX.length);
        if(mceRawCmd) {
            mceCmds.push(mceRawCmd);

        } else {
            console.log("ERROR: '" + MCEC_RAW_CMD_PREFIX + "' must be followed by a command to send");
        }
    
    } else if(mediaCommandStandardMappings.hasOwnProperty(mc)) {
        console.log("  standard HR command identified: " + mc);

        // standard commands have no special processing needed
        mceCmds.push(mediaCommandStandardMappings[mc]);

    } else if(mediaCommandModifierMappings.hasOwnProperty(mc)) {
        console.log("  modifier command identified: " + mc);

        // handle modifier commands by updating our internal modifier state for
        //  the relevant modifier key and then building command to send to MCEC
        var modKey = mediaCommandModifierMappings[mc];

        if(modifierStates.hasOwnProperty(modKey)) {
            // because apparently we don't have endsWith()...
            if(mc.slice(-2) == "On") {
                modifierStates[modKey] = true;

            } else if(mc.slice(-3) == "Off") {
                modifierStates[modKey] = false;

            } else {
                // toggle relevant shift state
                modifierStates[modKey] = !modifierStates[modKey];
            }

            console.log("  modifier state for " + modKey + " is now " + modifierStates[modKey]);

            // set up command to send to MCEC
            mceCmds.push(((modifierStates[modKey]) ? "shiftdown:" : "shiftup:") + modKey);

        } else {
            console.log("ERROR: unknown modifier key: " + modKey);
        }

    } else if(mediaCommandKeyMappings.hasOwnProperty(mc)) {
        console.log("  key command identified: " + mc);

        // keys do not require special processing
        mceCmds.push(mediaCommandKeyMappings[mc]);

    } else if(mediaCommandMouseMappings.hasOwnProperty(mc)) {
        console.log("  mouse command identified: " + mc);

        // mouse commands do not require special processing
        mceCmds.push(mediaCommandMouseMappings[mc]);
    
    } else {
        // unknown command
        console.log("ERROR: unsupported command: " + mc);
    }

    return mceCmds;
}

function onChangeRequest(device, attribute, value) {
    console.log("onChangeRequest called...");

    switch(attribute) {
        case "MediaCommand":

            var cmds = [ ];

            // get the appropriate MCEC command(s) for the specified
            //  Home Remote MediaCommand
            cmds = mcecGetCommands(value);

           // now we're ready to send the command(s)
           if(cmds.length) {
                for(var c = 0; c < cmds.length; c++) {
                    console.log("  sending MCEC command: " + cmds[c]);
                    // try to send the command; if it doesn't work, retry
                    for(var attempt = 0; attempt < MCEC_RETRY_ATTEMPTS; attempt++) {
                        if(mcecSend(cmds[c] + "\r\n")) {
                            // that send worked, break out of the retry loop
                            break;
                        }
                    }

                    // done with that command, move on to the next one...
                }
            }

            // all done
            break;

        default:
            console.log("ERROR: unsupported command type: " + attribute);
            break;
    }
}

function onConnect() {
    console.log("onConnect called...");

    var device = plugin.Devices[plugin.Name];

    if(device) {
        console.log("  building supported media commands...");
        
        var mediaCommandMappings = [];

        // build all of our actual MediaCommands
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandStandardMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandModifierMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandKeyMappings));
        mediaCommandMappings = mediaCommandMappings.concat(Object.keys(mediaCommandMouseMappings));        

        // add the special MCEC raw command prefix
        mediaCommandMappings[MCEC_RAW_CMD_PREFIX];

        console.log("  setting supported media commands...");
        device.SupportedMediaCommands = mediaCommandMappings;
        console.log("  set (" + device.SupportedMediaCommands.length + "):");
        console.log(device.SupportedMediaCommands);
    }

    // connect to the MCEC server
    console.log("  connecting...");
    mcecConnect();
    console.log("  connected");
}

function onDisconnect() {
    console.log("onDisconnect called...");
    mcecDisconnect();
    console.log("  disconnected");
}

function onPoll() {
    console.log("onPoll called...");
}

function onSynchronizeDevices() {
    console.log("onSynchronizeDevices called...");

    var device = new Device();

    device.Name = plugin.Name;  // override default naming conversion so it looks nicer
    device.Id = plugin.Name;
    device.DisplayName = "MCE Controller";
    device.Icon = "Remote";
    device.Capabilities = [ "MediaControl" ];
    device.Attributes = [ ];

    plugin.Devices[plugin.Name] = device;
    
    console.log("  done syncing");
}

Here is the “mhserver.js” file:
NOTE: because of fourm limitations, you need to save this/rename it as “mhserver.js” after downloading (that is, remove the “.txt” extension):
mhserver - 2015-09-12-js.txt (5.0 KB)

Here is the “index.html” file:
index - 2015-09-12.html (9.2 KB)

1 Like

HFN:

To be clear, the MCE Controller plugin you wrote… executes the keystrokes and mouse input on the same computer which is running MCE, right? I mean, lets say you have a computer \Plex which is running Plex Server and has access to your media library. This plugin would let you virtually type (e.g. to search for a title) on \Plex. On the other hand, it would not, for example, let you send keystrokes to another device, such as an Apple TV or a Roku. Is that correct?

If I understand correctly, then I believe the answer to your question is “yes”. The MCE plugin opens a connection to the MCE Controller (MCEC) server that is running on whichever computer you want the input to be received on. To slightly clarify your example, if you have a HTPC that is running some kind of Plex client (Plex Home Theater, Plex Media Player, Plex for Windows, etc) (which may be the same computer as the one running the Plex Media Server, but it doesn’t have to be), then you would run MCEC on the HTPC in server mode and the MCE plugin would connect to it and let you send the keyboard commands as if you were typing on the HTPC directly.

Similarly, if you were to also run the MouseHelper on the the HTPC, it would also connect the the MCEC server and provide a webpage (that you can open via a WebBrowser control in Home Remote) that acts as a trackpad to send mouse commands to the HTPC also as if you were using its mouse directly.

The MCE plugin (or the MCE Controller itself) cannot send keystrokes to another device like an Apple TV or Roku. Instead, you would want to use a plugin that can talk to those devices to send them keystrokes. I have also written a Roku plugin that supports sending keyboard commands (for this very use case): Roku (HFN) Plugin

In order to avoid having to create multiple keyboards to use with these different devices, I made sure that the MediaCommand names for sending the keyboard characters were the same between them. I then created a Keyboard page that I embedded in different pages via a Device Browser control so that the MediaCommands were sent to the correct device without having to customize the keyboards:

So I hope that clarifies things for you, but if not, please let me know–I’m happy to help! :slight_smile:

A couple of quick additional notes: First, the screenshot of the Plex Keyboard above doesn’t show the MouseHelper webpage correctly because I brought it up in the simulator which cannot normally talk to my HTPC (on a different network)–it normally looks like the screenshot in the original post.

Second, MCE Controller/Windows have some limitations that I have encountered, but managed to work around most of the time. For example, if a User Account Control (UAC) window pops up, Windows will not accept inputs from MCEC’s virtual keyboard (I’m guessing for security reasons), so the plugin and MouseHelper’s inputs will be ignored while it is on screen. This mostly comes up when software (such as PMP) needs to installed/updated and I have a separate bluetooth keyboard I use on those occasions (with much aggravation). If there are other situations where Windows is ignoring the virtual keyboard inputs of MCEC, the plugin/MouseHelper won’t work, but that is a limitation of MCEC/Windows and not the plugin/MouseHelper. I think the only places I’ve encountered it is in the UAC prompts and possibly when applications running as Administrator have focus.

1 Like