Charlieplexing

Note:  If you are seeing this note, then you are looking at an incomplete draft.  Keep checking back as I complete these series of blog posts over the next week or two.

You might be familiar with Charlieplexing with LEDS, but the same technique can be adapted for switches.   Let’s start by replacing LEDs and resistors with diodes and switches in the familiar Charlieplexing schematic layout.

For greater clarity, we are assuming the microprocessor has weak pullups enabled and can omit the external pullup resisors on the lines. This still looks insane and it gets even worse with more switches.   Let’s find a more systematic way to represent the connections.

All we’ve done here is swap the position of some diodes and switches, but the overall circuit is electrically identical.  Now we can begin to make the relation to a conventional scanning matrix more clear.

In this case, the common connection at the cathode for a group of switches would have been a column in a typical matrix configuration.  Let’s make the similarity even more clear by rearranging like this:

 

Basically, the columns are tied into the row lines.  Since we’re using a given row to select a “column”, we can’t use it to read that row at the same time. Thus, there is one less switch per row.  So with n lines, you can read n(n-1) switches.

When a “column” is selected, the other lines are tri-stated with pullups enabled to serve as inputs to read the rows.  For instance, when column 1 is selected, lines 2 & 3 function as input rows.  When column 2 is selected, lines 1 and 3 become input rows.

#define NUM_COLS (n)
#define NUM_ROWS (NUM_COLS - 1)

uint8_t keyMatrix[NUM_ROWS][NUM_COLS];

uint8_t buttonPressed(uint8_t row, uint8_t col)
{
    return keyMatrix[row][col];
}

void deselectCols(void)
{
    TRIS_GPIO0 = 1;
    TRIS_GPIO1 = 1;
    ...
    TRIS_GPIO(n-1) = 1;
}

void selectCol(uint8_t col)
{
    deselectCols();
    
    // Col LAT should already be set to LOW
    // so just enable the driver out of
    // tri-state mode so it can pull the
    // column low.
    switch (col)
    {
        case 0:
            TRIS_GPIO0 = 0;
            break;
            
        case 1:
            TRIS_GPIO1 = 0;
            break;
        ...    
        case (n):
            TRIS_GPIO(n-1) = 0;
            break;
    }
}

void readRows(uint8_t col)
{
    // Read Rows.  Switches will be active low,
    // so invert the reading.
    
    // Here is concrete example, using 3 GPIO
    // lines in a 3 x 2 matrix.  The key to
    // remember is there's n-1 rows and you
    // cannot read a row from selected col.     
    switch(col)
    {
        case 0:
            keyMatrix[0][col].input = !GPIO1;
            keyMatrix[1][col].input = !GPIO2;
            break;
        case 1:
            keyMatrix[0][col].input = !GPIO0;
            keyMatrix[1][col].input = !GPIO2;
            break;

        case 2:
            keyMatrix[0][col].input = !GPIO0;
            keyMatrix[1][col].input = !GPIO1;
            break;
    }
}

//call every millisecond
void scanMatrix(void)
{
    for (uint8_t i =0; i< NUM_COLS; i++)
    {
        selectCol(i);
        // Give time for voltage levels to stabilize
        __delay_us(50); 
        readRows(i);
        deselectCols();
    
    }
}

void initMatrixIO(void)
{
    //configure GPIO pins
    
    // All digital mode
    ANSEL_GPIO(0..n-1) = 0; 

    // Set all latches low. The latches needs to
    // remain low for the rest of the code
    //  to work. This way they can pull columns
    // low when no longer in tri-state.
    LAT_GPIO1(0..n-1) = 0;

    deselectCols();

    // Enable all pullups.  This is necessary
    // when pin is serving as input.
    // Should do no harm when it's an output.
    WPU_GPIO(0..n-1) = 1;

}

void initKeyMatrix(void)
{
    initMatrixIO();
    
    for (uint8_t i = 0; i < NUM_ROWS; i++)
        for (uint8_t j = 0; j < NUM_COLS; j++)
            keyMatrix[i][j] = 0;
}

The main drawback to Charlieplexing is that it’s no longer possible to resolve the ghosting problem when multiple keys are pressed at once.  The other is that you can’t use a keypad wired in a standard matrix.

I know you’ll stop being my friend if I leave you without a working example, so I hacked together a 5 x 4 Charlieplexed key array from five 1 x 4 key matrix strips.

<USB HID Keyboard example firmware using 5 x 4 matrix>

Ghosting and Masking

Note:  If you are seeing this note, then you are looking at an incomplete draft.  Keep checking back as I complete these series of blog posts over the next week or two.

A straightforward implementation of a scanning matrix works well enough when keys are pressed one at a time.  Pressing multiple keys at once can lead to ghosting and masking.

Ghosting makes it appear a switch has been pressed when in reality it hasn’t.  How does this happen?  Let’s examine the following scenario where the keys at (1,1), (1,3) and (3,3) are pressed simultaneously.

The controller starts by bringing column 1 low and since the switch at (1,1) is pressed, it brings row 1 low as expected.  Great so far!

Uh oh.  Looks like the switch at (1,3) is providing an alternate pathway to column 3, also bringing it low.  If you play a lot of Khet, all that follows should be painfully obvious.

Since the switch at (3,3) is also pressed, it brings row 3 low ahead of schedule; it shouldn’t be low until column 3 is actually selected.  The controller reads row 1 and row 3 as active and thus thinks a switch at (1,1) AND (3,1) have been pressed.

The switch at (3,1) has not been pressed, it’s just a ghost!  Hopefully that wasn’t the button for launching the nukes.

Masking happens in a symmetric fashion.  Imagine if you actually pressed the switch at (3,1) while still holding down the others.  If you were to release the switch at (1,3) it would still register as being pressed because pressing the one at (3,1) created a new ghost right at that spot.  This is the sort of thing that causes so many preventable deaths in the PC gaming world.

Fortunately there is an easy fix by incorporating diodes into the matrix so that current goes where it is intended.

Now let’s revisit the same scenario, only with the diodes inserted as previously shown. The diode at (1,3) becomes reverse biased, eliminating the alternate path along with any masking or ghost effects.

So now you know where babies come from… And why you see so many diodes in those joysticks you have been taking apart.

If diodes can fix this issue, why is the problem still present on keyboards?  Mostly because omitting the diodes saves on cost and the matrix is organized such that the problem doesn’t surface under typical usage.

Matrix Scanning

Note:  If you are seeing this note, then you are looking at an incomplete draft.  Keep checking back as I complete these series of blog posts over the next week or two.

Lately, I have had more of a need for custom input devices, like connecting an two 8-way joysticks and some buttons to my MAME cabinet or converting a really cool 1990s HOTAS controller setup from using a gameport to USB.  Soon, I will need to interface around 100 switches and buttons for my flight simulator cabinet in progress.  Why not use a commercial solution, or hack an existing joystick or keyboard?   It is far more expensive, less flexible and I wouldn’t learn anything in the process.

In any embedded project, one of the biggest constraints is the number of available IO pins.  Using one input pin per switch would be extremely wasteful. Let’s say we have 16 switches as shown:

There is a significant cost savings if we’re willing to trade off pin usage for a little extra latency and complexity in hardware and software.  The way this is done is by multiplexing the switches into a two dimensional matrix, scanning through columns and reading off the rows (or vise-versa.)

The matrix is scanned by pulling each column low in succession (with all others floating tri-stated) and reading the corresponding rows for that column.  If a switch on the selected column is open, its row input will read high due to the pullup resistor, otherwise it will read as a low.  Some microcontrollers have weak pullup resistors that can be enabled, obviating the external pullups shown.

Roughly speaking, the number of pins needed will be twice the square root of the number of switches.  For 16 switches, the number of IO pins needed is down from 16 to just 8 (four rows, four columns.)  That’s a decent improvement, but the advantage is more obvious for 101 key keyboard, which would only need 21 pins.  The main tradeoff is that it takes increasingly longer to scan through the matrix as the number of columns increases.

This approach works well as long as one switch is pressed at a time.  Pressing multiple switches at once can lead to ghosting and masking. <read more>

Here is a rough outline of what the matrix scanning code might look like in C.  The application would call initMatrix() at startup, and then periodically call scanMatrix() to scan the matrix.  Of course to read the status of a particular switch at (row, col), the application would call the function buttonPressed(row, col).

#define NUM_ROWS (n)
#define NUM_COLS (m)

uint8_t keyMatrix[NUM_ROWS][NUM_COLS];

uint8_t buttonPressed(uint8_t row, uint8_t col)
{
    return keyMatrix[row][col];
}

void deselectCols(void)
{
    // This has the effect of turning off the
    // previously selected column 
    TRIS_COL0_SELECT = 1;
    ...
    TRIS_COL(n)_SELECT = 1;
}

void selectCol(uint8_t col)
{
    deselectCols();
    
    // col latch should already be set to low,
    // so just enable the driver out of
    // tri-state mode so it can pull the column
    // down.
    switch (col)
    {
        case 0:
            TRIS_COL0_SELECT = 0;
            break;
        ...
           
        case n:
            TRIS_COL(n)_SELECT = 0;
            break;
    }
}

void readRows(uint8_t col)
{
    // Read Rows.  Switches will be active low,
    // so invert the reading.
    keyMatrix[0][col] = !ROW0_INPUT;
    keyMatrix[1][col] = !ROW1_INPUT;
    ...
    keyMatrix[m][col] = !ROW(m)_INPUT;

}

//call every millisecond or so.
void scanMatrix(void)
{
    for (uint8_t i =0; i < NUM_COLS; i++)
    {
        selectCol(i);

	// Give time for voltage levels to stabilize
        __delay_us(50);      
        readRows(i);
        deselectCols();
    }
}

void initMatrixIO(void)
{
    //configure col output pins
    
    //The latch needs to remain low for the rest
    // of the code to work.  This way they can
    // pull columns low when no longer in
    // tri-state.
    COL0_SELECT = 0;
	...
    COL(n)_SELECT = 0;
    deselectCols();

    (configure row input pins)
    (IE: tristate, enable pullups);
    
}

void initMatrix(void)
{
    initMatrixIO();
    
    for (uint8_t i = 0; i < NUM_ROWS; i++)
        for (uint8_t j = 0; j < NUM_COLS; j++)
            keyMatrix[i][j] = 0;
            
}

For a real world example using matrix scanning for a USB joystick, check this<link> out.

 

Suncom Tactical Flight Controller USB Conversion

Note:  If you are seeing this note, then you are looking at an incomplete draft.  Keep checking back as I complete these series of blog posts over the next week or two.

 

I am starting my own flight simulator cabinet build.  I want it to resemble the F15/16/18s I would see growing up on various Airforce bases and also be a good fit for some of the early 90s arcade flight simulators I played back in the day like Steel Talons.   I figured the best place to start would be the controls.  I went looking around for a nice HOTAS setup.  I really liked Thrustmaster’s Warthog, but I didn’t want to spend $500.  I turned to looking on eBay, finding an old school Suncom SFS Throttle/F-15 Tactical Flight Controller for $50- just in time for Christmas.  Christmas day, I ripped open my present and absolutely loved the look and feel of these controls!

They are amazing controllers, but go cheap because they require an obsolete gameport to connect.  I wanted so badly to try them out, so I hooked them up to a USB gameport adapter I had on hand.  Sadly, it didn’t work for any of the flight simulators I tried.  Even though the gameport adapter works, it appears the controls need special drivers to work.  I found the FUSBA from a cursory Google search on conversion kits, but it costs $100.

The case for a full on USB HID Joystick conversion was mounting- no need for an expensive USB converter or special drivers.  I ended up purchasing two $10  P-Star 25K50 Micro boards from Pololu and converted both controllers to USB myself.  The conversion wasn’t that difficult.  I converted the joystick to use a scanning matrix with some 1N4148 diodes, slightly modified the throttle’s scanning matrix and adapted Microchips USB HID Joystick demo.

I took apart the joystick grip to see what I am working with.

It appears to have some kind of encoding circuitry already, which I promptly desoldered, so I could replace it with a simpler 2 x 4 diode scanning matrix.  I modified the PCB so that I could put the hat switch on one row and the Guns, Missile, Thumb and Pinky switches on another.

Confused about matrix scanning and why there are diodes in the circuit? Check out my blog post on matrix scanning <read more tag here>.

Gameports typically used slope conversion, which measures the time it takes to charge and discharge a capacitor through an input resistance.  So the potentiometers are wired as variable resistors using two wires, which the successive approximation ADC on the P-Star can’t read directly.  To produce the voltage signal the ADC needs, I simply added a connection to the potentiometer that would let it be used as a voltage divider as shown:

If you can maneuver your soldering iron around the small space, I can tell you from experience it might be preferable to dismantling the joystick gimbal and spending the next hour trying to reassemble it.  Later, I will revisit the gimbal with some improvements that address the large amount of backlash present.

Another issue that surfaced was the fact that these potentiometers were 100K, way higher than the maximum 10K source impedance the PIC18F25K50’s ADC on the P-Star can accommodate.  This resulted in inaccurate readings during tests, until I upped the acquisition time (TACQ) to 64 TAD.  In this case TAD was set to 1.3 microseconds (FOSC/64 at 48Mhz.)  It might have been possible to add a small capacitor across the input terminals, but I wanted to keep the component count as low as possible.

Lastly, there’s a beautiful jumbo red LED right next to the hat switch.   Maybe I can have it flash during a missile warning or serve as some kind of master caution.  I wanted to make it accessible to flight simulators and MAME, so I wired it to the P-Star’s RB5 output for around 5 mA of drive current.  Of course I would have to incorporate an additional USB driver to tie into the simulator’s output subsystem, but at least the functionality is enabled for when the time comes.

You can find the software here, how it works <here> and how to modify it <here>.

Almost done with the joystick!  I finished up by organizing the wires, adding strain relief for the USB cable, soldering the headers and P-Star into a breadboard and mounting it as shown:

The throttle is considerably more complicated, probably to implement the keyboard interface and mapping.

I removed the circuit boards, disconnected the headers and reversed engineered the scan matrix.

I modified the matrix to have 3 rows and 8 columns by connecting the blue wire from the right throttle grip to the green wire on the left throttle grip.  Then, I connected the orange, brown, yellow and red lines on the left grip to the brown, purple, black and gray lines on the right grip respectively.

As with the joystick, I had to add a third wire to the potentiometers so that they can be read by the ADC.  I was able to avoid dismantling the throttle assembly by careful maneuvering during soldering.

The throttle’s potentiometers suffered from the same bad readings as those on the joystick and it was easily resolved by increasing the acquisition time to 64 TAD.  I’m not satisfied with the response curve from the linear pot arrangement, so I will be revisiting this soon and replacing them with rotary ones.

Finally, I organized the wires, added strain relief for the USB cable, soldered the headers and P-Star into a breadboard and mounted it as shown:

Of course, the software for the throttle was a pretty straightforward modification of the one I created for the joystick.  I had no problems getting these controls to work (even with Linux) on X-Plane, Warthunder and various MAME arcade games like Steel Talons, Wing war, F-15 Strike Eagle, Air Combat and Starblade.

Not only do I have a nice Joystick and Throttle, but a template for converting other controls, like a rudder or interfacing to things like landing gear, flaps, etc.  In the end, I spent around $75 on this project, so things are off to a good start.  Stay tuned for the rest of the flight simulator cabinet build, more to come soon!

Some progress

I used the Darknet Framework to produce the following images from Edvard Munch’s “The Scream”, similar to what’s done with Google Deep Dream.  It’s small step, but an important one.  The rendering took about 30 minutes, so the next step is to get GPU acceleration working.

scream_vgg-conv_10_000000

scream_vgg-conv_13_000000

Recovering from Surgery

I have had some big life changes lately, should have a lot more time on my hands.  I started an art project.  I haven’t made a formal artist’s statement yet, but the gist is a computer AI that makes art similar to Google Deep Dream or Deep Style.  I am going for something that can’t be dismissed by anyone as a philosophical zombie or merely an artist’s tool. I have started on the technical implementation, which will be using online, unsupervised learning.  Its design is deliberately aimed at removing any doubt that the machine is participating in art and expressing itself.  My approach reduces the programmer’s role as completely as possible and the machine takes on a independent life and existence of its own, growing just as we do.  A lifelong friend said the following about me and my project, which I thought was nice:

“If basically anyone else I know had posted that, I would wonder about their medication balance [due to unrealistic grandiosity]. You posting it, however, and I wonder how long it will take. “

My Goal for Starting a Youtube Channel

My primary goal is to wind up in a cringe compilation. I think a lot of great YouTubers got their start in cringe. Take Sky Williams for instance. I was just watching this cringe compilation long after Katie fell asleep waiting for me to fulfill her and I find cuts from this original video:

Before that moment, I had no idea who this gem of a man was. His thoughts pretty much echo the thoughts and feelings of geeks and nerds everywhere.  I immediately fell in love. You have to give it to the guy because in just under 4 months of posting that video he took care of that problem.

As of the time of writing, the dude is pretty successful with a subscriber base of nearly 900K. The cringe video I found Sky on is here:

There are several other YouTube gems in that video as well. I think the fact that I like and relate to a lot of the people featured in that video is a good sign.

YouTube Channel Ideas

When mentioning the idea of starting an entertainment YouTube channel to some people, I can feel silent judgement that I will be contributing to the mindless consumption plaguing our society. Entertainment is kind of a funny thing. Take things like rap music or crude/low brow humor- people are quick to judge it as base entertainment and dismiss the cleverness and genius behind making it. I think what makes art and entertainment “base” at times is that it’s so easy to consume. It’s not the artists, entertainers or the content that are base. It’s how quick and easily entertainment is consumed- it’s like people swallowing their food whole without really tasting it. Of course, I think the best way to understand something in-depth is to try to go through the motions over making it yourself at least in your head. I get the feeling that intellectualizing every little thing in life actually makes one a bit boring and sterile. Art, entertainment and other forms are an expression of our humanity and to those who regard it with disdain I say you’re missing out on the bigger picture. At a deeper level, science and art/entertainment are intimately connected. If you want to be a truly “deep” person, you should embrace both.

Killer Robot Idea

Starting to think more seriously on my evolvable furby concept. To those that don’t know, the idea is to kickstarter a robot that can evolve true, life-like AI. There will be a few hundred small robots that connect every night to a server over wifi, relaying the amount of sensory input and feedback it got from the environment. The robots receiving the least amount of pain and most amout of pleasure from the environment are ranked the highest. A simple example would be that Pain can be from strong shocks picked up from an accelerometer and Pleasure can be through touch sensors or facial recognition that measures attention. Of course the actual sensors and criteria would be a bit more complex. The highest ranking robots are selected for a reproduction round which receives modifed genetic operators-a combination of particle swarm optimization and conventional genetic programming. The genes, which are located on the server, control the topology and other characteristics of the robot’s neural net- such as the activation function and the equation dictating hebbian learning. The new gene pool is then downloaded by the robots and the cycle repeats. There are serialized memory portions that are retained between cycles that contains the weights for the orginal neurons that havent changed between cycles- this way i hope the robots can retain a bit of their identity and memory of their owner and surroundings. If a firmware update hoses the robot, or makes the owner unrecognizable or whatever, the user can press a button to either receive a different set of genes from the pool, or revert the change. Of course the reversion de-ranks that set of genes in subsequent generations. I’m betting over time the robots would evolve to be something quite life-like and enjoyable for people to interact with, which is the primary selling point to the consumer. The business idea behind all of this that sells the idea to an investor would be to market a fully working model of a strong AI with ever increasing robot capabilities/sophistication. The intial neural net can be derived by evolving the robots in a simulation to give them the ability to navigate, avoid falls, find their charger, etc. most people i mention this to look at me like i am crazy. But it happens too often that i decide not to pursue an idea because of that only to see someone else pull it off. Just gotta find people that think it could work and devote some time to it.