If you do a lot of typing, you probably would have upgraded to a mechanical keyboard. The travel and tactile/auditory feedback from mechanical keys are a joy, while also reducing fatigue by giving fingers a good amount of travel to decelerate rather than bottoming out on a membrane keyboard in the hopes of guaranteeing a key registration.

Since 2009, I’ve been a mechanical keyboard convert and I don’t regret it one bit, given how much typing I do as a part of my study, research, coding and blogging. They’ve been my peripheral of choice – I only tolerate rubber dome membrane keyboards when I “have to” (e.g. out and about on a laptop, or at work using their equipment).

Everything had been going relatively well with Cherry MX switches being considered the “standard” of the time, with a claim of 50 million press lifetimes. I’m not even sure the joints of my fingers could hold up. A typist at 100 words per minute, assuming five presses per word, typing everything onto one key would last close to 70 days of continuous mashing. Given a full size keyboard has over 100 keys … and I couldn’t possibly be typing that fast for that long, I thought that a mechanical keyboard would pretty much be buy it for life. That was … until it wasn’t …

The Problem & Claimed Fixes

The problem is called “chatter”. This refers to the multiple input which is registered on a single keypress. The origins are ultimately electromechanical – likely due to a combination of bounce in the contacts, dust, films or foreign particles affecting contact quality, wear of mechanical parts (possibly contributing the dust) and interactions with the controller or firmware debouncing algorithm. The result is that typing swiftly can result in words looking like “wwords” or “wowrds” or “thata” or “ppress”. Repeated keys can occur so rapidly to be next to one another, other times they are separated by long enough that rapid typists can get a key in-between. Commonly chatter manifests as doubling of a keypress, but severe chatter can cause triple or even more registrations of the key.

In my case, my main keyboard (a Gigabyte Aivia Osmium with Cherry MX Brown switches) had persistent chatter across a few keys which were really affecting my quality-of-experience. The chatter on “P”, “H”, “O” and “A” rank fairly high in the character frequency list in English, resulting in quite a few corrections in my typing (and a few e-mails with interesting typos). This would’ve been okay, if it were not for the fact that “backspace”, “space” and “return” had begun to chatter intermittently – depending on which angle they were struck at. This resulted in even more frustrations when it came to correcting typos and occasionally committing actions in scripts (e.g. “press enter to continue” ended up proceeding two steps, rather than just one). This started to get dangerous.

As a mechanical keyboard fan, my other ol’faithful Daskeyboard Professional (my first mechanical keyboard with Cherry MX Blue switches) didn’t fare much better. It had the opposite problem – the “Y” key started to register only intermittently – so as to “ignore” my presses at least half the time. This could be due to oxide, dirt or an insulating film building up on the key contacts. Despite all of this, I wouldn’t trade away the mechanical keyboard typing experience and go back to rubber-dome membrane keyboards, unless I absolutely had to.

This isn’t a totally unknown issue, but it’s the first time I’ve experienced it myself. Looking online, people have seemingly remedied the issue by drowning the switch in isopropyl alcohol and cycling the switch rapidly. Others have been able to press down on the switch and blast it with air to clean it out. Finally, others have said that using a contact cleaner like DeOxit solved the problem for them. I gave them all a try, but the only result I got was perhaps an improvement for a day or two. Nothing was permanent.

A Software Fix?

Given that the result is multiple key-press inputs, it would be possible to filter the repeated keystrokes, right? That’s what a number of keyboard chatter blocker tools aim to do. I gave the Frenetic LLC Keyboard Chatter Blocker a try.

The tool is small and featureful, allowing the configuration of chatter delay per-key and logging the chatters which were blocked.

The statistics tab is able to show just how many chatters were detected per key, but also indicates the number of keypresses per key. When running this for a while, it’s interesting to note just which keys are being used more than others.

The configure keys tab allows for individual key chatter thresholds to be configured. Alternatively, a global chatter threshold can be set, which means every key would be filtered.

There is the capability to disable the filter for certain programs – perhaps to avoid compatibility issues or problems with anti-cheat, but I’m not sure I can trust this. After all, this program does modify keyboard inputs. Thinking about it, as a piece of software that intercepts all keyboard input, I’m not sure I like that idea in the first place (although I suspect the software is “clean”) – it’s almost like having a keylogger …

There are some other settings which change how chatter is measured and allowing for accumulated statistics.

At the time of publication, the latest version was 1.19. On the whole, I think the software is quite a good effort – it does remove the chatter. But the problem, as a fast typist, is that I couldn’t find a reliable chatter threshold that could filter the chatter without filtering out legitimate double-presses. This created new typos where words like success would turn out as suces. I also didn’t like the potential of having something that could be logging keystrokes in a way that might open up security issues or cause anti-cheat bans. I would have loved for a convenient, free software solution that works, but alas, that was not to be.

Biting the Bullet – Saving the Patient

Come Friday this week, I just got so frustrated I started tearing down the keyboard with a keycap puller. I couldn’t take it anymore … as time-consuming as this was, I wasn’t giving up on the keyboard.

The keyboard in question is a Gigabyte Aivia Osmium, a relatively mainstream mechanical keyboard hailing from 2013 originally purchased to be paired with my secondary Intel 4th-generation desktop. It saw relatively light amounts of use until the last two-and-a-half years when I pressed it into everyday service with my primary laptop-based workstation. Even though it wasn’t at all cheap when I purchased it (around AU$200 from MSY) as mechanical keyboards were considered premium products and innovations such as keyboards with socketed swappable switches had not yet come to the fore, the fact it has lasted over a decade has made it worthwhile. It’s still over 90% functional, so why throw it away?

To be sure I didn’t get the wrong key, I coloured the stems of the bad keys with black permanent marker.

Disassembling the keyboard involves taking out four screws and then prying the case halves apart. The plastic itself took a bit of work to separate and some clips snapped – perhaps after 10 years, the plastic got a bit brittle. Nevertheless, releasing the PCB also requires unplugging the USB 3.0 extension cable, the USB 2.0 and audio combination cable from CON1 and also desoldering the shield drain wire from the PCB.

One potential cause of chatter is bad PCB joints. A crackly joint can certainly manifest itself as an inconsistent switch signal similar to a crackly contact. But a look at the keyboard itself shows relatively nice, regular lead-free solder joints. No signs of cracks or bad joints, although there is evidence of flux residues, none of which were causing critical issues.

Perhaps my biggest incentive to do the repair, aside from the inconvenience of the chatter itself, was the fact I had some of these Cherry MX Blue switches left over from a project that I could re-use (on the right). An extracted switch from the patient is on the left – the Cherry text on the Blue looks suspicious, but this was sourced through Mouser, so I hope it is truly genuine.

I was glad that the keyboard uses the black-framed older Cherry MX switch design, as the clear bodied switches don’t seem suited for LEDs in the same configuration. But a look at the back shows that the Aivia Osmium uses the keyboard plate to secure the keys into position and doesn’t rely on PCB fixation pins. As a result, I used flush side-cutters to trim off the fixation pins, but ordering the correct type in the first place (MX1A-G1NN) would have been better. That being said, given the reduced cost of such mechanical keyboards nowadays (under AU$100), even buying a small quantity of switches would cost a good fraction of what a whole new keyboard would cost especially when considering the shipping, which would make such a venture uneconomical when considering the time requirement of the repair. Having spare old switches to throw at it really made it more palatable to do it … but also the satisfaction of knowing that I’m not throwing away something that works, that I’m familiar with, that I like and that is still 90% functional. Perhaps another option is to buy cheap clone switches and use those instead, but I would then be concerned about lifetime (as disassembly is a pain in the butt).

To make my life easier, I used black permanent marker to ring around and “cross out” bad switches to identify which switches need replacing. To remove and replace them, I used a soldering iron, desoldering bulb, desoldering wick, leaded solder and a pair of tweezers.

My procedure involved:

1. Flowing fresh solder over the two LED legs to join them together. This would reduce the melting point (mixing leaded solder in) and allow for a single iron tip to heat both legs at the same time.
2. Reflow the LED solder and yank the LED out from the other side of the PCB with tweezers slowly. This allows the LED to come out cleanly with minimal damage.
3. Clean the LED pads by using solder wick and/or the desoldering bulb.
4. Flow fresh solder on each of the switch pins. They’re too far to connect together, so they have to be desoldered one at a time.
5. Heat up a single joint until it is quite liquid, then use the desoldering bulb to clean it thoroughly. As the hole and pad are very big, this makes cleaning out the solder even easier.
6. Repeat for the other joint. Once both joints are clear, use the set of tweezers to verify both pins are free from the PCB by wiggling on them.
7. Once clear, push down on the locating peg on the rear to pop the switch through the front plate.
8. Install the new switch into place (trimming off the PCB locating pins) and solder down the two pins.
9. Install the LED ensuring orientation matches the others on the board, then solder it down.

All done, the result doesn’t look so clean because I used Multicore 362 solder with no-clean rosin-flux. This results in amber residues that do not have to be cleaned – so I’m leaving it as it is, along with my permanent marker scrawls.

The resulting key switch swaps would probably irk many people … but I think this makes for an interesting board.

I also took a quick look around the board – this corner seems to have a Genesys Logic USB 2.0 hub and a Freescale MC9S08JM16 microcontroller.

Near the scroll wheel is a Cypress CY7C64215 EZ-USB E3 controller with a separate MXIC 25L4005E 4Mbit serial flash. Modern keyboards would have everything inside a single ARM microcontroller.

One thing I was particularly pleased about was that all the LEDs still worked after the transplant. The LEDs themselves didn’t have a flat on the negative side, so I had to rely on the shape of the “legs” to determine which way for them to go in. The LEDs can also be somewhat heat sensitive as well – but that being said, I rarely use the key backlighting, so even if it didn’t make it, I would have been okay with it.

A Peek Inside the Old Switches

Before I throw away the old switches, I decided to open one of them up to look inside. After all these years, I’ve not looked inside one …

… and I’d have to saw it is quite underwhelming. Most of it was empty space with a spring for the plunger that pushes on the switch contacts which occupy the top of the screen. The cross-point is a small, literal “cross” point.

The switch itself is just the tiny bump in the springy metal – just that part has a flash of gold over it.

That mates with this bump on the other plate. Keeping costs low by reducing the amount of gold used, while the small contact area should make for a reliable connection. Curiosity answered!

A Decade of Repair …

While I do have quite a few soldering irons, I’ve ordinarily used my Tenma (rebadged Atten) hot air station from 2014, which makes it a decade old. In that time, I’ve used it for building circuits, repairing products, doing work from home, desoldering parts and more. Everything from connectors, through-hole and SMD … it’s done it all and without having any parts replaced.

So after ten years of use and a lot of repairs, its time is numbered. The tip has now “thinned” in the middle, becoming more thermally resistive, but even worse is the retaining barrel which has started to split and corrode at its end. This results in a tip that is loose, potentially even on the verge of falling off the front of the iron.

Perhaps it’s all the accumulated flux acidity and wet sponge usage has caught up to the rebadged AT8502D handpiece. Ten years of work – a lot more than I had anticipated it lasting when I purchased it.

Thankfully I had purchased a whole replacement handpiece when it was being cleared out from element14 on discount, so I have a new handpiece and tip for when this one finally fails.

But this has me reflecting back on my electronics journey just like the spool of wire guy, except for me, it’s more like my spools of solder(ing wire) and soldering iron tips.

Conclusion

Electromechanical devices, such as switches, have a lifetime due to the nature of moving parts. While the claimed lives are extremely long – often more than I could ever imagine my finger joints to be capable of, these lives are based on laboratory testing that likely does not reflect real-word usage where keys are being hit at different angles, at different speeds, bottoming out occasionally, in rooms filled with dust and vapours in the air. The result is that, over time, these switches may not provide a “clean” signal much like how an old radio’s volume control can crackle, resulting in multiple keypresses being registered on a single push, a phenomenon known as “chatter”.

There are software solutions that filter the chatter based on filtering out repeated keypresses received within a configurable time window. This may well be a solution enough for some users, as it requires no mechanical intervention and is essentially “free” (albeit, with the risk of falling afoul of anti-cheat detection software). But in my experience, despite tuning it, I found that chatter detection occasionally filtered out legitimate double-presses when typing quickly – and being able to type quickly and comfortably was the primary reason I went mechanical in the first place.

So like it or not, the real fix is a hardware fix, which can be somewhat costly in time and parts. It is possible on such soldered boards to change the switches with a soldering iron, solder, desoldering braid, a desoldering bulb or solder sucker and a set of tweezers. But the amount of time and patience required is probably something not everyone has. Making sure you have the correct replacement part is also important for the smoothest repair.

As I had some left-over Cherry MX Blue switches from a previous project, my repair resulted in a keyboard that now has mixed switches. This would be something that could be very annoying to some – the sound and tactile feel varies depending on the key. But for me, functionality trumps all and I managed to get back up and running, with the original LEDs re-installed for key backlighting as well. While I could have swapped around switches such that the replaced Blue switches were in more inoffensive positions (e.g. in the macro keys, freeing up their Browns to go into the main alphanumeric sequence), this would have increased repair time and risk.

Now that the keyboard is now back to operation, I just hope the other keys don’t begin to chatter anytime soon. But knowing the nature of electromechanical things, it is indeed a possibility and another repair (or perhaps a complete replacement) may ultimately be on the cards. But this had me thinking about just how many things my soldering station has helped me save, now that its iron and tip seem to be on its last legs.

That being said, I previously repaired my Steelseries Sensei mouse more than once due to “double-clicking” issues (which is basically “mouse chatter” or switch bounce). That repair has held up, but since I’m using a Logitech MX518 on my primary laptop set-up, this one too has developed an intermittent double-clicking issue after five years of use. I suppose this might be another candidate for repair (along with the DasKeyboard Professional), once it becomes consistent enough to be annoying. But I do look at my two “spare” mechanical keyboards thinking … when is the right time to “give up” on a patient and just let them die? Not today!