As a software & electrical engineer, a major part of Nick Johnson’s job is figuring out why some aspect of the complex controlling hardware & software that’s used by industrial-sized equipment isn’t working exactly as it should. Personally, I couldn’t imagine debugging such a beast. Sure, there are plenty of sensors and log files, but they only paint a partial picture: who’s to say the recording equipment isn’t flawed, too?
Thankfully, there are plenty of folks like Nick who actually enjoy the challenge. One of Nick’s most recent assignments was figuring out why the timing on an automated sorting device kept getting thrown off. After several days of analysis, Nick finally isolated the problem to an incorrect speed variation on a particular motor that was responsible for pushing a rod back and forth. The question that remained, of course, was “why?”
As is the norm in situations like this, Nick spent the next few days constructing an elaborate test rig. He ended up with a mostly unmodified subassembly that incorporated a handful of additional sensors and a whole lot of code to monitor the rig. It was all connected to a testing PC through a National Instruments DAQ card over a RS-232 link. Late one Friday evening, he turned on the rig and left it over the weekend.
When Nick got to the office on Monday morning, he eagerly went over to the testing PC to see if the rig had reproduced the bug. However, the PC was having a lot of problems. Every time he tried to move the mouse, the cursor would move up and to the left. Or the right. Or randomly click the whole way around. Obviously, this made it difficult to debug the problem.
Nick’s first guess at what went wrong was the mouse. So, he grabbed a new one, plugged it in, and rebooted. The problem persisted, so he got back to thinking.
And then it occurred to him: someone must have cracked a password and was remote desktopping in to the computer. Nick rushed over and immediately disconnected the network adapter. But still, the mouse was acting wonky.
He performed a full virus scan, rebooted a few more times, replaced the mouse again, and still, the mouse was going haywire. At a loss, he went out to get some coffee. And that’s when it dawned on him.
Can you guess?
Over the weekend, the sensor sent some magic byte sequence over the RS-232 link, and Windows decided that the whole test rig was an old serial mouse. Sure enough, Nick checked the system’s device manager and saw a serial mouse installed. He unplugged the sensor, disabled that device, and it all started working again.
Lesson learned? Sometimes you can be too user friendly.