Vigyan Ashram Chemistry Lab

There are often two types of software development departments mentioned: the kind where software is the product, and the kind where software enhances or sells the product. ChemCo is a third type: a physical chemistry lab, one with extensive customization of lab setups and computer-controlled devices that need to be programmed, as well as a need for statistics and simulations to handle the results. The team includes one C/LabVIEW magician, one Octave specialist, one Java developer, and one Python scripter. Therefore, most of the computer-controlled setups have LabVIEW GUIs and C DLLs for the logic, though some have Python over top of the DLLs instead.

The year is 2016, and ChemCo has just bought a new signal generator. The team's newest engineer, one who knows only Perl, Octave, and R, is tasked with writing a DLL that can be called from their LabView application to control the generator. Faced with learning either C or C++, he chooses C because "the K&R is less than 300 pages while Prata has more than 1200."

The device speaks SCPI, but since ChemCo has a real problem with Not Invented Here syndrome, the new guy writes the library from scratch. During the period that follows, he discovers:

  • Fixing all crashing bugs in one's first C project doesn't mean it's now bug-free or even close to bug-free.
  • C stdio is unsuitable for serial ports in general because it buffers and tries to seek(), so the library gets its own I/O layer that speaks native WIN32 or POSIX handles instead of doing fprintf(port, ":system:beeper:state %d", enabled).
  • The device may send an additional '\0' byte for its replies, depending on whether it's connected via USB or a Bluetooth serial port, which his code mistook for a string terminator.
  • Writing repetitive getters and setters can get fairly boring, but macros (especially X-macros) may be used to generate a lot of functions following the same pattern.

Soon, our intrepid newbie discovers IUP and writes his own simple GUI for "testing reasons." On the lab's distro of Linux, IUP uses GTK as its backend, which calls setlocale(LC_ALL, "") on startup. Now all the string formatting is ruined, as the system is set to use a decimal comma instead of a decimal point. So the newbie rewrites his string layer of the library, starting with a temporary setlocale(LC_ALL, "C") and then back on every string operation, continuing with a hand-grown parser/formatter for numbers, and finally settling on a small piece of C++ code employing std::basic_ios::imbue.

It's around this time that the LabView magician notices that the generator is designed for high-impedance inputs, meaning it can't even control the low-impedance hardware. The project goes dormant.

The year is now 2020. Not only has the lab gotten some new hardware with high-impedance inputs, there's a grant deadline approaching too fast for comfort, so they need that signal generator up and running yesterday. The LabVIEW magician decides it's easier to reimplement just the parts he needs instead of pulling in the no-longer-newbie's bug-laden mess of a project. He asks a few questions (such as "So it's just strings in both directions?" and "I can't use stdio?") and sets off working on his own code.

There's no time for testing; it's grant season nownownow, and they need results without a moment to lose. Besides, testing means having test rigs or test hardware, and the only hardware they have in the lab is production hardware. Another LabVIEW monstrosity is built, and a long procedure involving at least 5 different pieces of hardware all from different manufacturers begins. Motors go "wrrrrr", lasers go "click-click", camera goes "huummmmmmmmm". The assembled techs let out their breaths and go to lunch, letting the program run its 30-minute schedule.

When they return, the system is in disarray. The signal generator had apparently stopped responding to commands, left the laser on, and let it cut all the way through the sample. Scratching their heads, they finally determine that the generator returns to normal when something drains the serial port of all pending input—something the newbie's library was doing to determine success, but the experienced tech hadn't bothered with. The magician asks a few more questions ("So it only replies with 'ok\r\n' or '?x\r\n'? So it's always four bytes?"), twiddles some code, and declares the problem solved: pending input is drained on startup, and four bytes are read every time a command is sent.

Experienced C devs will have guessed the issue by now, but our techs were too busy high-fiving to notice the problem. They run another test, take another lunch break, and the program fails again, this time further in the sequence—exactly 5 times the number of commands previously run, in fact. After more head-scratching, the newbie takes a look at the hex dump of what the generator sent back and finally notices the problem:

'o' 'k' '\r' '\n' '\0' 'o' 'k' '\r' '\n' '\0' 'o' 'k' '\r' '\n' '\0' 'o' 'k' '\r' '\n' '\0'

That pesky terminating '\0' was back, and the newbie hadn't thought to warn the magician about it because it had been so long he'd plum forgotten. That wasn't the only problem they had to fix, but it was the most memorable for sure. Nevertheless, they made their grant deadline with half a day to spare, and the work continued on. Never a dull moment when you're working with custom chemistry equipment, that's for sure!

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.