Forums
Tyler G.’s “engagement manager”, Sheila, had a new gig for him. The Global Chemical Society, GCS, had their annual conference coming up, and their system for distributing the schedules was a set of USB thumb-drives with self-hosting web apps.
“You’ll be working with two GCS representatives, Jeff and Graham,” Sheila explained. “They’ll provide you with last year’s source code, and the data for this year’s schedule. You’ll need to wire them up.”
Later that day, the four of them- Tyler, Sheila, and Jeff and Graham- joined a Skype call. Only the top of Jeff’s shiny, bald head could be seen on his webcam, and Graham had joined audio-only.
Sheila managed the introductions. Tyler started the discussion by asking what format they could expect the schedule data to come in.
Jeff shrugged, or at least that’s what they guessed from the way the top of his head bobbed. “Graham, do you know?”
“I think it might be XML,” Graham replied, his voice muffled with static and saturated with background noise. “I can’t say for sure. We’ll send a preliminary data dump first.”
The Blob
The data arrived that afternoon, as a single XML file.
The first time Tyler tried to open it, Notepad++ crashed in protest. After a few attempts, he finally coaxed the editor into letting him see the file. It had no uniform format. Individual fields might be HTML-formatted strings, indecipherable base64-encoded binary blobs (with no indicator as to what data was encoded), and even their plaintext encodings switched from 8-bit to 16-bit arbitrarily.
As soon as Tyler explained to Sheila what a mess the data as, she called GCS reps for another video conferece. Jeff’s shiny pate bobbed around as he listened to their complaints. Sheila finally asked, “Can you do anything to clean up the data?”
“Not really, no,” Jeff replied. “This is how we get the data ourselves.”
“Absolutely not,” Graham concurred.
“We did this last year,” Jeff replied, “and we didn’t have any trouble.”
A Lack of Support
For weeks, Tyler worked on an importer for the XML blob. He figured out what the base64-encoded data was (PDF files), why the encoding kept changing (different language encodings), and why some text was HTML-formatted and some wasn’t (the entries were copied from email, with some as HTML and some as plaintext).
Jeff and Graham had no interest in the action items assigned no them, and continued to be the largest obstacles to the project. They offered no help, they changed their minds nearly daily, and when Sheila started scheduling daily calls with them, they used those calls as an opportunity to be sarcastic and insult Tyler.
Sheila, who had begun the project in a cheerful manner, started balling her fists during each call with Jeff and Graham, now nicknamed “Statler and Waldorf”. After one particularly grueling call, she cursed and muttered dark things about “How do they get anything done?”
After weeks of frustration, pulled hair, and cranky calls, Tyler’s importer was finished. With a few days to go before the conference, they had just enough time to hand the software off and get the USB sticks loaded.
During that morning’s video conference, Jeff and Graham announced that the format had changed to CSV. Sheila, barely keeping her voice level, asked why the format had changed.
“Oh, the industry standard changed,” Graham said.
“And why didn’t you tell us?”
Jeff’s shiny scalp tilted as part of an offscreen shrug. “Sorry. Guess we forgot.”
The Bitter End
The CSV-encoded data, the final and official data-dump for the conference, arrived just one day before the app was due. It came in three files, seemingly split at random, with plenty of repetition between the files. It was all the same, insanely encoded data, just wrapped as CSV rows instead of XML tags.
Tyler crunched his way through an all-nighter. By morning, the importer was finished. He sent the code to GCS’s servers, went home, and collapsed.
The coming Sunday, attendees would arrive at the GCS conference. They would be given a USB stick, that they could plug into their laptops. The conference app would work perfectly, taking the fractured, convoluted data, and presenting it as a scrollable, interactive calendar of panels, presentations, and convention hall hours. Some graduate student, a lab assistant to a Nobel lauerate, would open the app and wonder:
“This programming thing doesn’t seem like a lot of work.”
[Advertisement]
BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!