It was a common occurrence for Jarosław F.’s brother, Andrzej, to call him for help. Andrzej worked as a data-entry specialist for the Polish Ministry of Employment, while Jarosław was part of the ministry’s IT department and moonlighted as a Python programmer.

One day, Jarosław answered another call from Andrzej. “No sane person can do this job,” Andrzej started. “There just aren’t enough hours in the day.”

“‘Hours in the day?’ You enter data and run reports using web software. It’s not that difficult.”

Bundesarchiv Bild 183-1987-1112-014, Dresden, Bibliothek, Ausleihe, Katalog

“No, you don’t understand. The ministry’s starting a new labor force program, and I’m in charge of the data entry. But there are too many records to enter by hand.”

“Andrzej, I’m really tired of this,” Jarosław said. “I’m going to write you a script to help with your work, and in exchange you don’t call me at work for six months. Deal?”

Eternity in a Cell

Jarosław had never actually seen the software Andrzej used to enter data. Just getting to Andrzej’s department required signing an authorization form at the front office, then a half-hour wait until his supervisor gave the go-ahead. But Jarosław knew writing the script for his brother wouldn’t take very long.

Andrzej showed Jarosław to his desk and opened a browser. The web application looked current circa 1998, as if someone mocked up a Windows GUI in Internet Explorer 4. There were rows and rows of grey tabular data, text set in black Tahoma. Next to each cell was a button marked Edit.

“You press the button to edit each cell, which takes a few seconds to load,” Andrzej said, demonstrating. “Each cell, and every row, must be entered one at a time”

“Surely there’s a ‘bulk import’ somewhere.” Jarosław scrolled down the page, looking through buttons labelled Income Report and Demographic Report. There was nothing marked that looked remotely like a bulk import feature. “How long does it normally take you to enter a row?”

“Five minutes. I usually get about 10 records an hour. There are 400 records, and my boss needs them done by the end of Sunday.” That left five days. “I still have reports to run while I do this. That’s why I need your help.”

Parallel Punishments

“Okay,” Jarosław said, “can’t you get someone to help you with this? Five people entering data would mean it’d get done in a day.”

“Even if we had that many people to spare, there can’t be more than one person entering data at a time. The system would crash.”

Jarosław chuckled. “Oh, come on. No web application forces a single user to enter data.”

“A year ago, we tried using two people at a time,” Andrzej whispered. “A whole dataset was corrupted. It set back the department’s mandate by six months, and my last supervisor was fired because of it. My new supervisor won’t tolerate any further delays.”

A CRUD-like web interface, with no parallel data entry allowed, it seemed deliberately designed to make one person’s life miserable … and that person was Andrzej.

“It’s a good thing I work in IT,” Jarosław said. “Because what you need is a tailor-made import script, and I’m the one who can write it for you.”

Automated Manual Entry

First, Jarosław installed Python on Andrzej’s computer. If Jarosław were about to exploit the ministry’s applications, he wanted to use a language he was familiar with. Next, he entered a few sample rows from the dataset Andrzej was burdened with. He logged every HTTP request the web application made, making a list of calls his script would need to make. Finally, he converted the Excel file the data came in to .csv so his script could consume it. It was tedious, but not as tedious as Andrzej’s burden.

“Let the script run overnight,” Jarosław said. “If anyone asks, tell them I did this on my own initiative. It should be done by morning.”

The next morning, Andrzej himself went to see Jarosław in the IT department. He looked relieved. “You just saved me four days of manual data entry.”

“You can use it on other datasets,” Jarosław said. “Just convert the files to .csv and pass in the filename as an argument from the command line. And make sure no one else is entering data while it’s running.”

“Thanks,” Andrzej said. “So, call you in six months?”

Jarosław shook his head. “This one’s on me. I had no idea what torture they put you through until you showed me.”

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!