- Feature Articles
- CodeSOD
- Error'd
-
Forums
-
Other Articles
- Random Article
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Why are businesses so manically obsessed with Excel?
I mean, sure it has its advantages for simple off-the-cuff tables and charts. But anything else and you just know that it'll turn into a unmaintainable quagmire of formulae that nobody understands any more and with the ever-increasing risk of a simple error costing millions of pounds (as has actually happened).
And yet...managers hoard their precious spreadsheets like they're the only thing preventing the end of the world
Admin
Because the Simple is the Enemy of the Good. Getting a statistically certified Steve in, setting up a SAS server, developing the SAS algorithm and certifying it is much more complicated than just opening The Spreadsheet and start plunking away.
Admin
Because it's easy to begin (with that off-the-cuff thing), and then, like Topsy, it just kinda grew. None of them set out to build a Franken-thing in there, but it happens in the end.
Admin
If all you have is a Spreadsheet, then everything looks like you nailed your foot with a nailgun.
Admin
Seen to many business relying on cobbled-together excel sheets by some non-programmer who left the company long ago. Even national banks on government bonds ...
Admin
99.99% of the time, Excel is fine. We hear about the cases where it isn't.
As for why Excel at all, what alternative commonly exists on office pcs?
Admin
...and then the sunk-cost fallacy stops them from ever switching to something better.
Admin
My SO would never claim to be able to program, but they happily hack together a spreadsheet and some macros any day of the week. That is not a conflict, to them, programming also entails a modicum of software engineering, which Excel does not require.
Admin
Well, I think their point is in that the cases where it doesn't work, it really doesn't work - and you end up with an unmaintainable, unverifiable mess at the core of a critical business process! Excel is adequate for simple or one-off reports, but it's not appropriate for complex statistical analysis; that's how you end up with a TDWTF post, and frankly, trying to use a tool for a job it's ill-suited for is a WTF in itself.
Admin
An experianced programmer can make great programs in Excel. Even big and complex ones. I have done a few that still are used after 15 years and need little to no maintenace and works for the area they was intended for.
Like every system there are areas where they are intended for and will work well. Going outside of that makes it a mess. Unfortunate business people with no IT (programming) background tend to add to this mess and also try solve other business probelms with Excel.
When you have a hammer ....
Admin
There are two areas where Excel is genuinely better than a full-fat programming language: (1) rapid prototyping and (2) providing visibility of intermediate steps. This is why, when I want to understand a complex calculation methodology, I often sketch it out in Excel before implementing it in Python or R. I've actually just finished doing that with a nifty numerical simulation method ("Copula based hierarchical risk aggregation through sample reordering", Arbenz, Hummel & Mainik, 2012).
Those are the good reasons why people use Excel. The bad reasons are (1) lock-in, and (2) the illusion that you can use Excel well without understanding software development. Lock-in arises from several sources, including: limited scope for technical training; inability to recruit or retain full-fledged programmers; lack of budget for process improvement; and plain old institutional inertia. Sometimes this can be overcome - on my team I've started using R notebooks for statistics-heavy projects, and am gradually nudging the team to adopt more and more best practices as momentum grows. I have a dream that one day I shall get them using Git, or at least Subversion.
Reason #2, though, is ultimately an expression of Dunning-Kruger on the part of non-technical managers: all those best practices that programmers go on about can't be that important, right? Surely they're just trying to feather their own nests? I could do it myself for a tenth the price, couldn't I?
The result: there are entire teams of software developers that don't realise they're software developers; they think they're administrators or managers. Consequently, and unsurprisingly, they write utterly crap software.
Admin
I was expecting a story about a spreadsheet that literally could not be moved lest it stop working. The one in the story was not quite that horrible.
Admin
Where Excel really backfires as when the investors or customers demand some sort of assurance that they aren't buying into a soon-to-explode disaster. It's really hard to do a bunch of things with Excel, including:
So, every time a spreadsheet gets important and the company gets big, regulations like Sarbanes Oxley swoop in and make the formerly "easy" spreadsheet very difficult because it is very expensive to build a change management process around an Excel spreadsheet.
Admin
The suck cost fallacy is about attachment to your original invest and psychological path commitment. Its not falicous at all to consider the costs of the alternatives.
"We have already got $25K of project budget invested in these excel sheets, so lets just stick with it." -> Sunk cost fallacy
"It will be $60K op-ex and $15K cap-ex to train these people on a new system" -> perfectly correct and legitimate consideration.
Admin
@Hal ref "... -> perfectly correct and legitimate consideration."
All true. IFF you include "But the change will then save $30K in Excel maintenance work every year UFN, and preclude decision errors worth $1M/yr thereafter".
The typical problem with costing is the bizidjits add up the 2 or three obvious costs that impede progress and discount all the countervailing benefits promptly to zero. Even if they have at their fingertips the actual maintenance costs that will be foregone.
Admittedly the costs foregone will be (partly) replaced by new maintenance costs on the new system. But assuming your company is ever capable of good work on any project, there ought to be reliable evidence of highly likely savings.
Admin
Excel is one of the most abused programs ever!
I just got in my e-mail, I kid you not, a COVID 19 warning poster to put up in our labs. Not in Word or Powerpoint but in excel, All the cells extremely small to better align the images i guess.
Admin
Because any old monkey can get as far as 2+3 or a crosstab of numbers in Excel, even 'non technical' people. And then the crosstab grows to be some six sheeted behemoth because monkeys never learn proper software development skills, but they can keep bashing their paws on the keyboard to create more formulae, so they keep adding stuff without considering robustness or maintainability.
And to be fair it does have a high value for simple calculation and reporting. You just have to know when you're stepping outside what you should be using it for.
Admin
If all you have is a three-headed hammer, everything looks like three nails.
I had a boss once who used PowerShell (eek!) as his three-headed hammer.
Admin
TRWTF is "waving the printouts" instead of mailing or whatever
Admin
A company I worked for once was acquired by a gigantic company. You remember "management by walking around?" They had "management by flying around" and "management by very large spreadsheet." They had a a spreadsheet that handled salary ranges for raises. It had a bug in it, and lots of folks got less than the budget. The mgrs were puzzled when they discovered they hadn't used up the salary budget. The individual contributors were, umm, puzzled (trying to be polite) that the company was doing so well and the raises were so low.
And this wasn't from something obvious like misunderstanding of FLOATs and DOUBLES. It was some kilobyte-long cell formula.
Addendum 2020-06-29 15:01: Excel is the Sawz-all of computing. It's good for taking out wiring and plumbing by mistake.
Admin
At one point, I was working on a numerics API for Emacs Lisp 0:)
Admin
Excel ... the perfect solution for those who don't understand their problem.
Admin
You're missing that the money to rewrite it comes out of a different budget to the one that pays the maintenance. And then you've got to consider whether it's this years money or from some future period.
Admin
SAS and XL both can do great things.
It's just that SAS is meant for analyzing and manipulating datasets and XL, surprisingly, isn't...
Admin
My company won (amongst others) a SAS maintenance contract a couple of years ago. That meant me having to teach myself SAS and reverse engineer thousands of lines of SAS code written by a smart guy, but a non-programmer.
Somehow that still beats reverse engineering spreadsheets like the one described here...
Admin
I agree with D-Coder's hammer-and-nails comment. Spreadsheets are often seen as the answer before the problem is even identified.
Admin
There's a certain level of skill-at-computers that sometimes leads a user to have just one hammer that they're really good with, and therefore try to apply that hammer to all problems.
I once had a coworker for whom that hammer was a 2D illustration program. (Maybe Corel Draw, but I can't remember for sure.) Need a printed poster? Corel Draw. Need a 3D image? Bunch of construction lines and 2-point perspective like you learned to do in high-school Technical Drawing, in Corel Draw. Spreadsheet? Rectangular array of numbers in Corel Draw.
He was a smart guy and very productive, but because he was always 100% busy, he never had time to climb the learning curve of a new piece of software -- getting this job out the door was always too urgent a priority.
Admin
At some point in the early nineties, I saw a very complex spreadsheet that completely designed windows (you know, the ones with glass sheets inside), with all materials needed, all the cost, everything. It took two hours to run on an Apple /// computer. And the guys would then take the materials as given by the spreadsheet, put them on a lorry, drive to a house, install the windows, and everything would fit.
I've never in my life before or since seen.a spreadsheet running for that long.
Admin
Presumably all of his PowerShell scripts start like this?
Admin
"Excel is the Sawz-all of computing. It's good for taking out wiring and plumbing by mistake."
I am totally putting that in my quotefile. It is so true.