When asked to choose among several possible tools to do a job, qualified technical people look at the manual and test to see if the tool actually does what they need it to do. Is it reasonably configurable? Must it have root privilege to launch, or can it be installed as your application login id? Smarter folks will do a load test to see if it will scale beyond a handful of records and work with the expected volumes of data. And all of this will be combined to form an informed opinion as to whether the tool is appropriate for the task at hand.
High Level Managers have a different approach. They are too busy to deal with mere technical details.
After numerous outages at a large multi-national bank, a high level manager decided that they needed to do something to stabilize things, so he put together numerous charts to compare the various software packages that were available to automate solving their problems. There were slide shows, spreadsheets and myriad documents detailing how one tool was better than the others and that it would solve all of their problems.
The only problem with his analysis was that it was not based upon actual features or testing, but on the sales brochures and promises made by the salesman.
Not to let the facts get in the way of managing a problem, several suitcases of money were provisioned and turned over to the salesman in exchange for a full all-bells-and-whistles site license for the new tool. The new tool was brought in house and ran through a few simple test cases. Then it went live in production. Then it hit the fan.
Bob was brought in to see why their applications were crashing in spite of their shiny new be-all end-all tool.
Queries that should have completed in milliseconds took several minutes to complete. The tool was sucking up 80GB of memory just to launch in basic mode. And we're not even going to go into how the tool mistook email addresses for websites it had to crawl.
The manager, realizing that the salesman had lied to him, had to deal with the spilled milk, and opted to forge ahead at all costs.
Bob created a web app that alleviated the worst problems by pre-massaging input and query results. He could not push away a gnawing suspicion that he was merely repairing damage rather than adding actual value to the company.
After about a year of this, the manager committed to drastic changes in the work processes. When Bob learned about this, he asked them if they'd even done rough, back-of-napkin estimations of the expected manual workload in the changed process; after all, they already had a wealth of data from the past year and estimations surely could be done given the new process was specified in substantial detail. After all, they had gotten burned on their 'analysis' of the product they bought to solve all the instability. He was met with blank stares.
The new process was put in place and the amount of manual work tripled overnight.
Bob put in a lot of overtime trying to fight all manner of fires. Still, he was only partially successful, as the task of developing an app to totally fix the situation for a huge and complex package on top of a pretty complex work process was out of the question for a single developer.
After many, many months of this ongoing failure, the manager who started all of this had analysed the cause of the all of problems. The entire team was called in by the manager to a meeting. As could be expected, it was announced that the productivity was deemed too low while the risk and cost were too high, and so the entire team; analysts, lower level managers and Bob were laid off.
The manager was promoted for recognizing the cause of the failures and was given more responsibility to oversee other projects in addition to his own.