- Feature Articles
- CodeSOD
- Error'd
-
Forums
-
Other Articles
- Random Article
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
those highlanders
Admin
I took 2 wheels off my car to reduce tire noise, now the car doesn't work. Should I put the wheels back on?
Admin
"Did you remember to plug it in?" on the industrial level.
Admin
TRWTF is a loud power supply.
Admin
TRWTF is waiting several weeks before looking at what the customer actually does. (But maybe this is the hindsight of a more matured industry speaking)
Admin
I wouldn't call it a major WTF, and of course power supplies get hot, too, and in normal "industrial(1) power supply" environments, the noise doesn't matter. (In normal use, it's locked away in some kind of server room, where the PSU noise hides itself behind the HVAC's gentle howl.)
In general, there's a fairly consistent relationship between the amount of air a fan must move and the amount of noise it produces, with the size of the fan having a "less is more" interaction - smaller fans => faster air-speed => more noise - at the same volume of air moved.
(1) In the sense of not-for-consumers rather than for-factories, although the comment is applicable in factories as well.
Admin
So, even though having multiple power supplies was a system requirement, and Chris already knew that having a single power supply could result in this particular problem, neither Chris nor anyone else in this multi-week, all-hands-on-deck exercise had the sense to ask "Hey, do you guys have both power supplies turned on?" or even "Hmm, what happens if I switch off a PSU in one of our lab units?"
Admin
"...even though they only required two to run. "
Sometimes, even a customer needs to RTFM.
Admin
I still remember the Apollo DN10000 (150MHz PA-RISC) machine that played a similar stupid trick with redundant power supplies. I was benchmarking it against the 40MHz 68040s we had in other Apollo computers at the Research Labs I was working in at the time. Running some simulation of a GSM radio channel I think.
One of the power supplies failed but carried on drawing a lot of current. I had to shut down the still running machine as the plug top with 13 amp fuse in it was smoking as the unit drew near 3 kilowatts of power...
It was only because it was plugged in to an outlet that was at eye level that I saw it.
And the extra megahertz really did not make a great deal of difference. We returned the DN10K unit with thanks but no thanks.
Admin
Sales guy took the initiative to bring the engineer to the site and the problem was solved quickly. This is why technical people need managers. Yeah, the customer didn't set it up correctly but that's going to happen...frequently. Sounds like Ben is the real hero here.
Admin
It's called In-rush Current. Every piece of electronic equipment has in-rush current when it's first powered on.
Admin
Sometimes you have to see the problem in person to have that "Aha!" moment. But, you know, internet-arm-chair-quarterback the story all you want there buddy.
Admin
Although it doesn't take much effort to make a basic troubleshooting manual. In this case, "are the three power supplies connected" would be the equivalent of helpdesk asking users "did you try to turn it off and on again".
Admin
Just ran into this problem. The very special IC's datasheet, private and confidential and do not reveal, Said in no uncertain terms that you had to power up the IC with a time from 10% to 80% voltage ramp to be under 400 microseconds. Chip would sometimes not power up properly. I finally put a scope on the Vcc line and the power supply, even though it had a fancy power-up IC to ensure correct powering up, the time was like 4 times over spec. Had to redesign the board Vcc feeds so we could, through software, control Vcc and also RESET the chip if it misbehaved. Suspenders and a belt.
Admin
The real WTF is that he didn't get called back to the office, since they couldn't bill the client for the rest of the week. Of course, the change fee for the airline ticket probably was larger than the hotel bill for the rest of the week.
Admin
Isn't holding down RESET until all power rails are good typical tho?
Admin
This is why remote/lights-out enclosure management is nice. I have a blade bucket that runs happily on 4 of 6 PSUs but starts to complain and throttle back on 3. We only have enough power for 4, and someone keeps perpetually stealing the feed to one of them. So whenever I get a warning about insufficient power, I know to go yell at the other admins.
Admin
There is no end to the "obvious" things nobody thinks to ask. Back when the Commodore 64 was a thing, I got a call from a friend who couldn't get some software working on his C64. I walked him through all kinds of steps to get his newly purchased software to load. Finally, I gave up and suggested that maybe his disk was bad and he should take it back to the store and get a replacement. It turned out he did the unthinkable and put the floppy in the drive, but didn't close the dead-obvious latch so it could be used. If I had been in person and not on the phone for an hour, the problem would have vanished in an instant.
Admin
Or to check themselves. Er, when they are there in person. Case in point...
Sherman! Set the WABAC machine to 1987!
So here we are in 1987, in the middle of summer. STC is working a summer job in a subsequently defunct company that made multiplatform(1) project management software(2). The company has a fairly close working relationship with IBM, and STC's job is to produce a bit of software that runs on, among other things, IBM's new Personal System/2 PCs.(3)
Chief among the distinguishing features of these machines are three things: MicroChannel Architecture (of which I will not speak further), a new thing called Video Graphics Array(4), and 3.5" floppy drives. Despite the technology being six years old, these drives are new to the general PC market, a point that becomes important in a moment.
For reasons that are lost in the mists of time, one of the saleswomen brought me a floppy to get a copy of my program. So far, so good, except that I could not get the disk into the drive. It went in that far, then stopped.
It took me several minutes to realise that she had stuck the large-format label entirely on the front of the disk instead of wrapped around onto the back, and therefore the metal shutter was partially covered by the label and would not open.
(1) IBM mainframes, VAXes, large DG boxes, Wangs, that kind of multiplatform.
(2) A colleague had a copy of a new thing from Microsoft, called "Project". Compared to modern versions of Project, and above all compared to the company's product, it was a toy.
(3) The original PS/2s. We had a Model 50 and a Model 60, and yes, the Model 50 had the jolly BIOS bug that meant that about once a month, I'd come in on Friday morning and discover that the CMOS clock had only advanced far enough for the machine to think it was about 3am on Saturday.
(4) No, the A in VGA does not stand for "Adaptor".
Admin
@STC: Ref "... MicroChannel Architecture (of which I will not speak further) ..."
That's a thing of which no one should ever speak further. As with Dread Chutthlu (spelling munged for survival), it's a thing whose name shall not be uttered. [Shudder].
Admin
Well Raj, let's put this into perspective - it's a CompactPCI rack. The power supplies are generally installed in slots with screws in the faceplate to keep them in. It's not like we're talking about separate supplies and power cords. (Image for example: https://www.dpie.com/systems/men-mikro-mh70i )
Remember back in the day of TVs with loads of tubes inside? (Maybe you don't). A technician's first words on the phone weren't likely to be "You didn't happen to take the cover off the back and remove any of the tubes, did you?" It's quite the same thing in this instance.
I recall a "field trip" I took about 12 years ago to the lovely town of Columbia, Maryland to figure out why some equipment didn't work at a remote site even though it worked in our local lab (and we had sent them 2 replacement sets already that had "failed" the same way). When I got there I asked where the other end of the Cat-5 cable was plugged into. Not trusting the answer I started pulling on the cable and eventually found the other end - unplugged. Explained why the Ethernet port on 3 different pieces of equipment didn't work. At least there was a 10pm flight back home so I only wasted a single day.
There are a myriad of ways that users can bork up equipment that you would never in a million years think to ask them about.
Admin
This seems oddly appropriate:
https://www.youtube.com/watch?v=mxF9yYaG1jE
A 1985 Dutch commercial for the IBM PC, the punchline of which is “Of je stopt de stekker erin” (“Or you plug it in”) when the nerdy kid who claims to know all about computers (“Just like at school” — must have been a wealthy school) can’t get his father’s new PC to work.
It’s a line that people still say sometimes, without realising where it’s from. TBH, I knew it was from a commercial, but didn’t recall it was from an IBM one until I looked it up just now.
Admin
Sweet memories and -I hope- somewhat related:
anyone remembers the SCSI-HDs of old (the ones where you had to wire cables from the enclosure to the pins on the HD so you could set the Device(Bus?)-ID without having to open the enclosure)? yeah, them...
Once upon a time the young IKnowItsLame built a -then- quite fat server with 6 or 7 of these HDs connected to a (*) RAID-Controller with R5 + "ColdStandby", installed NT4, ran tests (hot-pull a disk, replace it with the Standby -> auto-rebuild works?) and life was good.
So off to the customer the server goes: let's plug in all cables (RG58 for network, cards from 3Com or something) , flip the switch and ... the lights go out.
Cause:
Lessons learned: there was a special pin/jumper one had to set to toggle the HD from "spinUp on power" to "wait for <signal>"
Outcome: overall: my 'flip the switch' caused a power-outage in about half of the (nearly new) building and <some other people> received quite a stern talking to. personal: RTFM, always
(*) I'm quite sure that was a Mylex but my google-fu is weak today. Anyway think of a "full height" (2x CD-Drive | non-slim LTO) thingy with a display and some buttons where you could configure the RAID, for the life of me I cannot remember how it was connected to the EISA|ISA|VLB|PCI|(?)-Slot on the motherboard. OTOH I still remember the SCSI-Cards with "have a SoundBlaster too" (or visa versa): https://www.ebay.com/p/Creative-CT1770-SB-16-SCSI-ISA-Sound-Card-Sb16-Scsi-2-With-DSP/1408114027
...wow, I'm getting old, somebody could bother you with "in my time we used memmaker.exe once and from then on we knew our config.sys by heart"-stories.
/get off my lawn ;-)
Admin
12 years ago, and you didn't stop by Sykesville to harass me? I'm hurt, truly.
Admin
Ref Raj's dismissive complaint. ...
There's a whole 'nuther layer of WTF wrapped up in "questions you wouldn't think to ask". Namely:
If you ask the customer rep a question they often won't tell you the difference between something they assume is true, something someone told them is true, something that circumstantial evidence suggests is true, or something they verified for themselves as actually being true.
As professional debuggers we all understand the difference between those 4 things. Ordinary people (even non-dev techie people) often don't even understand that those 4 things ARE different, much less track which category each fact in their head belongs to. And of course "verified it myself" comes in many degrees of diligence, thoroughness, and logical validity.
As Brian Boorman just demonstrated, even "It's plugged in" has lots of room to be both true and simultaneously misleading.
I've got stories and stories just like his.
Admin
Scoping power rails can give so many clues (or quick answers, in your case). I'm embarrassed by how often I took way too long to scope a line.
Admin
It is the "Wayback Machine" not WABAC!!
Yes, Mr. Peabody!
Admin
You're wrong. It's pronounced "wayback", but it is indeed spelled "WABAC". It's a play on how computers were named back then (UNIVAC, ENIAC, etc.)
Admin
At least they didn't have to fly in to toggle the WiFi switch..... :P
Admin
Loud PSUs. A decade ago I bought a used IBM 2U server (with some xeons inside, so just like an ordinary PC, only heavier) for some home/university projects. It came with two PSUs wired for redundancy (as with PCs each hat its own socket). I didn't have a slightest hint of redundant power rails at home, so I thought I'd just plug one of them in and I'm done. Of course I'd expected the thing to be loud, but not his Rolls-Royce-Trent-1000 loud, and not getting any quieter miuntes after startup. Only hous later, when poking around the HW management console, I'd find out that the bloody thing would not wind the fans down unless both PSUs were connected. So in IBM's world more was less, at least then.
Admin
I remember (back around 2000 or so) seeing the triple redundant power supplies to Dell swervers (in a "lab" room where redundancy wasn't quite so important) hooked up with a 3-headed "hydra" power cord to a single wall plug.
Admin
I thought for sure that customer "solution" would be to always plug in a second power supply before inserting a board, then remove that second power supply after a few minutes.
Admin
TRWTF is the rack wouldn't beep and flash at you when it was missing two of three power supplies. Though I realize this was in the dark ages.
We had a device for plasma generation. It had three redundant systems to make sure sure the capacitors were truly discharged when powered down. I was not involved with the design or use of this at all, but I was there when a tech demonstrated dropping a metal rod between the terminals and it vaporized.
All three of the capacitor discharging systems had failed on it, but there were no indicators. So likely they had died at separate times and nobody noticed because at least one was still working... until the whole thing failed and they sent it back with the capacitors still lethally charged. Luckily nobody ever died, and the design was hastily modified.
Admin
Angela - I mostly travel to Aberdeen these days. Mostly.
Admin
There's nothing like an on-site inspection for certain problems, even if they end up being simple. I was told of a customer who had been sent several sets of floppies (tells you how old this story is), and they all quickly stopped working. The company sent out a rep to find out what was going on, and it turned out that the customer was taking the disks out of the drive and posting them on the side of a nearby filing cabinet with magnets. That sort of thing is hard to diagnose over the phone.