Comment On The Bug is in the Details

First!

That's the problem with testing.  Remember, if it compiles, SHIP IT!

Second!

 

More seriously, it makes you wonder if the developer wrote once and tested naught!

Oh my....

Do these folks not understand that error messages are supposed to convey something about what has gone wrong, comprehensible be even the dimmest user, as opposed to something only the programmer might understand? (these testers seem to have a pretty good grasp of things, compared with those I work with).

 

BTW: I assume by "03/30/2005", you meant "02/30/2005" (given the subsequent comment by the testers)

 

Anonymous:

Second!

 

More seriously, it makes you wonder if the developer wrote once and tested naught!

 

did you guys even bother to read the entire thing..? It's quite obvious here that the WTF is with the testers, not the coders.  The coders seemed to have done an adequet job of error proofing the application, but the testers seem to expect the thing to me idiot proofed X10

I'm confused. How are terse error messages for invalid date and email addres entries a showstopper? Is that the WTF?--the quoted memo, not the software?

So the WTF is that the customer wanted better error handling?

> 03/30/2005

I assume this was 02/30/2005 as used a bit lower down. The error message *is* deceptive, it indicates that the format is wrong but the actual problem is that the date is not valid.

On the email validation, anybody here who deals with people typing in emails knows that they get it wrong a lot of the time. I frequently see www.myname@aol.com or somebody@hotmal.com and plenty of others. The customer had it wrong to some degree--any domain such as .info can have emails as well--but still, it would be good to be more specific about what was wrong. A few regular expressions can handle this easily.

The WTF is the memo.

It seems like the customer is just trying to get out of the contract.  The only way this criticism would be remotely acceptable is if there were specific requirements for descriptive input validation. 

I think it is the memo/testers: 

We then typed in "MM/DD/YYYY" in the same field and it provided the same exact message

Make an idiot-proof application and the world will build a bigger idiot?   While I agree in principle with most of the testers' complaints, it sounds like they've never used any application on the internet before.

Are they serious?...



"We entered in MM/DD/YYYY and got the _exact_ same error message"



I guess it's hard to find testers that are capable of coherent thought. Oi vei.

Brendan Kidwell:

I'm confused. How are terse error messages for invalid date and email addres entries a showstopper? Is that the WTF?--the quoted memo, not the software?

That's definitely how I read it. The program DOESN'T necessarily know what is wrong with the email address, for example - it might just be doing a regex to validate it. If the code did more sophisticated checking, we'd point and laugh because the coder added ridiculous complexity where a one line regex would have sufficed.  I don't see how the terseness of error messages could possibly be a showstopper; not to mention that it would be pretty obvious to the user that ".....@.@.@.@@@@@" is an invalid address. Geez.

You joke, but people really do complain about these things. Especially when they *insist* that they entered something correctly.

I don't agree with testers' comments.  If someone wants a program that holds their hand and thinks for them and wants that level of description for errors, they get charged a lot more.

Ah testers...

Seriously, if the user types in "asdfasdfasdfa" in the email box, and gets an error saying the email address is invalid, good.  If the user doesn't know how to fix the problem, then the user needs to put all their stuff in a box, and take an early lunch... or how bout... just don't come back.

Helping users is one thing, but helping them to this extent, is wrong.  Users need to take some responsibility.

End user: When I pull this blue cable out of my computer, my Internet goes down!
Tech support: {OMFG...} Yeah, how about we leave that plugged in all the time.

Obviously, this testing team would like an alert to pop up... informing the user...

If their users are so stupid that they need that level of hand-holding, I don't see how they're even capable of using a toilet correctly.

Its called a datepicker people....use it.

So, the testers thought that they could not go past the first screen by entering in CORRECT information?



I mean, they knew what the error messages meant (given their suggestion
of what to do to improve the system), and they knew that the data they
were entering was wrong (presumedly to test error handling), why don't
they test the system with correct info to see if it FUNCTIONS!?



Sometimes the error is "user error"....

Anonymous:

Make an idiot-proof application and the world will build a bigger idiot?   While I agree in principle with most of the testers' complaints, it sounds like they've never used any application on the internet before.

Idiot proofing is one thing, but these people were trying their very best to be idiots.
Its kind of like that friend everyone has that tries to be funny by bein super-literal, and eventually gets punched in the face.

Its called a datepicker people....use it.

That solves the dates, but not email addresses.

Ok, so the real WTF® here is that the testers were too anal to get past the first screen, right? Well at least by the end of it you'll have the most helpful error messages known to human kind!




I'm curious as to what was snipped. Was the original very, very long?

Good luck coding an address validator!

I mean, while you can validate a the correctness of an email address with a quite simple regex (http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html), writing an error handler that explains exactly what's wrong with the address would be surprisingly tricky.

Bonus points if the error message ends "... (RFC 2822, section 3.4.1, sentence 3 - We know you don't care, we're just trying to annoy you)"

My opinion is that the tester is good but is also a jerk.

I think his comments are good. Help the user a maximum to find his errors is really great.
But his behavior "we've been unable to continue testing, it's too buggy etc.." is so lame. Why he can't continue ? Oh I know ... laziness !!!!

I'm not sure if I get this one... At first it says the user is told that "03/30/2005" is not in "MM/DD/YYYY" format when it clearly is, which would be a WTF on the software... but everything after that makes it sound like the wtf is the tester and not the software.  Am I missing something here?

Anonymous:

If their users are so stupid that they need that level of hand-holding, I don't see how they're even capable of using a toilet correctly.

Most likely their users are OK entering valid dates, it's the testers who come up with new requirements most likely not in the original requirements. There's no general way of knowing what is wrong with for example 02/29/2006 (Feb 29 2006), is it the day, month or year? Same problem with the email address, is it the domain or the user name that is incorrect? So if they want specific error message on certain inputs it should have been added to the requirement specification.

kipthegreat:

I'm not sure if I get this one... At first it says the user is told that "03/30/2005" is not in "MM/DD/YYYY" format when it clearly is, which would be a WTF on the software... but everything after that makes it sound like the wtf is the tester and not the software.  Am I missing something here?

It's amazing how fast the first 20 posts make it up here when you step away from your desk partway through typing a response. :)

At my last job we had the customer do alot of the testing as our company was very small.  I heard my boss repeat this comment many times - "Yes, I know you will find bugs.  We told you this.  It is imperitive that you do not stop testing after you encounter a bug or two...."   What I have learned from this is that there are alot of people that shouldn't be allowed outdoors without supervision...

Anonymous:

Its called a datepicker people....use it.

That solves the dates, but not email addresses.

damn your pretty smart. who would have thought you couldnt enter emails in a datepicker.

I mean its obvious that the users are not that swift, so if you can add a control or a dialog to select values it makes it easier for them. But this really comes down to who speced out the project and what was/not promised. Sounds like this was someones first try at making software for clients.

Guess they didnt do any prototyping.

Alex Papadimoulis:

Notify Email
This field is the most troubling of all. No matter what we put in the field (aside from an email address), the only message we receive is "The email entered is not valid." This includes everything from "AAAAAAAAAAAAA" to ".....@.@.@.@@@@@" -- all the same message.

Well duh.  [*-)]

Dear sir,



I tested the latest version of the software and there are STILL too
many bugs to continue.  For one thing, I intentionally misspelled
my last name in the Name field and it was not caught!  Then I
tried putting random ascii characters into the address field and it
simply said the address was invalid!  To top it off, I put in a
zip code of 37641 and it took it!  Like I would live in Chuckey, TN!  Me of all people!  That is simply insulting!



Oh and the logo clashes with my shoes.  Its got to go.



Signed,



Paula the Brillant Tester

They should re-code it so all errors are reported as: "A critical failure has occured between the keyboard and the chair."  I'll bet the testers replace their desks before they get it.

Anonymous:

Anonymous:

Its called a datepicker people....use it.

That solves the dates, but not email addresses.

damn your pretty smart. who would have thought you couldnt enter emails in a datepicker.

Simply pointing out that a broader issue is involved here.[8-)]

I assume "03/30/2005" should be "02/30/2005"
..but anyway.

If the specs for the app were to be for a 6-yr old then the fault lay on the developer(s)
But assuming that the users should have some incling how to use a computer then the WTF is the users.

I'm sick and tired of dealing with computer problems for people that turn out to be a user not knowing what the hell they are doing.

Reminds me of a similar situation with a previous employer.

They delivered a small inventory tracking system to a local antiques dealer. A few weeks later the owner called up and was upset with it. They visited him to find out more about the problem. The conversation went something like this:

Dealer, refering to an item displayed on screen: "See, it says the table is here in St. Louis."

Developer: "And were is the table?"

Dealer: "Its in California. I shipped it there a couple weeks ago!"

Developer: "Did you enter that you had shipped it?"

Dealer: "Well no, I shouldn't have to do that. It should KNOW where the things are!"

Like the above customer, he expected omniscient software that knew all for him.

 The employer could only dream of the fortune he would make if he could make that software program!

I suppose whether or not the tester's saying that the error handling is grossly misleading depends on what the requirements are.

Given: "we can not proceed beyond this first screen because the error messages are insufficient" is BS.

I've worked on various systems where the requirements ranged the gamut from:

let the application crash and burn for every little problem

to:

print a unique and descriptive error context-specific message for every

conceivable condition so the brain-dead users won't need to think

WWWWolf:

Good luck coding an address validator! I mean, while you can validate a the correctness of an email address with a quite simple regex (http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html), writing an error handler that explains exactly what's wrong with the address would be surprisingly tricky. Bonus points if the error message ends "... (RFC 2822, section 3.4.1, sentence 3 - We know you don't care, we're just trying to annoy you)"

Actually, you can't really validate a string as an e-mail address using just a regex.  I know everybody believes that, but you can't.  If you believe that, you haven't really read the RFC completely.

Sounds like the tester hasn't finished writing their test cases, so has
come up with a spurious reason to give themselves more time.



(Yes, seen testers do that....)

The customer is right that it would be better software if it told the user what was wrong. Sure, you could have an error message that says nothing more than “Huh?” and most of the time, the user will re-check their input an immediately recognize their mistake, but I’m sure we’ve all had moments where we stare and stare at some problematic input and only then realize (d’oh!) we type a “,” rather than a “.”, or the like. Sure would nice if the computer highlighted the exact character(s) causing the problem. Would speed up typing the correction if nothing else.

Now that doesn’t make the customer’s complaints genuine show-stoppers, but it seems the customers have an incorrect sense of how programs do input validation, and how much work would be involved. They thought the programmers were being lazy, so they dinged them for it by halting testing.

Lesson: customers are not a programmers. No WTF.

--RA

I wonder if the issues in

---- snip -----

were just as serious...

Anonymous:

Its called a datepicker people....use it.

That solves the dates, but not email addresses.

So have an e-mail picker :)

Anonymous:

Like the above customer, he expected omniscient software that knew all for him.

 The employer could only dream of the fortune he would make if he could make that software program!

Smells like GPS+RFID+stuff

Oh yes, of course, how stupidly retarded of me. Let me rework all of my code and center it around completely 100% retarded monkeys. Let me make sure that my verification code is five billion lines wrong and littered with stuff that will check stuff like:

(assuming we are using internet only e-mail)

Given:
someone@sumwheire.con
Response:
con is not a valid top-level domain name.

Given:
someone@sumwheire.com
Response:
(after 10 seconds)
sumwheire.com is not a valid domain name

Given:
someone@someotherplace.com
Reponse:
(after 15 seconds)
someotherplace.com is a valid domain name, yet it cannot be reached

Given:
someone@somewhere.com
Response:
(after 10 seconds)
somewhere.com's mail server says that the user someone does not exist

Given:
admin@somewhere.com
Response:
(after 30 seconds)
somewhere.com's mail server says you do not have rights to e-mail that account

MyResponse:
WHAT?!??!?!!? how did it know that!?!??!!??!
 lol

Anonymous:

Dear sir,

I tested the latest version of the software and there are STILL too many bugs to continue.  For one thing, I intentionally misspelled my last name in the Name field and it was not caught!  Then I tried putting random ascii characters into the address field and it simply said the address was invalid!  To top it off, I put in a zip code of 37641 and it took it!  Like I would live in Chuckey, TN!  Me of all people!  That is simply insulting!

Oh and the logo clashes with my shoes.  Its got to go.

Signed,

Paula the Brillant Tester

Thanks for the laugh.  It has been a very tough week.

GoatCheez:

Let me make sure that my verification code is five billion lines wrong and littered with stuff that will check stuff like:

(assuming we are using internet only e-mail)

(snip)

Given:
someone@somewhere.com
Response:
(after 10 seconds)
somewhere.com's mail server says that the user someone does not exist

Given:
admin@somewhere.com
Response:
(after 30 seconds)
somewhere.com's mail server says you do not have rights to e-mail that account

If a valid email adress is _really_ important for the system, you might decide to do exactly that.

ammoQ:

Anonymous:

Like the above customer, he expected omniscient software that knew all for him.

 The employer could only dream of the fortune he would make if he could make that software program!

Smells like GPS+RFID+stuff

Wow, how technology has changed! That is a good solution for mega-corp today!

But that story was for local small-time dealer in about 1992 spending a few hundred dollars, probably using something like DBase or Paradox.

Maybe he was just a few years ahead of the times for his expecttions!

Sorry Sridar, you got what you asked for.



Clearly the testing staff was not presented with a clear way of
describing the severity of a bug. Guidlines should stem from the
obvious categories "Crash the system?", "Corrupts data in the system?",
"Leads to invalid data entry?", "Did not act as expected?"  As we
all see, the testers only found "did not act as expected" issues. Seems
like THEY'RE not doing their job. And now, they refuse to do their job
("we can not test any further")



Did Sridar have specific requirements to accomodate the "did not act as
expected" issues? Probably not - but, if the project manager/team had
developed a good set of requirements then the testing people would have
something important to do, and our buddy Sridar would be able to point
to the Requirements doc and dispute wether or not he had complied.



Lesson learned is a repeat of the typical "failure to plan" issue. The project team/mgr dropped the ball on this one.

If the application had been coded in Lisp over 40 years ago (tm), each time an error gets raised it would initiate a live voice chat session (preferably with video) so that the programmer could explain face to face what the problem is, as well as offer some helpful suggestions for gettng the data input correctly.

The WTF here is definitely the testers' giving up.    When you find lots of bugs on the first screen, your reaction should be "yippee!"  and move on to the second screen to see what treasures are waiting for you there.    If buggy software doesn't fill you with a warm wonderful feeling and a thirst for more, you shouldn't be a tester.