• Steve-Parker.org (unregistered)

    I don't know IIS at all, but even with my most anti-MS hat on, I could not believe that a 404 on favicon.ico (a "standard" created by MSIE in the first place) could cause such an error on a MS-SQL/MS-IIS server.

    I don't think the actual problem has been diagnosed at all. 30x redirects sound like red herrings.

  • (cs)
    A tense few weeks passed with out a single crash reported

    TRWTF is that they write with out with a space instead of without

  • (cs) in reply to elias
    elias:
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    Their. Fixed that for you.
  • (cs) in reply to Say No to Cowboy Programming
    Say No to Cowboy Programming:
    You are right for many cowboy programming environments but not right for shops where quality matters. I've been in both shops and the cowboy programming shops let the problem go. The Agile, Iterative or disciplined shops do not allow crap like this to linger, let alone go on with a ridiculous loser solution like auto restart IIS. That is for kindergartners.

    Kindergärtners. Fixed that for you.

  • immibis (unregistered)
    66.77.93.50 - [08:34:29] "GET /access?action= _ _ forward&uri=%2Ferror.aspx HTTP/1.1" 302 - "-" "-" "-"

    66.77.93.50 - [08:34:29] "GET /error.aspx _ _ HTTP/1.1" 302 - "-" "-" "-"

    I think error.aspx redirected to /access?action=forward&uri=%2Ferror.aspx And of course /access?action=forward%uri=ANYTHING will forward to that URI (why anyone needs that is beyond me)
  • immibis (unregistered) in reply to Cloak
    Cloak:
    elias:
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    They're. Fxd tht 4 u.
  • zzp (unregistered)

    The real WTF is that a single malicious user would still be able to crash their website by repeatedly requesting a non-existent page with cookies disabled, because Bob B. didn't bother fixing the session handling code...

    I recommend Bob B to read "Session Expiration for Dummies". Great book.

  • Unknown (unregistered) in reply to PJ

    Technically, Ohaiyo gozaimasu means "Its early", but its used in the same way english speaking people say "Good Morning".

    And knowing is half the battle.

  • Grrr (unregistered)

    The other WTF is why would server keep idle connections alive for days. I mean - if AOL browser opens a new connection when it fails, apparently the old one is a zombie, right?

  • Wanja (unregistered)

    Oh, what a sweet story. Leaves me with a teardrop in my eye and a sigh on my lips. But before I could read it to my children, would you please change the beginning to "Once upon a time there was a programmer" and the end to "..and they lived happily ever after"? It's not that I don't think the story is true, it's just that it suits bed-time-stories better.

    Who would read such a story to his children? Well.. depends on the children doesn't it?

    [image]
  • Request for Help (unregistered)

    Hello,

    i don't understand the problem. Can someone explaine the error with that favicon?

    Thank you

  • HAX0R (unregistered)

    CAN SOMEONE SEND ME TEH CODEZ FOR THIS HAX? KTHXBY

  • ha (unregistered)

    They shouldn't fix the error. It is far more easier to sue that user and put him in jail for several years for hacking their site. And do not forget about compensation like several millions dollars. This is a very profitable deal!

  • a goat (unregistered)
    James:
    DeLos:
    sweavo:
    Wait... when did Ohio get internet?

    What is an "Ohio"?

    I believe it means "Hello" in Japanese.

    Zing!

  • #FF00FFbeard (unregistered) in reply to immibis
    immibis:
    Cloak:
    elias:
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    They're. Fxd tht 4 u.
    Tharr. Fixed that for ye, matey.
  • JimM (unregistered) in reply to zzp

    [quote user="zzp"]The real WTF is that a single malicious user would still be able to crash their website by repeatedly requesting a non-existent page with cookies disabled, because Bob B. didn't bother fixing the session handling code...[/quote] Or, if you'd read the article: [quote user=TheArticle"]Bob quickly added a favicon.ico to the root folder and patched up the infinite redirection problem.[/quote] I'd guess, since Bob figured out the cause of the problem and patched the system, that he'd have made sure that error requests no longer left open sessions. Otherwise he was teh dumb. Plus, a malicious user could only perform that attack if they had inside knowledge of the workings of the error system on the website; and that's not something you generally publish in big letters on your homepage...

  • JimM (unregistered)

    Anyway, I'm still not sure I get this... the favicon.ico is missing, and instead of returning 404 "not found" the server returns 302 "it's over here really!" pointing to access?query-string. When that's requested the server again returns 302 "it's over here really!" pointing to /error.aspx, and when that's requested it gets 302 "it's over here really!" pointing to access?query-string?

    Where in all of that is the connection to SQL server initiated (the article states it was SQL Server reporting the out-of-memory problems), and what were the developers smoking the night before they wrote the app... because I wants me some of that ;^)

  • (cs) in reply to David Schwartz
    David Schwartz:
    Something doesn't seem right. If the problem with the favicon file was simply that it didn't exist, the problem would reappear as soon as someone else tried to retrieve a file that didn't exist.

    It's hard to imagine that doesn't happen all that often. Even the very smallest websites see the occasional "URL from space" in their logs.

    For any normal webpage, the infinite loop would be ended by the user closing the browser window after a relatively small number of iterations. The favicon thing happened in the background, unnoticeable to the user, whenever that AOL browser decided to renew the favicons displayed in its Bookmarks, and as long as it was running.
  • (cs) in reply to Steve-Parker.org
    Steve-Parker.org:
    I don't know IIS at all, but even with my most anti-MS hat on, I could not believe that a 404 on favicon.ico (a "standard" created by MSIE in the first place) could cause such an error on a MS-SQL/MS-IIS server.

    I don't think the actual problem has been diagnosed at all. 30x redirects sound like red herrings.

    JimM:
    Anyway, I'm still not sure I get this... the favicon.ico is missing, and instead of returning 404 "not found" the server returns 302 "it's over here really!" pointing to access?query-string. When that's requested the server again returns 302 "it's over here really!" pointing to /error.aspx, and when that's requested it gets 302 "it's over here really!" pointing to access?query-string?

    Where in all of that is the connection to SQL server initiated (the article states it was SQL Server reporting the out-of-memory problems), and what were the developers smoking the night before they wrote the app... because I wants me some of that ;^)

    Presumably it was the app (not IIS) that was written in such a crappy way that it created new user sessions even when the page request resulted in a failure.

    Summary: It's not Microsoft's fault, it's not AOL's fault, it's not the HTTP standard's fault, it's the fault of the app programmers.

  • Tei (unregistered) in reply to HAX0R
    HAX0R:
    CAN SOMEONE SEND ME TEH CODEZ FOR THIS HAX? KTHXBY

    HERE IS THE VBAZORZ CODEZ

    $victim = "http://www.somesite.com/404.html";

while(1){
 `wget $victim -o /dev/null`;
 `wget $victim -o /dev/null`;
 `wget $victim -o /dev/null`;
 `wget $victim -o /dev/null`;
  system("wget $victim -o /dev/null");
}

    WATHZ U HOMAIL?

  • Tei (unregistered) in reply to Wanja
    Wanja:

    As a child, I used to read police manuals. I know a bit about criminology, laws, frauds, etc.. Is more fun than watch "Starky & Hatch".

  • (cs) in reply to Steve
    Steve:
    The goggles! They do nothing!

    Seriously, these colors are awful. How long are you going to subject us to these?

    Until all the morons that post bitching about the site stop.

  • (cs) in reply to JimM
    JimM:
    Where in all of that is the connection to SQL server initiated (the article states it was SQL Server reporting the out-of-memory problems), and what were the developers smoking the night before they wrote the app... because I wants me some of that ;^)

    SQL Server not having enough memory doesn't mean that someone had to connect to it. Use your head.

    If SQL Server is running on a server with 1 GB of RAM, and you launch 10 instances of FireFox on that server, and open 1000 tabs in each instance of FireFox, I guarantee you SQL Server will complain about not having enough memory if anyone tries to do anything at all related to SQL Server.

    This obviously means that the connection from Ohio didn't have to try and open a connection to SQL Server; the fact that anyone tried to connect to SQL Server would cause the memory issues with SQL Server.

  • (cs) in reply to #FF00FFbeard
    #FF00FFbeard:
    immibis:
    Cloak:
    elias:
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    They're. Fxd tht 4 u.
    Tharr. Fixed that for ye, matey.
    Hello there, kind sir. Could someone mail the teh fixs?
  • (cs) in reply to Martin Dreier

    [quote user="Martin Dreier"] The real WTF is that some people assume that everyone in the world knows everything about the USA[/quote]

    The real WTF is that (according to rumor) most people in the world know more about the USA (geographically) than most US citizens. And let's not even talk about the rest of the world.[/quote]

    The rest of the world? You mean like California?

    Addendum (2008-02-08 10:16): gotta learn to preview, and check that my quote tags match up...

  • Anonymous (unregistered) in reply to Botzinger Gulm
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    WTF is Ohio.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
  • (cs) in reply to Jetts
    Jetts:
    #FF00FFbeard:
    immibis:
    Cloak:
    elias:
    Botzinger Gulm:
    CynicalTyler:
    Jason:
    Mattkins:
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    There. Fixed that for you.
    Here you go, good as new.
  • zzp (unregistered) in reply to JimM
    I'd guess, since Bob figured out the cause of the problem and patched the system, that he'd have made sure that error requests no longer left open sessions.
    The article is not that explicit. To me it looks like Bob just fixed the infinite redirection, but not the inherent session leak when processing a request for a non-existent file without a cookie...

    I work in the security industry, and trust me, this kind of pb would be detected by any decent pentester. No need for inside knowledge...

  • College Ruled, White Paper (unregistered)

    I agree. Shenanigans! Management mantra is "Reboot, not Research".

  • jack (unregistered) in reply to Outlaw Programmer

    yeah, this doesn't quite pass the smell test. You mean to tell me some AOLuser had enough bandwidth to process a million redirects a month (roughly 1400/hour or one every 2 seconds if running 24x7), taking down the web server as a result? I assume they're broadband AOL since they had a fixed IP address, but still, that's a mess. And IIRC with IIS don't sessions expire after a bit and are then reclaimed? Was this disabled?

    What's really puzzling is why there was only one client doing this. if indeed it was a bug in AOLs browser, certainly they would have seen this more than once? AOL is surprisingly popular, I'm sorry to say.

    Thinking about it from the Ohio family's POV: If it was taking the server down, I wonder how many times that poor sod had to reboot? And once the favicon was established, I'll bet their response times took a noticable jump!

    re: the workaround -- Outlaw is right. Too many shops would have applied the "duct tape" solution: simply block the IP address at the firewall and move on.

  • jack (unregistered) in reply to JimM

    Plus, a malicious user could only perform that attack if they had inside knowledge of the workings of the error system on the website; and that's not something you generally publish in big letters on your homepage...

    Of course not. That's what this site is for. (And Slashdot, on a slow day)

  • Carlos (unregistered)

    I have a couple of sites in HostMonster and I read somewhere they forces a favicon in your site if you don't provide it, because a similar problem in their servers.

  • Carlos92 (unregistered) in reply to Outlaw Programmer

    You know...there are places where they don't even dare restarting things until they completely fail. And there are places where nobody know how to automate a task.

  • Unoti (unregistered) in reply to Outlaw Programmer
    Outlaw Programmer:
    I call shenanigans. Management would have just told the developer to schedule a task that restarts the 2 services every night. At the places where I've worked, any bug that has a workaround never gets fixed; the workaround just gets added to the user manual!

    That's utterly retarded. A bug fixed merely by workaround is almost invariably a hidden problem waiting to come back and bite you somewhere else. Whenever something bad is happening, you should fix it. If you try to tip toe around it, and hope it's going to be ok, then you're no longer in charge of your system. This is something you will learn with experience, if you're intelligent and observant.

  • Peter (unregistered)

    Lots of WTFs on this story.

    First is the AOL WTF. If you don't get the answer you want the first time. STOP. favicon.ico is not important. As stated the user may have been running old software. This is why keeping your software updated is important.

    Second WTF is redirecting a (what should have been a) 404 error to pages that require session management. 404 are there for a reason, "This shit is not here, stop looking for it." Redirecting the user, possibly back where they got the message in the first place is poor design.

    Thrid WTF is looping on cookie failure. "You don't have cookies -> Redirect to page that requires cookies -> You don't have cookies -> Redirect to page that requires cookies = FAIL. Browsers generally catch this kind of crap. But, as other posted have pointed out, there are cases that seem to slip by. Just browse the internet one day will all cookies turned off and you'll see a number of sites fail badly.

    Fourth WTF. Poor session clean up by application/server. After the client terminated its session (by lack of cookies) that session should have been closed. I'd have to guess that the app is keeping the session information in memory in case the client decides to restart the connection. In the case of a valid connection this is what should occur. If a user is shopping on your website and stops for 10 minutes, they don't want their shopping cart to be empty when they come back. Some one probably set the session timeout length to something incredibly long (day, weeks, months).

    You see this kind of crap in SMBs servers all the time. Normally they don't get enough traffic for it to be noticed. Or you'll hear the I just reboot the server once a month to keep it running fast stories.

  • (cs) in reply to Peter
    Peter:
    Lots of WTFs on this story.

    First is the AOL WTF. If you don't get the answer you want the first time. STOP. favicon.ico is not important. As stated the user may have been running old software. This is why keeping your software updated is important.

    Second WTF is redirecting a (what should have been a) 404 error to pages that require session management. 404 are there for a reason, "This shit is not here, stop looking for it." Redirecting the user, possibly back where they got the message in the first place is poor design.

    Thrid WTF is looping on cookie failure. "You don't have cookies -> Redirect to page that requires cookies -> You don't have cookies -> Redirect to page that requires cookies = FAIL. Browsers generally catch this kind of crap. But, as other posted have pointed out, there are cases that seem to slip by. Just browse the internet one day will all cookies turned off and you'll see a number of sites fail badly.

    Fourth WTF. Poor session clean up by application/server. After the client terminated its session (by lack of cookies) that session should have been closed. I'd have to guess that the app is keeping the session information in memory in case the client decides to restart the connection. In the case of a valid connection this is what should occur. If a user is shopping on your website and stops for 10 minutes, they don't want their shopping cart to be empty when they come back. Some one probably set the session timeout length to something incredibly long (day, weeks, months).

    You see this kind of crap in SMBs servers all the time. Normally they don't get enough traffic for it to be noticed. Or you'll hear the I just reboot the server once a month to keep it running fast stories.

    Why on earth would this have anything at all to do with the browser? A browser is just a bunch of pixillated crap with gooey buttons and munchy cookies and (in some cases) hidden back doors. It works on the most brain-dead protocol imaginable. Jeez, you could write the request/response in QBasic, and it would still show up on a Commodore 64 ... admittedly, as garbage. But it would still show up.

    This is an error on the server side. No proper, scalable, server software should ever permit this lunacy.

    That is why I, as a server-side programmer, am paid millions -- yes, millions -- per year.

    Admittedly, it comes in sacks full of peanuts, but since I tend to work with monkeys, I feel I'm being properly rewarded for what I do.

  • Long John Silver (unregistered) in reply to Outlaw Programmer
    Outlaw Programmer:
    Management would have just told the developer to schedule a task that restarts the 2 services every night. At the places where I've worked, any bug that has a workaround never gets fixed; the workaround just gets added to the user manual!

    They are ready for a wtf tale :-)

    LJS

  • (cs)

    Ahhh, you just gotta love AOL, dontcha?? Yeesh.

  • Anonymous (unregistered) in reply to DeLos

    It's IIS... what's else needs to be said?

  • In Agreement (unregistered) in reply to Outlaw Programmer

    The funny thing is, I work for a large bank, and I thought this comment was funny, because its true. I think its just in the IT field - things move too fast for anything to ever be more than a 'tactical' solution.

  • Project2501a (unregistered) in reply to Outlaw Programmer

    Boss? what are you doing here?

  • Glynn Zeederberg (unregistered) in reply to Outlaw Programmer

    IIS automatically closes inactive sessions after a specified period of time (usually 20 minutes default).

    I doubt that the broweser was keeping those sessions open as that machine would die before the server would (unless the server is on a 5 year old machine :P)

    Also, for SQL to throw out of memory is strange. I'd assume it would start paging and just give a rediclously slow response.

  • Richard (unregistered) in reply to Outlaw Programmer
    Outlaw Programmer:
    I call shenanigans. Management would have just told the developer to schedule a task that restarts the 2 services every night. At the places where I've worked, any bug that has a workaround never gets fixed; the workaround just gets added to the user manual!

    +1 Million. That's the M.O. at every publicly traded company I've every worked at. They run too lean to actually have someone do a deep dive into a problem.

  • HttpStatusCodesAreThereForAReason (unregistered)

    looks fishy..

    if favicon.ico were actually missing you;d get a 404 as the http status code in the log .. not 302..

    if its really a 302 then its a misconfigured redirection...

  • Anonymous (unregistered) in reply to Outlaw Programmer

    Making them reboot the server every 2 hours would be pointless if the bug only occured once every few days/weeks.

  • eric bloedow (unregistered)

    reminds me of the infamous "E-mail forwarding loop". account 1 forwards to account 2, which forwards back to account 1...

Leave a comment on “The Most Favoritest Icon”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article