• Mathew (unregistered) in reply to ClaudeSuck.de
    ClaudeSuck.de:
    TRWTF is the article.
    +1

    And commenting is feeding the troll.

  • Gomez Addams (unregistered)

    This isn't a WTF at all - it is a workaround to a "feature" of perl running under windows.

    The perl "close" doesn't always close the file immediately - at least some versions of perl running under some versions of windows.

    I have seen this behavior myself. I spent hours debugging a perl program that closed a file and immediately unlinked it. The unlink would occasionaly fail with a "permission denied" error. Putting a sleep(1) after the close permantly fixed the problem for me.

    I know that at least one other person has seen this same behaviour. He posted it the on the win32 perl users mailing list

  • (cs) in reply to Gomez Addams
    Gomez Addams:
    This isn't a WTF at all - it is a workaround to a "feature" of perl running under windows.

    The perl "close" doesn't always close the file immediately - at least some versions of perl running under some versions of windows.

    I have seen this behavior myself. I spent hours debugging a perl program that closed a file and immediately unlinked it. The unlink would occasionaly fail with a "permission denied" error. Putting a sleep(1) after the close permantly fixed the problem for me.

    I know that at least one other person has seen this same behaviour. He posted it the on the win32 perl users mailing list

    +1, I've also heard of some antivirus that could cause similar problems.

    So, TRWTF is Perl. If close() does not call sync(), there's something very wrong.

    PS: How's your brother?

  • TheJonB (unregistered) in reply to Medinoc
    Medinoc:
    So, TRWTF is Perl. If close() does not call sync(), there's something very wrong.
    I would imagine that perl's close() calls the underlying O/S close.

    What do you imagine it doing?

  • Gomez Addams (unregistered)

    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

  • TheJonB (unregistered) in reply to Gomez Addams
    Gomez Addams:
    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

    So, I do a bit of windows stuff, although not with perl that fits so much better on linux.

    I'm curious, when closing a file, what must you do - that Perl might not be doing - to release that file fully back to the O/S?

  • Gomez Addams (unregistered) in reply to TheJonB
    TheJonB:
    Gomez Addams:
    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

    So, I do a bit of windows stuff, although not with perl that fits so much better on linux.

    I'm curious, when closing a file, what must you do - that Perl might not be doing - to release that file fully back to the O/S?

    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

  • TheJonB (unregistered) in reply to Gomez Addams
    Gomez Addams:
    TheJonB:
    Gomez Addams:
    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

    So, I do a bit of windows stuff, although not with perl that fits so much better on linux.

    I'm curious, when closing a file, what must you do - that Perl might not be doing - to release that file fully back to the O/S?

    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

    1. It doesn't.
    2. Windows won't let you delete a file that another app has open, you must code for this.
    3. It isn't.

    It seems that you are TRWTF.

    Goodbye.

  • Gomez Addams (unregistered) in reply to TheJonB
    TheJonB:
    Gomez Addams:
    TheJonB:
    Gomez Addams:
    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

    So, I do a bit of windows stuff, although not with perl that fits so much better on linux.

    I'm curious, when closing a file, what must you do - that Perl might not be doing - to release that file fully back to the O/S?

    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

    1. It doesn't.
    2. Windows won't let you delete a file that another app has open, you must code for this.
    3. It isn't.

    It seems that you are TRWTF.

    Goodbye.

    Try to keep up here. As I stated, my code simply opened a file, wrote to it, CLOSED IT, then unlinked it. Sporaticly, the unlink would fail. No other process would have the file open (except perhaps AV).

    I suspect that whoever had originally written the code quoted in the article had ran into this and simply gotten into the habit of using a sleep() after every close.

  • Engelbart (unregistered)

    We have sleeps like that in a lot of our scripts... It comes down to code like this

    copy fileA fileB extract some_info from fileB

    -- Error FileB does not exist!

    WTF?

    This happens about 1 in 20 times when FileB is on a windows share... It never happens when dealing with local file systems.

    copy fileA fileB sleep 5 extract some_info from fileB

    Never has this problem, even if fileB is on a share.

    The newb was probably testing against an environment that ran everything off of local disks.

    The production execution was probably moving stuff around between different network locations, hence the sleeps were needed.

    Newb deserved the Stare(TM), the veteran probably got a support page at 2AM when some job failed.

    Not sure why he would have increased 5 to 6 if 5 had worked all of those years.

  • John Hensley (unregistered)

    Hopefully since this job Dave has grown a little backbone with regard to dirty looks.

  • Cat (unregistered) in reply to Gomez Addams
    Gomez Addams:
    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

    I've seen the exact same thing in C#. Writing temp files that will be closed and then deleted, in a small number of cases (maybe 1 in a thousand) the delete fails for no discernible reason.

  • TheJonB (unregistered) in reply to Gomez Addams
    Gomez Addams:
    TheJonB:
    Gomez Addams:
    TheJonB:
    Gomez Addams:
    I had considered the possibility that antivirus was causing the problem. If I get some free time (hah!) I may try testing the close and unlink on a system without AV installed and see if it fails there.

    So, I do a bit of windows stuff, although not with perl that fits so much better on linux.

    I'm curious, when closing a file, what must you do - that Perl might not be doing - to release that file fully back to the O/S?

    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

    1. It doesn't.
    2. Windows won't let you delete a file that another app has open, you must code for this.
    3. It isn't.

    It seems that you are TRWTF.

    Goodbye.

    Try to keep up here. As I stated, my code simply opened a file, wrote to it, CLOSED IT, then unlinked it. Sporaticly, the unlink would fail. No other process would have the file open (except perhaps AV).

    I suspect that whoever had originally written the code quoted in the article had ran into this and simply gotten into the habit of using a sleep() after every close.

    SPORADICALLY FFS.

    It ain't Perl, as Perl simply uses the underlying O/S open and close, Perl's close is not threaded, it is not magic, it does not give you AIDS, it does not require a little rest nor a moment of quiet contemplation before or after calling..

    I also cannot reproduce the behaviour you somewhat vaguely describe (XP, with AVG).

  • L. (unregistered) in reply to BR
    BR:
    L.:
    Would you care to point out for what else exactly Perl is the *best* choice?

    In our case, it's when we already have a ton of it, and it works.

    But it's also very good for text processing (a part of which you pointed out), acting as a glue language, basic networking stuff (web scraping, Nagios stuff, etc), administration scripts, etc. It's also good when you need to get a lot done fast. But I do agree that unless you're careful, that can lead to the "write-once" pejorative everyone is so fond of throwing around.

    Would you use it to write a GUI desktop app? No. Grabbing the output of apache's /server-status and throwing the info into an in-house monitoring app? Sure.

    Outside of the text processing .. are you really seriously considering Perl as better than C ??

    And the glue .. lmao ? sh, php, py, just about anything does it just as fine.

  • Greeno (unregistered)

    On the face of it, this looks stupid as there is clearly no relationship between the close mechnism and the following sleep. However depending on the complexity of the application, it's entirely possible that the sleep may be (possibly inadvertantly) preventing race conditions or deadlocks.

    You can't just go changing (even stupid) things on a hunch without confirming that QA has available resource to perform suitable regression testing.

  • TheJonB (unregistered) in reply to Greeno
    Greeno:
    On the face of it, this looks stupid as there is clearly no relationship between the close mechnism and the following sleep. However depending on the complexity of the application, it's entirely possible that the sleep may be (possibly inadvertantly) preventing race conditions or deadlocks.

    You can't just go changing (even stupid) things on a hunch without confirming that QA has available resource to perform suitable regression testing.

    There's no threading problem that is best solved by waiting for a bit.

    Any such solution is itself a WTF.

  • (cs)

    The fact that nobody made a Meatloaf reference proves there are no women reading this site.

  • Cbuttius (unregistered) in reply to frits
    frits:
    The fact that nobody made a Meatloaf reference proves there are no women reading this site.

    I'll give you an answer in the morning.

  • L. (unregistered) in reply to Cat
    Cat:
    Gomez Addams:
    Nothing explicit. My code would work most of the time - it only failed sporaticly. I figure:

    1. the internal code of perl does some threading that leads to a race condition on multi-cpu machines. This bug would not show up on unix because you can unlink and open file, and when it is eventually closed, the end result is what you expected.

    2. Antivirus software is holding the file open.

    3. The windows file close function is returning to the caller before the close completes in some cases.

    I never investigated any further after finding that the sleep(1) fixed the problem, but I must say that my curiosity is piqued. I may play around a bit more.

    I've seen the exact same thing in C#. Writing temp files that will be closed and then deleted, in a small number of cases (maybe 1 in a thousand) the delete fails for no discernible reason.

    Let me guess, that was on windows right ?

  • Miroslav (unregistered) in reply to L.
    L.:
    Miroslav:
    Well, hard drive is a physical device. Those 16 Mb will not just magically appear on the drive. Read/write head must have time to record every single byte. HD memory cache will help, but if smaller than your total file size(s)...

    I'm actually somewhat surprised that you don't need longer than 1 ms between writes. I guess HDs are a lot faster nowadays.

    You DO realize there is an OS and drivers between you and the hard drive and you don't just write from perl into the harddrive right ?

    Yes. And they are not likely to make things any faster, either.

  • L. (unregistered) in reply to Miroslav
    Miroslav:
    L.:
    Miroslav:
    Well, hard drive is a physical device. Those 16 Mb will not just magically appear on the drive. Read/write head must have time to record every single byte. HD memory cache will help, but if smaller than your total file size(s)...

    I'm actually somewhat surprised that you don't need longer than 1 ms between writes. I guess HDs are a lot faster nowadays.

    You DO realize there is an OS and drivers between you and the hard drive and you don't just write from perl into the harddrive right ?

    Yes. And they are not likely to make things any faster, either.

    Alright . you don't get it do you ?

    When you code : write(filename,filecontent) in any language, it asks the OS to do that alright ?

    Then the OS uses the drivers to have the HDD do it

    Then it receives the ACK for the last write instruction

    THEN it returns.

    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.

  • TheJonB (unregistered) in reply to L.
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.
  • L. (unregistered) in reply to TheJonB
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

  • (cs) in reply to L.
    L.:
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

    Disk write caching anyone? Anyone?

  • TheJonB (unregistered) in reply to frits
    frits:
    L.:
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

    Disk write caching anyone? Anyone?

    Again, if the O/S doesn't flush on close then waiting a millisecond or two isn't going to do it either.

  • TheJonB (unregistered) in reply to TheJonB
    TheJonB:
    frits:
    L.:
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

    Disk write caching anyone? Anyone?

    Again, if the O/S doesn't flush on close then waiting a millisecond or two isn't going to do it either.

    I should have also said that the O/S should be aware of that as well, so the file system will still be consistent for a subsequent delete.

    The filesystem is the whole caboodle, O/S, memory, cache, raid controller, device drivers, flash ram, disk surface. It's not just the disk platter surface.

    When you write, you write to the filesystem, the filesystem decides whether it goes on the disk or not. Similarly when you read it needn't have gone anywhere near the disk at all, but the filesystem will be consistent, because that's what filesystems do.

    ...if they aren't broken.

  • L. (unregistered) in reply to frits
    frits:
    L.:
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

    Disk write caching anyone? Anyone?

    Of course there's disk write caching.

    1. os, please write stuff
    2. OS writes (including optimization and write caching and even file system caching i.e. not even hdd cache itself)
    3. OS says all done
    4. os, please read stuff
    5. OS reads stuff

    IF IT DOESN'T happen that way, your OS is broken.

    B R O K E N

    If you don't get that, you should stop working in IT, you're only going to create more broken stuff.

    hint: even with disk caching, your os can and should behave like said above AND the only way it would fail is if you had asked for an async system call and in that case, you are TRWTF for going async + wait instead of sync.

  • (cs) in reply to L.
    L.:
    frits:
    L.:
    TheJonB:
    L.:
    There is no valid reason to wait 1ms after that, unless the above is not true, i.e. your OS is broken.
    Some sanity at last, I was starting to think that it was me.

    Nah it's just that people got used to coding fail-OS workarounds right into applications - to the point where they don't even realize how ridiculous the root cause is.

    Disk write caching anyone? Anyone?

    Of course there's disk write caching.

    1. os, please write stuff
    2. OS writes (including optimization and write caching and even file system caching i.e. not even hdd cache itself)
    3. OS says all done
    4. os, please read stuff
    5. OS reads stuff

    IF IT DOESN'T happen that way, your OS is broken.

    B R O K E N

    If you don't get that, you should stop working in IT, you're only going to create more broken stuff.

    hint: even with disk caching, your os can and should behave like said above AND the only way it would fail is if you had asked for an async system call and in that case, you are TRWTF for going async + wait instead of sync.

    Your hubris amuses me. Please continue posting here.

  • TheJonB (unregistered) in reply to frits
    frits:
    Your hubris amuses me. Please continue posting here.
    Oh FFS he's just trolling and not an idiot after all.

    Got both of us there. >:(

  • (cs)

    TRWTF is CVS. Seriously? In 2012?

  • AdT (unregistered) in reply to Rootbeer
    Rootbeer:
    You need to escape your parentheses.

    I escaped my parentheses once, but the police brought me back and my parentheses were very angry. :-(

  • (cs)

    the real WTF is that Paradise By The Dashboard Light was by Meat Loaf, not Alvin Stardust..

  • (cs) in reply to Engelbart
    Engelbart:
    ...

    and now it appears even Engelbert Humperdinck wants his piece of the action.

  • GeraldDew (unregistered)

Leave a comment on “Let Me Sleep on It”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article