The Daily WTF: Curious Perversions in Information Technology

2013-03-11 Reply Admin

Race conditions ftw :D

2013-03-11 Reply Admin

Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use

I had a similar issue where I wrote a program for a closed system: When a technician plugged in a USB drive (only they had access to the USB ports) the system automatically checked the drive for an update file. The problem was that the anti-virus seemed to be quicker sometimes and lock the file for access. Thus I needed a way to check when the file became available. I used the method above.

I must say I used a loop in conjunction with a fail timer (if the file did not become available after a few seconds).

2013-03-11 Reply Admin

Basically, this means that they bought cc-wiki licensed code without the required attribution? :p

2013-03-11 Reply Admin

I wouldn't do that, at least not with closing the stream after checking. What happens if another process is locking the file just right after your check and before reopening the stream?

2013-03-11 Reply Admin

I would be interested in the code she used instead. Whenever I need to check for file locks in C# I use the try-catch statement, too. Thanks in advance

2013-03-11 Reply Admin

stream = file.Open(FileMode.Open, FileAccess.ReadWriteFrist, FileShare.None);

ZPedro · 2013-03-11 Reply Admin

Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use

The problem is that the very notion of "checking if a file is" whatever, before you do an operation, is not correct in the first place (so a "correct way" to do it is meaningless). You try with the semantics you need (e.g. tell to write, or tell to write without overwriting if there is already a file), then handle failure.

In some cases you need to do some checks, e.g. to check the temporary file you opened is not a symlink to another part of the file system, for security reasons, but you do so after you opened the file (since it's not going to change once you opened it).

Notice you might want to know whether the fiile doesn't exist, or you lack permissions to open it or write to it or anything else, and therefore do checks on files without opening or whatever (which may include a try-catch), but you do so only for user indication, as part of the "handle failure" thing, you do not take any decision based on these extra checks.

2013-03-11 Reply Admin

Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use

It appears as if that's the true source of this code, even the comment is identical.

xorsyst · 2013-03-11 Reply Admin

Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use.

Right, everybody go to that question and upvote the actually correct answer, which is slightly further down. We can fix this!

2013-03-11 Reply Admin

How does this give you a stutus? It just throws the exception ;-)

Medinoc · 2013-03-11 Reply Admin

ZPedro:
Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use

The problem is that the very notion of "checking if a file is" whatever, before you do an operation, is not correct in the first place (so a "correct way" to do it is meaningless). You try with the semantics you need (e.g. tell to write, or tell to write without overwriting if there is already a file), then handle failure.

In some cases you need to do some checks, e.g. to check the temporary file you opened is not a symlink to another part of the file system, for security reasons, but you do so after you opened the file (since it's not going to change once you opened it).

Notice you might want to know whether the fiile doesn't exist, or you lack permissions to open it or write to it or anything else, and therefore do checks on files without opening or whatever (which may include a try-catch), but you do so only for user indication, as part of the "handle failure" thing, you do not take any decision based on these extra checks.

This is true. However, there may be some performance advantage in first trying something that doesn't throw exceptions (like File.Access()) because:

Exceptions are expensive
Exceptions show up on the debugger even if caught before they reach your code (the only debugger options are "thrown" and "not handled at all", with nothing in-between). But that is purely a shortcut, so one should always handle failure.

Steve The Cynic · 2013-03-11 Reply Admin

Talib:
I had a similar issue where I wrote a program for a closed system: When a technician plugged in a USB drive (only they had access to the USB ports) the system automatically checked the drive for an update file. The problem was that the anti-virus seemed to be quicker sometimes and lock the file for access. Thus I needed a way to check when the file became available. I used the method above.

Replace the anti-virus system. It is defective. It allows one thread to fail an I/O request on the file that the A/V is working on in another. Instead, it should cause the thread to wait, so that (timing issues left to one side) it does not alter the system behaviour.

((Many years ago, I worked on an on-access A/V scanner for a major vendor of the time. Not the actual scanning component, but the component that captured file access calls on Win95, so I know something about what's needed.))

2013-03-11 Reply Admin

Hey guys, have you heard about that awesome invention? It's called a mutex and it's really cool.

2013-03-11 Reply Admin

ZPedro:
Notice you might want to know whether the fiile doesn't exist

Oh, wait, I know this one! Since isFileOpen is a boolean function...

ZPedro · 2013-03-11 Reply Admin

Maciej :
Hey guys, have you heard about that awesome invention? It's called a mutex and it's really cool.

Oh, please tell me the filesystem API you are using so that I can trivially DOS attack everything that uses it by taking that mutex then never releasing it.

2013-03-11 Reply Admin

Maciej:
ZPedro:
Notice you might want to know whether the fiile doesn't exist

Oh, wait, I know this one! Since isFileOpen is a boolean function...

Me too; true, false, FileNotFound.

ZPedro · 2013-03-11 Reply Admin

Medinoc:
ZPedro:
<snip>
This is true. However, there may be some performance advantage in first trying something that doesn't throw exceptions (like File.Access()) because: 1) Exceptions are expensive 2) Exceptions show up on the debugger even if caught before they reach your code (the only debugger options are "thrown" and "not handled at all", with nothing in-between). But that is purely a shortcut, so one should always handle failure.

Hmm, I work more in the context of POSIX filesystem APIS (open(), creat(), read() and the like) so by "try then handle failure" I did not necessarily mean something that could throw exceptions, but things may be different in the C#/.NET world. Is it really impossible to try/handle failure with filesystem operations without throwing an exception in case of failure in .NET? If so, then it's a Microsoft/.NET WTF.

(of course this side discussion barely applies here, since the "first trying something" in today's WTF does involve an exception itself, and is apparently the «"“correct”"» way to do such a check)

2013-03-11 Reply Admin

2013-03-11 06:05 • by mm (unregistered) Race conditions ftw :D

2013-03-11 06:39 • by mm (unregistered) What happens if another process is locking the file just right after your check and before reopening the stream?

Race condition: when it takes you 39 minutes to write your post but another post said the same thing in just 5 minutes.

Oh, wait! Both posts were by the same person?

What, did you take a nap and forget what you already posted?

2013-03-11 Reply Admin

Medinoc:
ZPedro:
Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use

The problem is that the very notion of "checking if a file is" whatever, before you do an operation, is not correct in the first place (so a "correct way" to do it is meaningless). You try with the semantics you need (e.g. tell to write, or tell to write without overwriting if there is already a file), then handle failure.

In some cases you need to do some checks, e.g. to check the temporary file you opened is not a symlink to another part of the file system, for security reasons, but you do so after you opened the file (since it's not going to change once you opened it).

Notice you might want to know whether the fiile doesn't exist, or you lack permissions to open it or write to it or anything else, and therefore do checks on files without opening or whatever (which may include a try-catch), but you do so only for user indication, as part of the "handle failure" thing, you do not take any decision based on these extra checks.
This is true. However, there may be some performance advantage in first trying something that doesn't throw exceptions (like File.Access()) because:

Exceptions are expensive

Exceptions show up on the debugger even if caught before they reach your code (the only debugger options are "thrown" and "not handled at all", with nothing in-between). But that is purely a shortcut, so one should always handle failure.

Unless you have some stupendously fast harddrives, I'd wager that the time spent with throwing an exception is easily eaten by the time spent doing I/O, thus I dub this worry as premature optimization.

2013-03-11 Reply Admin

Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?

2013-03-11 Reply Admin

This brings back misty memories of:

Open disk directory (the only one on the disk)
Read filename
Filename = the one we want? No: go to 2.

As it turned out, this approach was particularly handy in Atari DOS when you needed to open the second file with the exact same name. Just keep looping past the first one.

And how do you get two files with the same name? Easy:

Create file ABC
Create file DEF
Rename DEF to ABC

2013-03-11 Reply Admin

I know this one, I have two myself.

Uh. wait- uh..

2013-03-11 Reply Admin

Maciej :
Hey guys, have you heard about that awesome invention? It's called a mutex and it's really cool.

I'm a little busy right now, can you tell me about it later?

TheEgg · 2013-03-11 Reply Admin

mm:
I wouldn't do that, at least not with closing the stream after checking. What happens if another process is locking the file just right after your check and before reopening the stream?

INH! TRRrrrRRuuuust mmeeee!

Nagesh · 2013-03-11 Reply Admin

try catch block is acceptable method of handling this situation. what else could AJ do?

2013-03-11 Reply Admin

ZPedro:
Maciej :
Hey guys, have you heard about that awesome invention? It's called a mutex and it's really cool.

Oh, please tell me the filesystem API you are using so that I can trivially DOS attack everything that uses it by taking that mutex then never releasing it.

I figured that since the code comments said "another thread", they meant that another thread in the same application can be using this file. If another application is using the file, catch the exception and show an error message, there's nothing more to do anyway.

no laughing matter · 2013-03-11 Reply Admin

No one complains about TRWTF?

WriteToFile happily returns without any exception or other indication that the write was not successful if the file is locked!

dkf · 2013-03-11 Reply Admin

ZPedro:
Hmm, I work more in the context of POSIX filesystem APIS (open(), creat(), read() and the like) so by "try then handle failure" I did not necessarily mean something that could throw exceptions, but things may be different in the C#/.NET world. Is it really impossible to try/handle failure with filesystem operations without throwing an exception in case of failure in .NET? If so, then it's a Microsoft/.NET WTF.

No, that's just how the result of the OS call is mapped in the language. It's the same in a lot of languages, and is a natural thing to do when you've got exceptions built into the language. Languages without exceptions tend to do it with return codes when then need to be explicitly checked: the number of programmers who never do this indicates that that's a poor choice when you can instead throw something that the programmer's got to do something about. (Even if they just throw it away… Ho hum.)

The only time I've seen anyone actually upset by the try/handle-failure pattern (whether in its C#/Java incarnation or its classic C incarnation) was with someone who was complaining that the software was filling up the audit log, which was recording every single failing system call. That was definitely TRWTF. Also a facepalm moment.

2013-03-11 Reply Admin

Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?

The purpose of kids is to have grandkids. You just have to wait 30 years. The purpose of grandkids is when you are over 50 someone will still talk to you and think you are cool.

2013-03-11 Reply Admin

Romano:
I would be interested in the code she used instead. Whenever I need to check for file locks in C# I use the try-catch statement, too. Thanks in advance

Uh, just put the try-catch around the original open()? Makes no sense to do an open just to see if it works, then close and reopen. Oh, and handle the exception a little more detailed than just an IO exception maybe.

2013-03-11 Reply Admin

I see no WTF in the method copy/pasted from Stack Overflow.

The thing with file locks is that they are a filesystem way of addressing the 'Dining Philosophers' problem, where of n entities, only one can 'eat' from the plate at a time, while the others think.

A race condition is somewhat inherent in the setup; if you try accessing a file before the lock has been removed by the current consumer, well, you shouldn't be able to access it. That try/catch block looks for the lock, and if it detects that the file is locked, tells the app that the file...is locked. Pretty self explanatory.

What did AJ do to it, remove the comments?

Cbuttius · 2013-03-11 Reply Admin

Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use
I had a similar issue where I wrote a program for a closed system: When a technician plugged in a USB drive (only they had access to the USB ports) the system automatically checked the drive for an update file. The problem was that the anti-virus seemed to be quicker sometimes and lock the file for access. Thus I needed a way to check when the file became available. I used the method above.

I must say I used a loop in conjunction with a fail timer (if the file did not become available after a few seconds).

It is the correct way to check if a file is locked if, presumably, the purpose is that if it is not locked you open it and perform some operation with it.

Because if you call some method that returns a boolean that says it is not locked, then you may try opening it only to find that a race condition has occurred and it now really is locked.

So if you are going to handle that condition anyway, just open it.

If you don't really want to do anything with the file but are asserting a file is not locked as part of some unit test, then do what this does by all means. Who cares if it takes a few extra resources, it's a unit test.

File system operations are likely to be far more expensive than any call-stack unwinding that takes place during an exception. I find in general that developers generally are told that exceptions are inefficient and therefore tend to avoid using them for premature optimisation. I do not recall a time where I optimised an application where I did it by removing exceptions (i.e. I found exceptions were making the code slow).

I have also never seen exception safety making code slow. (Weak exception safety is not an issue in C# but if you want a strong exception guarantee that applies in C# too).

2013-03-11 Reply Admin

So the real wtf is the spurious inclusion of husband and kids, in order to make tenuous metaphor with code, amirite?

shoddycoder · 2013-03-11 Reply Admin

FileStream stream = null;

Never seen the point of this, the object is null if uninitilized anyway and you've taken out a compiler check (in Java) by intializing to null

2013-03-11 Reply Admin

Xarthaneon the Unclear:
I see no WTF in the method copy/pasted from Stack Overflow.
The thing with file locks is that they are a filesystem way of addressing the 'Dining Philosophers' problem, where of n entities, only one can 'eat' from the plate at a time, while the others think.

A race condition is somewhat inherent in the setup; if you try accessing a file before the lock has been removed by the current consumer, well, you shouldn't be able to access it. That try/catch block looks for the lock, and if it detects that the file is locked, tells the app that the file...is locked. Pretty self explanatory.

What did AJ do to it, remove the comments?

Thread A: if (!IsFileLocked()) { int a = ReadFile(filePath); WriteFile(filePath, a+1); }

Thread B: if (!IsFileLocked()) { int a = ReadFile(filePath); WriteFile(filePath, a+1); }

Execution order: A: if(!IsFileLocked()) // opens file, returns true closes file B: if(!IsFileLocked()) // A closed file, B opens file, returns true, closes file A: int a = ReadFile // a = 1; B: int a = ReadFile // a = 1; A: WriteFile(... 2); // Opens file, begins to write... B: WriteFile(... 2); // File IO Exception, fails to write...

Not only did they both try to write the number 2, instead of 3 being written by one of them, B threw a file exception...

Double WTF

2013-03-11 Reply Admin

I have some annoying blood relatives as well. Also, I have had to work with some annoying code.

These are related, because both are annoying; its almost as if the relatives are a living breathing embodiment of the the code, and the code is a sort of mathematical representation of the relatives.

I think I've made my point.

Cbuttius · 2013-03-11 Reply Admin

Valued Service:
Xarthaneon the Unclear:
I see no WTF in the method copy/pasted from Stack Overflow.
The thing with file locks is that they are a filesystem way of addressing the 'Dining Philosophers' problem, where of n entities, only one can 'eat' from the plate at a time, while the others think.

A race condition is somewhat inherent in the setup; if you try accessing a file before the lock has been removed by the current consumer, well, you shouldn't be able to access it. That try/catch block looks for the lock, and if it detects that the file is locked, tells the app that the file...is locked. Pretty self explanatory.

What did AJ do to it, remove the comments?

Thread A: if (!IsFileLocked()) { int a = ReadFile(filePath); WriteFile(filePath, a+1); }

Thread B: if (!IsFileLocked()) { int a = ReadFile(filePath); WriteFile(filePath, a+1); }

Execution order: A: if(!IsFileLocked()) // opens file, returns true closes file B: if(!IsFileLocked()) // A closed file, B opens file, returns true, closes file A: int a = ReadFile // a = 1; B: int a = ReadFile // a = 1; A: WriteFile(... 2); // Opens file, begins to write... B: WriteFile(... 2); // File IO Exception, fails to write...

Not only did they both try to write the number 2, instead of 3 being written by one of them, B threw a file exception...

Double WTF

The correct API if you want a safe way without exceptions is a try_lock() method.

This returns to you immediately whether or not you have successfully locked. If you have you can use the file and need to unlock. If not you can choose to wait or move on to do something else.

C doesn't have exceptions and POSIX has try_lock for mutex and the concept can be applied for other resource locks too.

I don't know the C# library so I don't know if it exists there.

2013-03-11 Reply Admin

eVil:
I have some annoying blood relatives as well. Also, I have had to work with some annoying code.
These are related, because both are annoying; its almost as if the relatives are a living breathing embodiment of the the code, and the code is a sort of mathematical representation of the relatives.

I think I've made my point.

 vvv
(4/4)
  =
1+6+1
 /1\
 2^3

2013-03-11 Reply Admin

I've used this exact snippet. Here's a scenario: I'm batch processing files that are dumped into a share by photo copiers. However, sometimes when the batch is kicked off, the photo copiers will still be transferring a file. So, if a file is locked, just kick it to the end of the line and if it's still locked after processing everything else do an exponential back off. The key thing in my situation though, is that there is only one thing that will be locking the file so race conditions aren't a big deal.

2013-03-11 Reply Admin

xorsyst:
Talib:
Well surprisingly this seems to be the correct way for checking if a file is in use according to: http://stackoverflow.com/questions/876473/is-there-a-way-to-check-if-a-file-is-in-use.

Right, everybody go to that question and upvote the actually correct answer, which is slightly further down. We can fix this!

The right answer to that questions is actually the answer to another question: http://stackoverflow.com/a/3202085/137451.

2013-03-11 Reply Admin

For those that still think Exceptions are 'expensive', I suggest you read the below before trying to optimize them out.

http://www.developerfusion.com/article/5250/exceptions-and-performance-in-net/

(TLDR: Exceptions are only expensive in a) loops that throw 1000's of them, and b) in the debugger)

chubertdev · 2013-03-11 Reply Admin

Cbuttius:
File system operations are likely to be far more expensive than any call-stack unwinding that takes place during an exception. I find in general that developers generally are told that exceptions are inefficient and therefore tend to avoid using them for premature optimisation. I do not recall a time where I optimised an application where I did it by removing exceptions (i.e. I found exceptions were making the code slow).

You're really, really lucky then. I had a system that was brought to a grinding halt just from the exceptions inherent in the utility methods used thousands of times per postback. Fixing those improved the speed of the app to the point where it was actually usable, and actually, was quite impressive.

I know that's probably an "edge" case, but you have to remember that 80% of developers can't program, so there will be a ton of apps like that.

2013-03-11 Reply Admin

Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?

The purpose of children is so that, when you retire, there is someone to pay social security taxes to support you.

Glad I was able to clear that up for you.

2013-03-11 Reply Admin

Hmm, this code is certainly flawed. If you test if you can open the file, and then if you can you go ahead and open the file, you create the possibility of a race problem, where someone else manages to lock the file or delete it or otherwise make it unopenable in between your test and your attempt at actual execution. The right way to do this is to try to open the file and catch the exception on the attempted open.

Also, if the file does not exist, then retrying the operation is not likely to be useful if you do nothing to create the file. Well, unless we're expecting that there is some other process that creates the file.

But the point of the article seems to be to ridicule the idea of retrying repeatedly after a failure. But if a likely cause of failure is that the file is locked out by another process, what would you suggest as an alternative? If there are multiple apps that occasionally lock a file and write to it, should we abort if we find that someone else has the file at the moment? Why? Just keep trying until it's free. I've done this many times. I always put some upper limit on it so it doesn't keep trying for hours.

I'd be interested to see what her "correct" code was.

herby · 2013-03-11 Reply Admin

Dave:
Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?
The purpose of kids is to have grandkids. You just have to wait 30 years. The purpose of grandkids is when you are over 50 someone will still talk to you and think you are cool.

Sorry, the real purpose of kids (and grandkids for that matter is to pay for the government you have now. This includes things like Social Security (misnomer), and Medicare (another joke). The big reason though is so they will take care of you in your advancing age.

Unfortunately for me, all I have are nieces and nephews (SIGH).

Please note that this thread might wander into Godwins law very quickly if we aren't careful.

Lastly: The Muppet show was REAL COOL for both adults and kids!

2013-03-11 Reply Admin

herby:
Dave:
Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?
The purpose of kids is to have grandkids. You just have to wait 30 years. The purpose of grandkids is when you are over 50 someone will still talk to you and think you are cool.
Sorry, the real purpose of kids (and grandkids for that matter is to pay for the government you have now. This includes things like Social Security (misnomer), and Medicare (another joke). The big reason though is so they will take care of you in your advancing age.
Unfortunately for me, all I have are nieces and nephews (SIGH).

Please note that this thread might wander into Godwins law very quickly if we aren't careful.

Lastly: The Muppet show was REAL COOL for both adults and kids!

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Something about this reminds me of that other something, which is a lot like Hitler. Quisque a diam eu justo cursus ornare quis non lorem.

2013-03-11 Reply Admin

herby:
Dave:
Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?
The purpose of kids is to have grandkids. You just have to wait 30 years. The purpose of grandkids is when you are over 50 someone will still talk to you and think you are cool.
Sorry, the real purpose of kids (and grandkids for that matter is to pay for the government you have now. This includes things like Social Security (misnomer), and Medicare (another joke). The big reason though is so they will take care of you in your advancing age.
Unfortunately for me, all I have are nieces and nephews (SIGH).

Please note that this thread might wander into Godwins law very quickly if we aren't careful.

Lastly: The Muppet show was REAL COOL for both adults and kids!

HITLER!! (nailed it)

2013-03-11 Reply Admin

Mike:
Kids are TRWTF, AJ. I mean, what useful purpose do they serve, other than to make a mess of the place you had to move to because his bachelor pad wasn't "homey" enough, destroy all the expensive crap you made him buy you, and leave the lights and TV on until they're old enough to finance a major drug addiction?

Clearly we'd all be better off if Mike's parents had been as smart as him.

2013-03-11 Reply Admin

Gabe:
I've used this exact snippet... The key thing in my situation though, is that there is only one thing that will be locking the file so race conditions aren't a big deal.

Not a big deal - until it is. So why not just solve it properly in the first place? Its not like the solution is difficult.

2013-03-11 Reply Admin

Interesting that a file is defined as "locked" if it doesn't exist. Is there even a way to reproduce this in a way that wouldn't be prone to race-conditions?

The Right Way to Find a File

Leave a comment on “The Right Way to Find a File”