• Yawn (unregistered)

    So it's a WTF now to not know every nook & cranny of the standard library, and spend 10 minutes writing a small bit of code instead of searching the web? Screw that high-handed elitism. It's used once, so who cares if it's not exactly the canonical way or the most efficient way?

    Standards have seriously dropped at TheDailyWTF when this kind of code is considered to be a WTF. Is nobody submitting any real WTF code, like Paula Bean's code or True/False/FILE_NOT_FOUND any more?

  • Quite (unregistered)

    "Which is what Anneke did."

    ... and immediately on code release to Production, broke an important edge case which had been specifically crafted to take unfair advantage of certain abstruse aspects of the triangular coffee table approach, hosing a vast swathe of customer data, indirectly causing a worldwide depression and thereby making a significant contribution to the end of WestCiv as we know it.

  • Jim (unregistered)

    “Java” is a funny misspelling of “Smalltalk”

  • (nodebb)

    Pretty darn sure the two designs will yield different results for the general use-case [rather than just looking at the simple/single example shown..

    "I want to stress, this is the only use of this method. " - How can this be proven??? Demonstrate that the author did not use such method in another unrelated project, perhaps even at a different company!

  • HdS (unregistered)

    If you use Python 3.5+, you could and should pathlib.Path

    path = pathlib.Path("some/path") path.Exists()

    I like this API - a filesystem is NOT comprised of strings.

  • Robert Morson (google) in reply to Yawn

    It's a WTF not to think, "This seems like a common process. I wonder if it's in the standard library?" and then take literally TEN F*CKING SECONDS to find out. It used to be this would take longer, but in today's world of ubiquitous Google access, I claim this is an accurate use of the word "literally." Anyone who doesn't use it lacks common sense.

  • bvs23bkv33 (unregistered)

    The wheels on the bus go round and round,

    Round and round, round and round.

    The wheels on the bus go round and round,

    All day long!

  • RLB (unregistered)

    Ok, but... who is Anneke? Or is she a feature of the DailyWTF.contributors.name() system library function we're not sufficiently familiar with?

  • IP Guru (unregistered)

    The pythonic way to do this would be to not even bother checking if the file exists

    simply TRY to open the file & then set the appropriate EXCEPTions to take the apropriat action when it fails

    Python prefers "better to ask forgives" approach to the "look before you leap" approach s it is quite possible for the file to cease to exist in the time between checking it & actual using it

  • Triangular Wheel (unregistered) in reply to bvs23bkv33

    The wheels on the bus go up and down,

    Up and down, up and down.

    The wheels on the bus go up and down,

    All day long!

  • Racer (unregistered) in reply to IP Guru

    Checking if a file exists before opening is useless on any language. It is always possible for an external process to create/delete the file between those two operations.

  • Qazwsx (unregistered) in reply to RLB

    She's the submitter; the clue is the one-sentence paragraph near the top that says

    Anneke sends us one such method.

  • Yawn (unregistered) in reply to Robert Morson

    Maybe the dev couldn't be online at the time (yes, that's actually a thing that happens), wrote it with what they know, and called it a day. But yes, let's look it up, shall we? Hm, it'll probably be in high-level file operations, so that's shutils ... oh it's not there. Maybe it's somewhere around the stat module? Nothing there, right, let's ask Google ... "python find out if file exists" ... oh here's one, os.path.isfile, oh dear, but there seem to be some interactions with symlinks and directories ... maybe exists? But it has the same kind of issues.

    And at this point, I say "Screw it, writing my own. It's only ever used once", and then I move on with my life. It's a whole EIGHT LINES oh noes! Horror! Entire codebase ruined, right? What a huge WTF it is.

    But here's my real beef: did we run out of real WTFs? Did spaghetti codebases all stop existing yesterday? Project management failures no longer happen? Nobody using Lovecraftian error-handling "strategies"? No? Well if we have some real WTFs out there, how about you write about them instead?

  • djingis1 (unregistered)

    TRWTF is checking if a file exists, before doing something with it.

  • Foo AKA Fooo (unregistered) in reply to Yawn

    Well, the new code also has "interactions with symlinks and directories". Which ones? I bet neither you nor the WTFer even thought about it. Does the code find valid symlinks? Broken symlinks? Does it matter? At least a standard function has well-defined behaviour; this code at best has incidental behaviour.

    Then file name problems, Remy mentions substring matches; even worse joining with spaces. So if you have files "a" and "b" it will claim a file "a b" exists. What if full_path contains regex-special characters? Seems like looking for "a+b.txt" will happily find "aaaab-txt".

    Furthermore, it's inefficient (as mentioned) and misnamed (nothing about "output" in the implementation).

    If you don't think all of this is a WTF, please find another job. Maybe not the biggest WTF ever, but a WTF for sure.

  • Tekay37 (unregistered) in reply to djingis1

    Absolutely. Because if you arrive at a river, you should first try to walk over it. Only if you fall into the river one should try to swim through it. And only if you are eaten by a crocodile you should search for a bridge to cross the river safely.

    That's the pythonic way as it greatly honors where the name python came from.

  • kktkkr (unregistered)

    Not having guards around the filename in that regex search leaves a lot of space for error, especially if that filename can be arbitrary. What if it has a space? What if prefix+filename exists? What if the filename has regex-active characters (like a dot)? How bad can the complexity of this function be, considering that it returns every single match rather than a boolean like it's supposed to?

  • djingis1 (unregistered) in reply to Tekay37

    The issue with files is that your check hits the disk, just like the actual operation would. Hitting the disk can take a long time. And the answer is useless, since by the time you get around to actually reading the file, it may be gone.

    Because of that, you have to handle the failure case of reading a non-existing file. Since you have to handle that case, why bother hitting the disk to check if it exists?

    To bring this back to your river metaphore, you don't have to jump into the river to verify that it's there. You can look at it from a distance. You can also find a bridge before jumping in the river. And unless the bridge is very fragile and prone to collapse, it also won't simply vanish when you're on it.

    None of this has anything to do with python. When doing I/O, you have to deal with the case of acting upon your target when it doesn't exist regardless. Checking for existence is expensive and pointless.

  • Harrow (unregistered)

    The triangles on the bus go bumpitty-bump CRASH,

    Bumpitty-bump CRASH, bumpitty-bump CRASH.

    The triangles on the bus go bumpitty-bump CRASH,

    ...Oh, look, we're here! I guess the wheels don't have to be round after all.

  • BernieTheBernie (unregistered)

    What a destructive fix! The codez does not wok anymore! Because the file was actually named "mypath.dataz" on disk...

  • D-Coder (unregistered) in reply to Harrow

    The triangles on the bus go bumpitty-bump CRASH,

    MythBusters actually tried square wheels on a pickup truck. It was bumpy, less so at higher speed, until (after about 100 yards) one of the bolts holding the wheel on sheared. Ouch.

  • Matt (unregistered) in reply to Yawn

    "And at this point, I say "Screw it, writing my own. It's only ever used once", and then I move on with my life. ... But here's my real beef: did we run out of real WTFs?"

    As long as coders like you exist, no, we will never run out.

  • Randal L. Schwartz (google)

    I've been repeatedly told by Pythonistas how much more perfect it is compared to Perl. Today, I am shocked (Shocked!) to finally find out you can write crappy code in any language.

  • Meir (unregistered)

    You can actually run a vehicle on triangular wheels (https://en.wikipedia.org/wiki/Reuleaux_triangle#Rolling_cylinders) - and for that matter, on square wheels on an appropriate surface (https://en.wikipedia.org/wiki/Square_wheel).

  • Tekay37 (unregistered) in reply to djingis1

    While I totally agree with what you wrote, I am a bit sad, that your answer is not funny. Thus I suspect that you didn't get the joke.

  • Foo AKA Fooo (unregistered) in reply to D-Coder

    See, that's why you use triangles, not squares: One less bump per revolution! So obvious!

  • Zenith (unregistered) in reply to Yawn

    I have to agree here. The issue with way too many developers is that they always defer to a packaged product and don't even think about corner cases. These are two separate but related problems.

    Deferring to a packaged product is passing the buck. You don't know if that developer did something that like whatever you're criticizing. Having preemptively shifted blame for future failures, you then take the operation at face value without proper testing. Eventually, this leads to a third problem, programmers that can't program, because all you're ever doing is plugging together packaged products (or telling users, and even other developers, something is impossible if you can't find one).

    I can think of several places the built-in library does something foolish or useless. Off the top of my head, there's .NET's string trim function. Passing null trims whitespace (default), passing a list of characters trims those characters, and passing an empty list...does the same as null even though what it should do is trim no characters at all. I get a feeling that it stems from so many Win32 functions failing to distinguish between blank and null but come on.

  • Zenith (unregistered)

    (continued, because textbox length and postback validation disagree on maximum length...)

    Then there's IO handling like in the example. The .NET framework has never provided path validation functions. Oh, it'll validate existing paths but not hypothetical ones. So bad paths that can't exist sail through the test and crash on writing. That's if the path doesn't invoke redirection and silently write elsewhere. There's a little more help (not much though) in SHLWAPI but that requires interop. Oh, I could do interop, but I have to get permission from the style police, micromanager, or some other level of interference that has a policy of turning down everything.

    Thus, functions like this are born. Oh, sure, sometimes workarounds are bad. But programming often reminds me of the saying about advanced science being indistinguishable from magic. Sometimes that works in your favor. Other times, ignorance puts you on trial for witchcraft.

  • typo (unregistered)

    3rd-to-last paragraph: "...so we turn the list of files into an array and search it using..." should be "...so we turn the list of files into a string and search it using..."

  • kg (unregistered)

    @Zenith you're really saying it's better to rely on four library functions and produce definitely incorrect behavior (no guards on the regex) than use a known filesystem utility?

  • Mister Monk (unregistered)

    It's not that it's so terrible the programmer wrote his own version of os.path.exists, it's that there are so many things in it that could have been done better.

    • The directory is taken off with the os.path lib, but the filename is not.
    • The weird format thing to copy the path.
    • Getting all the files in the directory with glob rather than a function just to list the files and not using the glob functionality to get the file we're looking for.
    • Putting the files in a giant string instead of just checking if the path was in the array
    • Using re.findall to search for a substring when no regex is used
    • Creating an array instead of just returning True
    • The array would just contain the filepath passed if it actually found anything
  • David (unregistered) in reply to IP Guru

    That's not the Pythonic way, that's the thread-safe way, and it's what you should do in any language. Assuming the file will still exist in the lines following the "exists" check is the real WTF.

  • Pathlib (unregistered)

    ... and once again pathlib gets no love.

    It's even been backported to 2.7 for years. :(

  • (nodebb) in reply to Randal L. Schwartz

    And here's me thinking crappy code was the exclusive reserve of PHP. (obphpbashing)

  • (nodebb) in reply to djingis1

    To bring this back to your river metaphore, you don't have to jump into the river to verify that it's there. You can look at it from a distance.

    A better metaphor would be you arrive at the point where the map days a river should be, but it's a dry river bed, you check, yep it's dry. So you start driving across it. Suddenly a flash flood occurs (it has been raining upstream) and you die.

    Or you arrive at the river and it's full of water, you check, yep. So you get your boat out and start to cross it. Suddenly there's a drought and the water disappears. Because you now have a boat and not a car you are now stuck.

    Computers are so fast both of these are possible.

    Better would be to be using a vehicle that can adapt to driving, floating, diving, flying, etc.

  • (nodebb)

    All of this proves that languages and their libraries are getting VERY complex, and there are very few places where it is all documented. The current python book from O'Reilly is around 3 inches thick and with how the language is moving, is probably obsolete.

    Sure there are libraries, but every new thing that someone invents (or re-invents) generates a new one that MIGHT be similar to what you want. Of course it might even be expanded since you last used the library, and your retained knowledge is obsolete as well.

    This leads to people doing what they "know" and re-inventing the triangle/square/wheel as they deem necessary. It is a never ending cycle of trying to keep up with a moving target. Skip a bit and you find yourself behind.

    Good luck. We all need it.

  • Murray (unregistered) in reply to Zenith

    I assume that you only drive a car you built yourself because buying a pre-made car is passing the buck? After all, there are known examples of cars with flaws.

    "plugging together packaged products" Yes, that's what all programmers do. And all the other people who have jobs. It's this novel concept called "division of labor".

  • (nodebb) in reply to Murray

    It's this novel concept called "division of labor".

    Three things stuck in my memory from actually reading The Wealth of Nations.

    First up: the 67-page diversion on the historical progression of the price of silver, which was less than totally enthralling and of marginal relevance to the surrounding text. Second: his observation that the (essential) function of government in a free market is the regulation of what he called "force", that is, extortion, predatory pricing, and so on. Third, and most relevant here: The amount of time he spent discussing division of labour as it applied to, of all things, making pins. It did suggest to me that he didn't know very much about making pins, as I could think of a thousand better examples of objects that you could make in a divided-labour manner, even in his time.

    Addendum 2018-02-01 06:37: EDIT: I hate this forum software. Those were meant to be separate paragraphs, but apparently them being on different lines is not sufficient clue.

  • Zenith (unregistered) in reply to kg

    No, I explicitly said that while this function may be bad, that doesn't mean there aren't real reasons to go around the built-in libraries.

  • Zenith (unregistered) in reply to Murray

    Your car analogy would work if everybody who drove a pre-built car pretended they were auto workers or vehicle designers. "Programmers" who don't program -and they are legion - still like to call themselves programmers.

  • anonymous (unregistered)

    When I got to the regex part my immediate thought was about file extensions. The "." in a file extension matches any character when it's used in a regex. So for example "bar.c" will match both "bar.c" and "barzc" (and many other names that contain "bar" followed by a single character followed by "c"). This doesn't matter when grepping through a list of filenames to find something but it does matter when writing real code.

  • LzzrdBorth (unregistered) in reply to Zemm

    That's why we need flying cars, dammit!

  • Murray (unregistered) in reply to Zenith

    Auto workers don't build cars from scratch, either. They build cars from parts made by other people, using machines made by other people. According to your logic, they would need to build those machines themselves, and make the parts from metal they mine themselves, using pickaxes they also made themselves.

    "Programmers" who don't program -and they are legion This is known as the True Scotsman fallacy.

  • (nodebb) in reply to TheCPUWizard

    How can this be proven??? Demonstrate that the author did not use such method in another unrelated project, perhaps even at a different company!

    You can only look at the code you can see… but if they're relying on a function called output_exists returning a list, especially from code they don't control, they deserve a world of pain.

  • Zenith (unregistered) in reply to Murray

    And what you're doing is called reductio ad absurdum.

  • foxyshadis (unregistered) in reply to herby

    While Python gets lots of new capabilities all the time, and a reorganization every decade or so, the function in question -- os.path.exists() -- has been around since the very beginning, I believe even in 1.0.

Leave a comment on “The Pythonic Wheel Reinvention”

Log In or post as a guest

Replying to comment #492787:

« Return to Article