The Daily WTF: Curious Perversions in Information Technology

blakeyrat · 2016-01-09 Reply Admin

ben_lubar:
It's actually SIGQUIT.

It's actually SIGWHOGIVESAFUCK

FrostCat · 2016-01-09 Reply Admin

blakeyrat:
It's actually SIGWHOGIVESAFUCK

Judging from the way you keep replying to stuff @Ben_lubar says, you.

David_C · 2016-01-10 Reply Admin

Yazeran:
Actually the worst is that people use , blindly as a separator. ... Especially since some country-settings specify to use , as the decimal point (we do in Denmark) causing all manner of fun when some rows contain floating point values (other places it is used as a 1000 - separator which also can result in hilarity).....

Any system that uses a delimiter character needs to have a system to allow that character as data - by quoting the string or possibly escaping the character. If you have a system for doing this, then any choice of delimiter should work equally well (as long as you don't choose the quote/escape character.) And if you don't then stuff is going to blow up on you sooner or later, because it is never valid to make assumptions like this about your input.

PleegWat:
That's because Ctrl+Letter on a terminal doesn't code for a character by default. It codes for a signal. In the case of Ctrl+\, I believe the signal is SIGABRT, which generates a coredump, though I'm not sure of that or what in general the mapping is between letters and signals.I do know Ctrl+V is escape, so Ctrl+V, Ctrl+\ will input FS.

Not quite. It's all part of your TTY configuration. With the default settings, Ctrl-C sends SIGINT, Ctrl-\ sends SIGQUIT. There are a bunch of others that do other things (cursor motion, escape, other signals, etc.) If your application changes its TTY settings, then different keys will trigger these actions. If it sets its TTY into raw mode, then there is no translation and the keystrokes are simply passed to the application.

Use of Ctrl-V for escape is part of the vi editor and other similar editors. Some shells also support this (as some support Emacs's use of Ctrl-Q for escaping keystrokes that otherwise perform actions) but it's just a matter of what your shell supports, not something inherent to UNIX terminals.

PleegWat · 2016-01-10 Reply Admin

TIL. I do have bash configured in vi mode.

boomzilla · 2016-01-10 Reply Admin

blakeyrat:
It's actually SIGWHOGIVESAFUCK

Don't you have some Windows updates to watch prepare to update?

Tsaukpaetra · 2016-01-11 Reply Admin

David_C:
then any choice of delimiter should work equally well (as long as you don't choose the quote/escape character.)

/me is realizing this more and more as he tinkers with someone else's Expression Evaluator that doesn't quite get it right. Any ideas how the following is supposed to be interpreted? """""" :facepalm:

HardwareGeek · 2016-01-11 Reply Admin

Tsaukpaetra:
Any ideas how the following is supposed to be interpreted? `""""""`

Am I missing something? It's a string containing two " characters.

Outermost pair of "s, it's a string. Inside the string, each "" => literal ". There are two pairs, thus two literal "s.

Tsaukpaetra · 2016-01-11 Reply Admin

HardwareGeek:
Am I missing something? It's a string containing two " characters.

You're not missing anything. Just the program I'm fidgeting with is. It's supposed to be a string with two "s, but it ends up giving back a single " back by the end.

HardwareGeek · 2016-01-11 Reply Admin

Tsaukpaetra:
it ends up giving back a single " back by the end.

De-escaping it twice? If not, I don't think I want to know. :facepalm:

dcon · 2016-01-11 Reply Admin

David_C:
Any system that uses a delimiter character needs to have a system to allow that character as data - by quoting the string or possibly escaping the character. If you have a system for doing this, then any choice of delimiter should work equally well (as long as you don't choose the quote/escape character.) And if you don't then stuff is going to blow up on you sooner or later, because it is never valid to make assumptions like this about your input.

Line 1:"Field : 2":"Field 3":Fld 4 Line 2:f2:disco-wants-to-help as I type:no

Hmm... wonder how many csv "parsers" will fail on that... (I forget - Excel fails one way but not the other - I think it will generate that - but it can't read it)

FrostCat · 2016-01-11 Reply Admin

What? That's a string containing two double-quotes that are, themselves, double-quoted instead of using an escape character. That is, in C, the equivalent string is """".

David_C · 2016-01-11 Reply Admin

Tsaukpaetra:
Any ideas how the following is supposed to be interpreted?"""""" :facepalm:

Ow. I assume it will depend on the language they're modeling.

This is one of those cases where C/UNIX style quoting makes it simple. That expression is just the empty string (well, three quoted empty strings concatenated together.)

I absolutely despise systems where they use "" to parse into a single " within a string (like BASIC). With a model like that, I assume your expression (six double-quotes) would have to parse as a quoted string (strip out the outer-two quotes) containing two double-quote character (each pair parsing as a double-quote character.) Which in C would be expressed as "\"\"" - also ugly, but with an unambiguous interpretation.

When I Googled for this, I ran across even more approaches to this problem, but I still think doubling-up on the quote character is the worst. The [URL=http://en.cppreference.com/w/cpp/language/string_literal]C++11 raw litera[URL] seems useful on the surface but I'm very quickly getting visions of people using this to produce truly horrible unreadable code.

Tsaukpaetra:
the program I'm fidgeting with is. It's supposed to be a string with two "s, but it ends up giving back a single " back by the end

Definitely a bug....

dcon:
`Line 1:"Field : 2":"Field 3":Fld 4 Line 2:f2:disco-wants-to-help as I type:no`
Hmm... wonder how many csv "parsers" will fail on that... (I forget - Excel fails one way but not the other - I think it will generate that - but it can't read it)

I'm not surprised that parsers choke on this sort of thing. Specifying a particular kind of "CSV" is one thing. Coming up with a generic engine that can try and deduce what the delimiter and quote characters are likely to be seems doomed to failure by design.

Tsaukpaetra · 2016-01-11 Reply Admin

FrostCat:
What? That's a string containing two double-quotes that are, themselves, double-quoted instead of using an escape character.That is, in C, the equivalent string is "\"\"".

That's what it's supposed to be, sure, but that particular logic is failing somehow.

Edit: If I can get away with it, I'm trying to figure out how to make it use \ style escaping, and from there allow the other things that a single-line expression can't contain.

FrostCat · 2016-01-11 Reply Admin

[image]

pass or fail? The fields are displayed truncated to 12 characters but the entire string is stored. The embedded newline in field3 appears to have been preserved as a single character, based on the 7, which was is the string length.

Source code, for the masochistic:

def var c as char no-undo extent 4 format "x(12)".

def stream s.
input stream s from "quoted.txt".
repeat:
    import stream s delimiter ":" c.
    display c[1] c[2] c[3] c[4] skip
        length(c[1]) length(c[2]) length(c[3]) length(c[4])
        with font 2 width 80 down.
    down.


end.
input stream s close.

An explanation of non-obvious details is available upon request.

dcon · 2016-01-11 Reply Admin

David_C:
I'm not surprised that parsers choke on this sort of thing. Specifying a particular kind of "CSV" is one thing. Coming up with a generic engine that can try and deduce what the delimiter and quote characters are likely to be seems doomed to failure by design.

I forgot - I usually use ":". If you change that to a comma, then excel (in the US!) does import that properly. (I could have sworn you used to be able to do an import and specify things in Excel - am I just confusing that with Access?)

HardwareGeek · 2016-01-12 Reply Admin

dcon:
you used to be able to do an import and specify things in Excel

Yes, import has a dialog where you can specify fixed-width or delimited, what the delimiter is, um, header row with column names or not, skip columns, and maybe some other stuff I don't remember off the top of my head. Delimiter defaults to tab, not comma, even for CSV files.

dcon · 2016-01-12 Reply Admin

HardwareGeek:
Delimiter defaults to tab, not comma, even for CSV files.

In my quick test, 2016 properly parsed what I previously posted after I did :%s/:/,/g to it.

HardwareGeek · 2016-01-12 Reply Admin

dcon:
2016 properly parsed what I previously posted

It's been a while since I used genuine Excel, but yeah, I think it will parse reasonable CSV format (anything I've tried to give it, anyway) if you open it. If you import it, you get the dialog, which defaults (or used to) to tab.

FrostCat · 2016-01-12 Reply Admin

dcon:
(I could have sworn you used to be able to do an import and specify things in Excel - am I just confusing that with Access?)

You can, but sometimes you have to open the file a particular way. I think it's "do file | Open" rather than double-click.

David_C · 2016-01-12 Reply Admin

dcon:
I forgot - I usually use ":". If you change that to a comma, then excel (in the US!) does import that properly.

If I save that text as a file with a ".csv" extension and double-click it, Excel (2011 for Mac) puts each line of text in A1, A2 and A3, quotes and all. It was probably falling back to the default text-import (tab delimiters).

If I do a File->Import on it, and select CSV, then I get a 3-screen wizard. The first screen is to select if the fields are delimited or fixed-width, the first line to start importing from, and the text encoding of the file.

The second screen lets me select delimiters (tab and comma are selected by defualt). You can also choose whether consecutive delimiters are treated as one, and the quote character (single-quote, double-quote or none). If I select "Other" as a delimiter and type a colon, and select a double-quote text qualifier, it almost works. The newline confuses it and makes it start a new row, even though it's between double-quiotes.

The third screen lets you select formatting for each column imported, including an Advanced button where you can select the decimal and thousands separators.

And yes, you're right, it exports code it can't import. If I create a cell with a newline in it and save it as CSV, I get a newline in the quoted text, but on import, it is interpreted as going to a new row of cells. (But if I open that file, it loads properly!)

Bad Microsoft. No biscuit.

dcon:
In my quick test, 2016 properly parsed what I previously posted after I did: %s/:/,/g to it.

I guess they finally fixed that in the 2016 version.

FrostCat:
sometimes you have to open the file a particular way. I think it's "do file | Open" rather than double-click.

Ditto. You only get the wizard on a file->import. Opening the file reads it using default settings (which puts each line in a single cell when the colon-delimiter is used).

Correct processing of a quoted newline works (when comma-delimited) on an open, but not an import.

And finally, the file-open dialog appears to look at the file type/creator, even though double-clicking doesn't. A CSV that I created in Emacs could be double-clicked and imported, but the open dialog wouldn't show it as a readable file. Changing its type/creator from all-zeros (since Emacs doesn't use these) to TEXT/XCEL (what Excel generates) makes it readable via the open dialog.

HardwareGeek · 2016-01-12 Reply Admin

David_C:
Excel (2011 for Mac) ... Emacs

:doing_it_wrong:

Maciejasjmj · 2016-01-12 Reply Admin

Tsaukpaetra:
That's what it's supposed to be, sure, but that particular logic is failing somehow.

Tsaukpaetra:
Any ideas how the following is supposed to be interpreted?`""""""` :facepalm:

Maybe because you have five quotes instead of six?

Arantor · 2016-01-12 Reply Admin

There. Are. Six. Quotes.

Maciejasjmj · 2016-01-12 Reply Admin

*checks*

*rechecks*

...double quotes are stupid.

Arantor · 2016-01-12 Reply Admin

And here I was expecting a whole 'there are six quotes and we've always been at war with Eurasia.' type comment.

accalia · 2016-01-12 Reply Admin

Arantor:
There. Are. Six. Quotes. [image]

PTFY

urkerab · 2016-01-12 Reply Admin

isthisunique:
Yazeran the worst is when people keep changing delimiters to avoid collisions instead of using an escape sequence.

I'd been happily importing escaped delimited CSV files to SQL Server 2000 using its native import in preparation for migration from an ASCII database. Then along came SQL Server 2005 whose native import does not support escaping delimiters! Fortunately there was one printable character not used within my customer's data set so I could arrange for the source database to export CSV using that as a delimiter.

urkerab · 2016-01-12 Reply Admin

Maciejasjmj:
double quotes are stupid.

Does """""" look like ten apostrophes to you?

FrostCat · 2016-01-12 Reply Admin

urkerab:
Does """""" look like ten apostrophes to you?

Next time you're playing a 4X game, name your ships names that are various combinations of ', ", ,, `, and so on.

YellowOnline · 2016-01-14 Reply Admin

Thank you for saying exactly what I wanted to write.

Also, I can't imagine this script is real. Scripting in PoSh being half of my job, I've seen (and done) some ugly stuff. Nothing reasonable can explain this code though. Like reading a value, writing it to a file to subsequently only read it out again. That doesn't make sense in bash, perl or python either. My guess is that it was a terrible script, but that it has been made worse for shock value.

HardwareGeek · 2016-01-14 Reply Admin

YellowOnline:
it has been made worse for shock value

I wouldn't bet on it.

YellowOnline:
Like reading a value, writing it to a file to subsequently only read it out again. That doesn't make sense in bash, perl or python either.

I've posted before about a script I encountered that parsed some files, wrote the data out as a CSV file, discarded the parsed data, then read the CSV file to get the same data it had just discarded. Fixing this was beyond the scope of the performance issue I was trying to fix, but at least I sped it up by a couple orders of magnitude by eliminating more than 100000 redundant `open()`s on the CSV file that was already open.

After that, this doesn't surprise me.

urkerab · 2016-01-19 Reply Admin

HardwareGeek:
I sped it up by a couple orders of magnitude by eliminating **over 9000** redundant open()s on the CSV file that was already open.

FTFY

HardwareGeek · 2016-01-19 Reply Admin

urkerab:
FTFY

No, the open file was reopened for every field and every field delimiter on each row (then closed at the end of the row and opened again for the next row), so maybe 15 – 20 calls for each of the 10000-ish rows. Definitely at least 8000 rows, at 15 calls per row, would be 120000 `open()`s. Minimum. When I wrote "more than 100000," I meant it.

DCRoss · 2016-01-19 Reply Admin

But even so, it's still OVER 9000!!!!!!

HardwareGeek · 2016-01-19 Reply Admin

Ah, a meme. Google says Dragon Ball Z. I've heard of it, but I have essentially zero interest in anime :belt_onion:, so I didn't recognize the meme.

DCRoss · 2016-01-19 Reply Admin

There you go. You're one of today's over 9,000.

Good Idea, Bad Idea

Leave a comment on “Good Idea, Bad Idea”