• bvs23bkv33 (unregistered)

    hax0r

  • dpm (unregistered)

    If that's all there was to it, why couldn't other applications open it?

  • Anonymous (unregistered)

    @dpm Probably file extensions. If (on windows) you give a file a weird extension (like a text file called notes.dat), opening it by using the GUI requires the user to override a prompt. I suppose that Ben never tried to open the .dat file in a text editor before, thinking it's a binary file.

    Then again that data format is a WTF enough that you could see it and not realize the encoding, since it's not reasonable to store text data this way.

  • (nodebb)

    A valiant effort, sadly let down by a data representation defect of its own...

    The Froglandish word "voilà" is NOT spelled with an upper-case Portuguese A with tilde... (It is, in Froglandish, correctly spelled with with an à, except if it is "voila", the 3rd person singular of "voiler", to conceal something as if with a veil.)

  • No Fun (unregistered)

    And the WTF is...?

  • LCrawford (unregistered)

    Were the spaces actual spaces (hex 20) or just nuls, and therefore the .DAT file was just UTF-16 or Unicode?

  • Pavel (unregistered) in reply to Steve_The_Cynic

    Well, I suppose it was a joke for the older of us, who still remember the fun of the pre-unicode times.

  • Dan Bugglin (google) in reply to LCrawford

    Sounds to me like it was probably just UTF-16 as well. Notepad can handle that just fine.

  • snoofle (unregistered) in reply to Steve_The_Cynic

    It was "voilà" in the original - transcription error?

  • XXXXX (unregistered)

    So what was in the notes? The suspense is killing me... According to TV, there should be a secret message encoded in the notes that let's him know he's been an unwitting pawn in a global conspiracy.

  • Tree (unregistered)

    There are only two file formats worth using, text files and zips of text files.

  • (nodebb)

    Retrieval of archived information is a BIG problem in the industry. I attempt (with moderate success) to keep working hardware/software (tested about once a year) for accessing old information as I am somewhat frequently approached to do so. Just last year someone had a stack of 4mm DAT along with various ZIP and Jazz drives that they needed information from....

  • Yida (unregistered)

    The illustration, by the way, is not of a Visor; it's a Treo 90. I had a Treo 270 which looked similar but was a smartphone. I loved having a real mini physical QWERTY keyboard for mobile texting and emailing.

  • Oliver Jones (google)

    This kind of thing happens in a recurring nightmare of my brother the librarian. What happens when there are no players left to play all those CDs in their collection? His library has actual stored data from a millenium ago. It's stored on paper and parchment, and they handle it very carefully.

    But, hey, there are plenty of programs open any file and do their best to render it.

    The real WTF? A particular company has persuaded a lot of libraries to stop their subscriptions to various journals of record, and use the searchable online service instead they sell instead. That company has two redundant data centers. They are located IN THE SAME RIVER FLOOD PLAIN AS EACH OTHER.

  • Ben S (unregistered) in reply to LCrawford

    They were hex 20 space characters.

  • Little Bobby Tables (unregistered) in reply to Oliver Jones

    Well that's all right, as long as they're not both on the ground floor.

  • (nodebb) in reply to Dan Bugglin

    Sublime Text can open UTF-16 files as well. Sounds to me like the encoding was space-separated hexadecimal digits stored as ASCII.

    Otherwise, using a hex-to-text converter wouldn't make any sense.

    There's probably an xkcd comic to illustrate this.

    https://xkcd.com/257/

  • Brian Boorman (google) in reply to Watson

    Exactly, the file contents probably looked like this in a text editor:

    54 68 65 20 44 61 69 6c 79 20 57 54 46 20 53 75 63 6b 73 21
  • Bishop (unregistered)

    So why didn't he just Google it? http://lmgtfy.com/?q=convert+Datebook

  • ZB (unregistered)

    I find it hard to believe that a mobile device from that era would use such a verbose format to store data. It's basically tripling the storage space required.

  • Simon Clarkstone (unregistered) in reply to ZB

    @ZB: It's not necessarily the format the data was stored on the device, just the format it was stored on the PC.

  • Jabuco (unregistered)

    This reminds me of my efforts of parsing lost stikynote of windows 7 , when I switched to windows 10 and discovered that it yet had no sticky notes app and my notes would be lost.

    I ended up writing a script extracting all notes from the archive which actually worked, just to find a sticky notes app later.

    But it was fun trying to regain my notes back from that silly format.

  • Duke of New York (unregistered)

    Assuming that Ben is a programmer (he uses Sublime Text, after all) TWRTF is that he found this out "one day" rather than immediately. I deal with so many devs who don't understand the concept of "looking at things."

  • Brian (unregistered) in reply to Duke of New York

    Yep, whenever I'm curious about the contents of a file, the first thing I do is open it in a text editor. Sometimes it's the sort of gibberish you'd expect from viewing binary data as text, but surprisingly often there is actual readable info there. Or really not so surprising, given that even a binary-formatted config or data file will usually have text strings interspersed throughout.

  • (nodebb)

    Not sure if TRWTF is the format or waiting 10 years to look at notes that you so diligently wrote every day on a solo bike tour that, one would think, is pretty damned important to a person.

    Also, PDA was a WTF in itself.

  • Bob (unregistered)

    A text file is not a data file? Wtf?

  • vonskippy (unregistered)

    Meh - if you haven't needed to look at those notes in 10+ years, the notes have no value. So who cares if you eventually figured out the mystery code, except for nostalgia, the value of that text is long long long gone.

  • I Am A Robot (unregistered) in reply to Little Bobby Tables

    It doesn't matter, they're both redundant.

  • Brian Boorman (google) in reply to ZB

    I find it hard to believe that you would assume that the data file format on the PDA would be exactly the same as how it's stored on the PC.

  • code_goddess (unregistered) in reply to Brian Boorman

    44 6f 65 73 20 6e 6f 74 21

  • (nodebb)

    The "strings" command in Unix is your friend. It tells lots about a file.

  • eSyr (unregistered)

    Sounds like bullshit. Palm OS stores data in PDB (Palm DataBase) format, and HotSync just copies them over, in addition to fancy syncing to desktop software (microsoft outlook or palm desktop software). PDB format is pretty well-documented and there are a lot of tools to work with it. And, yes, you can use Palm Desktop without an actual device.

  • Lerch98 (unregistered)

    This is what is good about pen and paper. Don't need a computer to read it or #### up the file conversion. Anybody can view it 10 or 50 years later.

  • Friedrice the Great (unregistered)

    If you're planning for a post-apocalyptic world, get your copy of "How Things Work" as traditional books.

  • John Adriaan (unregistered) in reply to Brian

    And that's why I hardly ever use a text editor - I go straight to a binary editor (HxD is my favourite: does physical drives, too!) The patterns that you can see in byte-binary data let you go a long way towards understanding many file formats - and sometimes the Magic Values near the beginning let you identify immediately which tool will read it "natively". I'm just wondering what I would think if I was to look at this data though. The true Hex in the left pane, while the "ASCII Hex" in the right pane, would definitely make my brain do a double-take with a "WTF?"

  • Tim (unregistered) in reply to Lerch98

    I defy anyone to read my handwritten notes, including myself, even 1 day after I wrote it ...

  • RLB (unregistered) in reply to Oliver Jones

    No, TRRWTF in that case is building on a flood plain at all. You don't build anything on a flood plain you don't want flooded. That goes for houses (and, Uinen help us, entire suburbs) as much as for offices, warehouses or data centres. Flood plains are not meant for habitation or industry; they're meant to be flooded.

    Yes, as a Dutchman, this is a point I get @^&&ed off about. You don't build on flood plains, dammit!

  • fsckoff (unregistered) in reply to Pavel

    That's what I assumed too. Pre-unicode days were darker times indeed.

  • Zenith (unregistered)

    I don't think developers these days are smart enough to handle binary formats. Megabytes of layered XML for kilobytes of data is more their speed.

  • Lacerto (unregistered) in reply to Zenith

    Our clients want me to export XML with the proper indentation as they want to read the stuff that is exchanged between their tools... Man, open it in an editor and click pretty-print or something like that. Then they of course tend to edit the files by hand and claim that my importer sucks.

  • RonJeremyfan (unregistered)

    @TheCPUWizard - "re: Just last year someone had a stack of 4mm DAT along with various ZIP and Jazz drives that they needed information from...." Great! You can help me! Someone stole my ZIP drive many years ago, I need to access my important information on my old ZIP disks (my old pr0n collection)...

  • Chakat Firepaw (unregistered) in reply to RLB

    In North America it's sometimes not practical to avoid flood plains. There are flood plains that cover such a large area that avoiding them means having a gap wide enough you would still need to build a hotel in the middle for people crossing it.

  • (nodebb) in reply to Zenith

    I found that out very recently. I have a system pumping 1 million datapoints (4 byte floats) a second for 4 days. I write this as an HDF5 file. Gave it to the people who need to analyze the data. Next day I get an e-mail, MS Excel won't open the file. I reply that opening a 1.3 TB file in excel is impossible and pointed them to matlab and/or pyhton. Next day I get an e-mail, matlab or python is to hard, could I export it to CSV, then they could import it into excel. Not willing to fight the tide I asked which precision they needed, and a place to store the file. I got access to an server and they wanted at least nine after the comma. Their server gave up on the file; somewhere at the 20 TB mark ;) There is now a great discussion going on who is responsible for analyzing the data.

  • (nodebb) in reply to Dlareg

    Nice! Experience is often a more effective teacher than logical argument.

    Even current Excel will only do just over a million rows before giving up, so unless they wanted a separate worksheet for every second of data (and what a fun structure that would be to work with!), they were going to have trouble anyway, even before you consider performance. Of course, you know this.

    I do find it somewhat surprising that people who had a need to work with that amount of data would (a) apparently not know how to use anything capable of it, and (b) think that Excel would be fine. It doesn't take that much experience with Excel to find out where its limits are. Heck, I recently had to run a reconciliation between two half-million record sets and managed to lock Excel up irretrievably two or three times in the process.

  • (nodebb) in reply to Scarlet_Manuka

    This project just got better and better.   Initially I advised doing the data analysis on the device it self. Had a working solution. This meant that instead of 4Mbyte/sec they would receive 0.6kbyte/sec. But ofcourse they needed the RAW data.   So I supplied the 32 bit (4 byte) ADC output directly. But they would not be floats so:"less accurate."I tried to explain that you actually lose accuracy converting from 32bit integer to 16bit+ 16bit floats but that would not stick because:"fractions are always more accurate as integers". So I got overruled.   Now this... it is getting better and better.

  • Really (unregistered)

    .. and then Ben S told this story 800 million times to all his friends and apparently one of them was impressed enough to submit this simple nonwtf to thedailywtf and they were slow so they posted it...

  • Fernando (unregistered) in reply to Bishop

    "Just googling it" produces results for the Palm datebook, not Visor datebook, which, as the article points out, is incompatible.

  • A Non Mouse (unregistered)

    So this story happened 10 years after he got rid of the Windows PC he had until 2010? Data conversion sure sucks in the future.

  • b.a. freeman (unregistered) in reply to Brian Boorman

    actually, Brian, Ben is probably just a windows guy who took a long time to learn what a unix guy learns early on: when a datafile doesn't work, look inside the file to examine the data for yourself. when U use a GUI, U often end up depending too much on its preprocessing. use 'file' to see what it thinks the format is, use 'strings' to see if there are any strings, etc. so no, 54 68 65 20 44 61 69 6C 79 20 57 54 46 20 69 73 20 2A 47 52 45 41 54 2A 21 21

  • A.K. (unregistered)

    Fresh out the oven - porting a site from a "beyond broken" Wordpress install to a clean install, which required salvaging what worked, and re-creating the bits that didn't.

    Theme - Export Options - get a JSON. Cool. Switch to 2nd site. Theme - Import Options - try to upload a JSON. "Sorry, this file type is not permitted for security reasons."

    Really. Same platform. Same release. Same theme. Same server. You can HAVE the settings, but you can't actually USE them.

    Yeah, I got around it, but REALLY. The JSON file was ~30 lines of relatively simple variables that COULD HAVE BEEN DONE AS TEXT WHY THE **** DIDN'T THEY.

Leave a comment on “The Proprietary Format”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article