- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Another steel mill story? wow... and my senior project was related to this (the steel mill stuff, not the signature handling XMLAOTD (XML Abuse Of The Day)).
Note: my senior project mostly caught fire. Thermal runaway in motor drivers is my worst nightmare.
Admin
XML meta: ~50 chars payload: 2 to 3 chars 6% payload;
The real WTF is that they didn't use JSON!
Ah but seriously, what's with the two layers of tags? Someone was itching to use this new-fangled XML stuff.
Admin
Admin
It took me a second to realize that didn't say gargantuan ladies.
Must remember to have coffee before reading.
Admin
Ummm, that's not X Y data. At best it might be the value of each consecutive pixel, with the X Y coordinates implied. But, you said it wasn't a color image.
I call BS. Or confusion, which is close enough to the same thing.
Admin
I would've expected <signature> <pixel> <x>3</x> <y>53</y> </pixel> ... </signature>
I don't even understand how they use this stuff or what crazy thoughtprocess made this come into existence.
(Yeah I do understand that using xml for this is a total wtf)
Admin
the Real WTF is that they didn't use Ice (ZeroC) or it's crappier ancestor CORBA.
but JSON would have been ok was well...
for an ICE payload it would be very little overhead + the binary size of the array. overhead is fixed. O+N*sizeof(typeof(N))
for a JSON payload [assuming 2 byte average per datapoint] 2 + 2N + N where 2N characters of that are payload and 2+N are overhead. if you don't pack it with spaces (Which are unneeded since it is generated data anyway)
Admin
you can address an X-Y space with consecutive numbers if you map it correctly
N = the value given Sx = Width Sy = Height Origin = topleft.
X = int(N / Sx) Y = N % Sx
IIRC.. i tried to remember that real fast.. might be in error.
Admin
Geez, whatta XMLn00b.
The best, most clearly-normalized format would have to be <signature> <pixel x=0 y=0> 0 </pixel> <pixel x=1 y=0> 0 </pixel> <pixel x=2 y=0> 1 </pixel> <pixel x=3 y=0> 0 </pixel> ... </signature>
Really.
The basic rule of XML is "XML is like violence. If it isn't solving your problem yet, you're not using enough of it."
Admin
No, no, no, if you want to waste bytes, put your X Y data in VRML!
Admin
I mis-read "gargantuan ladles" in the first paragraph as "gargantuan ladies". Then got all excited when it started talking about the "hot strip mill"...
Creating a hostile workplace environment aside, this story is an excellent example of why XML is not always your friend.
Admin
Compare that to 50 bytes per block times 1000 such blocks for 50kB of data, and your actual payload is .3%.
Even with full 32-bit color, they could have stored it with 4 bytes per block. The size of the XML at that point, though...
(Also, had they used JSON, they could have serialized the whole thing as "[127,43,54,...]" and gotten a ~75% payload.)
Admin
Admin
So XML is "uncompressed bitmap data" now?
TRWTF is the contradictions in this story, which can be summarized as "Past developer used XML badly." You're welcome, Daily WTF. I just saved you 20KB in "uncompressed bitmap data" (or "characters" if we're being all LOGICAL).
Admin
Admin
Presumably the thinking was this:
"Signatures...Hmmmm...Well, in handwriting analysis, you don't just look at the picture, you actually look and see whether they wrote it in the right order (like they started from the top of the letter vs from the bottom). It's true, I saw it on CSI last night. We'd better store each individual pixel they go over, in order, so that we can check that it's not a forgery."
Never mind that the requirement was just to store the images. And they would never need to check for forgeries. And they didn't have an algorithm to detect forgeries anyway.
On that note, the real wtf is that they didn't store pressure data which is effective for detecting forgeries!
Admin
"And not the "everyday" things like the gargantuan ladles that carry hundreds of tons of molten steel."
I read this as "gargantuan ladies"
Admin
I think a lot of us in the wee hours of the morning misread that as "ladies" Me included.
Admin
No, I'm guessing it's more likey the original dev was just plain stupid.
Honestly, binary data is not exactly black magic...
And if you absolutely had to send it via a webservice, wouldn't a binary protocol like Hessian make this trivial?
Admin
Admin
And, rereading the article, I see that the signatures are actually 300x100. So 30000 bits, 3750 bytes. Thus, this approach is pretty comparable to recording just the black bits in a sparse array. Given a decent packed encoding (number the pixels consecutively, so you can store them in 2 bytes), and 1000 black pixels, you've got the entire thing stored in 2kB.
Intelligent choice of data structures ftw!
Admin
Admin
Fahrenheit? Speak SI, dammit!
Admin
Still fun to visualize, though... that's the true essence of a WTF!
Admin
Admin
When I first read that the 300x100 black and white image took up 290kb, I said to myself in my head "Haha, what, are they storing it in XML or something?"
I'm not laughing any more.
Admin
2000 degrees Fahrenheit is approximately "a billion degrees" in Celsius and "a billion - 273" Kelvin.
Admin
If billion === thousand, and 273 == -273.
Oh god, 273 == -273. The universe goes boom.
Admin
The thing is, you could just as easily do: <signature> <pixel> 3,53 </pixel> ... </signature>
...And cut your xml in half. To be even crazier, you could remove the xml entirely and just do a big comma separated list: x,y,x,y,x,y,x,y,etc.
Really though, you should just offload the work to the data input device and get it to send you a fricking jpeg. This isn't 1981.
Admin
Ah, XML. The cause of, and solution to, all of life's problems.
Admin
I've dealt with too much COBOL in my time to try and optimize the data storage for size in all but the most extreme circumstances. Size and Cycles are cheap; the data structure needs to be optimized for speed and reliability.
Not that this does that either. But the size isn't what bothers me about it.
Admin
Not that they did this the right way at all, but remember that they were trying to capture signatures here. Not just images of signatures. Things like the direction a letter is drawn might be checked if you have the consecutive point data rather than a monochrome bitmap.
Admin
So, you're saying, if I notice my developers working out calculus functions on a white board while trying to figure out how to store a signature bitmap...I should be concerned?
Sometimes, I like to insert this type of problem solving technique into other professions in my mind, such as a doctor!
"Sure, we could remove the splinter in your index finger with tweezers, but I just got all this fancy new laproscopic equipment and I've been dying to try it out! Let's try getting at it through the belly button!"
Admin
or it (the bit shown int he article) is a snippet of a sparse array
Admin
Whenever you hear or read something like this, you know that things are going to get very, very ugly.
Admin
Ok, firstly: a lossy image format for signature data? You're nuts, friend.
Secondly, the web service probably required XML input (that's how these requirements go: "thou shalt use SOAP and XML for the webservice"), so "just sending an image" wouldn't be a great solution.
There's also the need to think ahead about future requirements that might change the image size or require color (stupider regulations have been passed).
So here's what I would do:
A 300x100 monochrome image is just an array of 30000 bits. I would have packed this into 3750 bytes, then Base64 encoded the whole thing, and slipped it into the webservice payload thus:
<signature width="300" height="100" color="no"> <image encoding="base64">....encdoded data here...</image> </signature>Admin
Admin
In a similar vein I worked on a job where we wanted to punt around large amounts of numerical data from a Hot Strip Mill setup program to a logging system.
We proposed an ASN.1 solution that would minimise the data volume and was easily extensible if you wanted to add more data to the log.
But did they go with that?? Nope .. they ended up with an XML solution which multiplied the dataset size by a huge amount and if you wanted to change the dataset you had to edit and recompile the setup program.
Admin
Doesn't seem like a bad idea, very enterprisy in my opinion. At least the won't have a hard time comparing signatures... oh wait!
Admin
Why do we need to pass it to the server if it's base64?
Admin
JPEG doesn't have to be lossy, does it? I thought at 100% in certain encoders it is lossless.
At any rate, just stick the damn thing in as a PNG or a GIF or something. I don't think JPEG is very efficient in monochrome (although I could be wrong).
Admin
Admin
That makes perfect sense iff 273 == i
Admin
You're not wrong. JPEG is really freakin' awful for monochrome images. It's also pretty bad at any line-art like a cartoon or logo, color or mono. JPEG compresses smooth color gradients, not sharp transitions. Encoding a line drawing as JPEG gives it all sorts of really nasty compression artifacts.
JPEG would be the worst possible image format for a monochrome signature. With the possible exception of XML, of course.
Admin
http://www.google.com.au/search?q=2000f+in+c
But the second part is "a billion - 273" as in 999999727 "another anon Kelvin".
Admin
XML is a virus.
Admin
This sounds like the app I worked on at my previous job. We bought a company that made remote inspection software, and it was full of crap like this. It sounds exactly like Echo by Infowave (previously Telispark). The guy who wrote the original did some crazy stuff like storing strokes of the signature instead of storing the binary bitmap.
I would not at all be surprised if this was Telispark...
Admin
Thats no fun - I would have converted it to ascii art.
Admin
Admin
It must be Friday. I read the second sentence as 'And not the "everyday" things like the gargantuan ladies that carry hundreds of tons of molten steel.'