• (cs)

    51st!

  • Jay (unregistered)

    I'm still trying to figure out why the hash is a big deal, or why he even needed a hash. Let's accept the premise that the bottleneck is creating the TIFFs. I don't know what criteria go in to determining a price at each store for this chain or just what goes on their price sign -- I'd assume there must be a product description of some sort and a price, maybe there are some other codes or marketing text or whatever. But I would think the obvious logical way to do this is to create a record in the database for each possible sign. Let's suppose the natural key is sku + region + fudgefactor + otherfudgefactor -- where the fudge factor's account for any differences in pricing or whatever coding you might have between stores. Then you generate the TIFF for each sign, and either stuff it in the record as a BLOB or write it as a separate disk file with a sequence number for the file name and you stuff the sequence number in the record. If none of the natural key data has changed, you don't regenerate the TIFF. Then when you figure out what TIFFs to send to each store, you just read the record and pull the already-created TIFF.

    As another poster pointed out, if you really want to take the burden off the servers, don't generate the TIFFs on the server at all. Just send parameter data to the store, and have an application that generates the sign and prints it on the store's computer. That not only distributes the work to create the TIFFs but also eliminates a lot of bandwidth. Howerver, it does create a potential configuration-maintanance headache: Every time the sign-generation application is updated, you have to push new versions out to all the stores. I'd vote for making it an applet, but whatever.

  • Ryan Barrett (unregistered)

    Why not just throw the sign data into an database, then delivery the signs via Jasper Report or SQL Server reporting services? Or use the PDF generators and throw those to the stores.

    You could probably do everything on the DB server in stored procedures.

    Heck, you could even export everything to Word via mail-merge and send that to the stores.

  • Jay (unregistered) in reply to JoeCoder
    JoeCoder:
    Shial:
    yes, but would you want to poke around in some b@#@#@ spawn of codethulu or just suggest they sacrifice some more hardware to it while you run away?

    umm, because you are supposed to be a professional?

    And if you're not going to maintain software, what, exactly, are they paying you for? If I was the boss and every time I asked an employee to do a job he replied that he had studied the situation and concluded that it wasn't worth doing, I think I'd pretty quickly conclude that if he's right, than we don't him and we should abolish the position and lay him off; and if he's wrong, then we should get someone else to do the work and again, lay him off. Either way, we lay him off.

  • And now I are one! (unregistered) in reply to JoeCoder

    "Being professional" does not mean "protecting management from the consequences of their decisions" or even "doing what I want anyway even after management said no." Those are called "being dysfunctional" instead. There are risks, paybacks, and costs involved with any course of action in this scenario; the role of the engineer is to estimate them, and the role of management is to choose among them. "Shiny" attracts engineers, "cost effective" and "low risk" attract managers, and these are not the same. Throwing $50,000 worth of servers at a problem in exchange for three years of not having to worry about it is certainly more management attention unit-effective and quite possibly more cost-effective than throwing $10,000 worth of engineer at a problem just to size it.

  • Anon (unregistered) in reply to snoofle
    snoofle:
    Since most stores don't put the name of the widget on the price sign

    Have you not be to a store for a while? I think most stores put at least the name and sku on the sign (in addition to price, of course), otherwise how can a customer confirm that the price on the shelf is actually the price for the item they just took off that same shelf? Stuff tends to get moved around by customers.

  • You know, I don't (unregistered) in reply to RayMarron
    RayMarron:
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Yeah, you can always tell when you're in the thick of it.

    then, when all was quiet, two elephants (followed by a symbol) fell off a cliff...

    ba-dum tsh

  • You know, I don't (unregistered) in reply to JoeCoder
    JoeCoder:
    Shial:
    yes, but would you want to poke around in some b@#@#@ spawn of codethulu or just suggest they sacrifice some more hardware to it while you run away?

    umm, because you are supposed to be a professional?

    nice understanding of grammer there, my friend

  • You know, I don't (unregistered) in reply to You know, I don't
    You know:
    JoeCoder:
    Shial:
    yes, but would you want to poke around in some b@#@#@ spawn of codethulu or just suggest they sacrifice some more hardware to it while you run away?

    umm, because you are supposed to be a professional?

    nice understanding of grammer there, my friend

    grammar*

  • Steve (unregistered)

    Viscous: mere typo, or brillantly subtle pun referring to the application's performance?

  • (cs) in reply to You know, I don't
    You know:
    RayMarron:
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Yeah, you can always tell when you're in the thick of it.

    then, when all was quiet, two elephants (followed by a symbol) fell off a cliff...

    ba-dum tsh

    That "tsh" y'all heard was a symbol crash.

  • (cs) in reply to Davy
    Davy:
    Hey Scott. Do some work. "Weighed the effort of recommending throwing more hardware at the problem against actually doing any work at all", more like. You had an opportunity to seriously improve the performance right there, and you just didn't bother. Poor show.
    Not at all.

    New hardware was on the cards anyway. The cost of making it significantly more capable to hide the original problem probably undercut the cost of improving the code.

    That bit where he "weighed the effort" of more power against optimization is called a "cost benefit analysis". That is not a WTF.

    It takes a big man to say "no".

  • (cs) in reply to Code Dependent
    Code Dependent:
    You know:
    RayMarron:
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Yeah, you can always tell when you're in the thick of it.

    then, when all was quiet, two elephants (followed by a symbol) fell off a cliff...

    ba-dum tsh

    That "tsh" y'all heard was a symbol crash.
    It's spelt "tcsh".

  • (cs) in reply to JoeCoder
    JoeCoder:
    Shial:
    yes, but would you want to poke around in some b@#@#@ spawn of codethulu or just suggest they sacrifice some more hardware to it while you run away?

    umm, because you are supposed to be a professional?

    So a professional always eschews hardware upgrades in favour of throwing more money at the problem in terms of development time? Is that your point?
  • some guy (unregistered)

    why bother generating the tiff in the first place. a little php could whip together a sign through a quick html template, include the price (font or pre-gen images), get the sku and a generated barcode (which shouldnt be changing all that often if at all) and pass it out to client machines at the stations which then run a batch print job.

    Of course thats assuming the tiff generation is where the problem is. heck the whole app could probably be rewritten in php, couldnt be worse than vb5.

  • Altimeter (unregistered) in reply to rfsmit
    rfsmit:
    Code Dependent:
    You know:
    RayMarron:
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Yeah, you can always tell when you're in the thick of it.

    then, when all was quiet, two elephants (followed by a symbol) fell off a cliff...

    ba-dum tsh

    That "tsh" y'all heard was a symbol crash.
    It's spelt "tcsh".

    It's also spelled cymbal, unless that sound was ™ or © following the elephants.

  • (cs) in reply to Altimeter
    Altimeter:
    rfsmit:
    Code Dependent:
    You know:
    RayMarron:
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Yeah, you can always tell when you're in the thick of it.

    then, when all was quiet, two elephants (followed by a symbol) fell off a cliff...

    ba-dum tsh

    That "tsh" y'all heard was a symbol crash.
    It's spelt "tcsh".

    It's also spelled cymbal, unless that sound was ™ or © following the elephants.

    Not in this case. The symbol followed the elephants off the cliff. It crashed when it hit bottom.

  • (cs)
    Managers at their twenty-eight locations absolutely hated having to write out, by hand and with a big marker, the hundreds of new price signs that came through every morning.
    I worked my way through college as a stocker, not at a little convenience, store but at a huge grocery store that carried tens of thousands of items. The regular price of an item is printed on a small tag that clips on the shelf edge; it changes about once a year and so needs printing about that often. Sale items change on a weekly, not daily, basis. For most sale items, a sale price tag is printed which overlays the regular tag for the duration of the sale. These get machine-printed, just like the originals. About the only thing that gets a large sign hand-lettered with a big marker is a display created especially for the sale, or something that has room for the sign, like ice cream (the sign goes on the door). There were usually about 20 signs like that per sale. 20 hand-lettered signs a week isn't really a big deal. And remember, this is a grocery store, about 20 times the size of a convenience store.

    There must be a great deal of obfuscation in the telling of this story.

  • Dirge (unregistered)

    Seriously. The fastest quick fix would probably be to generate the hash based on e.g. the SKU with the date and/or price appended instead of the TIFF. The next fastest would be to stop using a hash altogether and reference the TIFFs by that same type of criteria. The first should be easily accomplished by any non-inept developer at a cost of far less than even one replacement server. If you're a developer who is too lazy to fix this huge problem, maybe you should look into a different career.

    The long-term ideal approach should be what others have suggested - stop using TIFFs. Send PDFs, or HTML files or something. That would be expensive and should involve moving the app to a modern language, but it needs to be done at some point to ensure that the app can continue to run.

  • Dave (unregistered)

    This WTF just clarified, in my mind, why the human race will be extinct in 100 years. The cost of the hardware does not include the cost of the electricity, pollution, packaging and cooling required to print these signs. I wonder if you added up all the true costs.. Do these signs even produce a profit?

  • (cs) in reply to Dave
    Dave:
    Do these signs even produce a profit?
    An experience I had suggests not. When I was a stocker (see related post above) I had just made and hung a large sign advertising frozen strawberries for 39 cents, when a customer rolled her shopping basket up to me and said, "Can you tell me where to find those strawberries on sale?"

    I couldn't resist. I turned and pointed at the sign, which was on the glass door right beside me, and said, "Yes ma'am, they're right there behind that sign that says, "Strawberries on Sale".

  • (cs)

    Special Sign: Hash, $1.99 / pound

  • TopCoder (unregistered)

    Just because you do not like VB5 or any other variant of VB does not constitute a WTF. Anything written in Perl,BASH or macros in Access can be considered ES; and I abhor Access for anything other than a desktop database.

    I see a WTF here in the method chose to do the generation of TIFF's. I find the posters comment on buying a new MAC card and then smashing it as the best method offered.

  • sup yo (unregistered) in reply to Code Dependent
    Code Dependent:
    Managers at their twenty-eight locations absolutely hated having to write out, by hand and with a big marker, the hundreds of new price signs that came through every morning.
    I worked my way through college as a stocker, not at a little convenience, store but at a huge grocery store that carried tens of thousands of items. The regular price of an item is printed on a small tag that clips on the shelf edge; it changes about once a year and so needs printing about that often. Sale items change on a weekly, not daily, basis. For most sale items, a sale price tag is printed which overlays the regular tag for the duration of the sale. These get machine-printed, just like the originals. About the only thing that gets a large sign hand-lettered with a big marker is a display created especially for the sale, or something that has room for the sign, like ice cream (the sign goes on the door). There were usually about 20 signs like that per sale. 20 hand-lettered signs a week isn't really a big deal. And remember, this is a grocery store, about 20 times the size of a convenience store.

    There must be a great deal of obfuscation in the telling of this story.

    work in tesco. price change every day. long time, yo.

  • Fedaykin (unregistered)

    this is a very minor fix....

    You don't have to wait for the fuggin' image to be created, you just have to compare the parameters (product, price, location, etc.) used to generate the image..

    One table which stores the parameters of each unique sign (and a key), and another table to associate images with entries in that particular table via the key. Hell, you could even generate a hash of the parameters and use that as the key if that saved more changes.

    Granted, if the system was utter shite it might take a while, but my god it's not like this would be a fundamental change to the system.

  • (cs) in reply to Charles400
    Charles400:
    Special Sign: Hash, $1.99 / pound
    Actual signs made by a store manager:

    Hot Lays .99

    Ass. Jelly $3.49

  • ben (unregistered)

    Scott is an idiot.

    That is all.

  • (cs) in reply to some guy
    some guy:
    why bother generating the tiff in the first place. a little php could whip together a sign through a quick html template, include the price (font or pre-gen images), get the sku and a generated barcode (which shouldnt be changing all that often if at all) and pass it out to client machines at the stations which then run a batch print job.

    Of course thats assuming the tiff generation is where the problem is. heck the whole app could probably be rewritten in php, couldnt be worse than vb5.

    As they say, "You can write something worse than VB5 in any language" ... even PHP, believe it or not. Conventionally, it's done in VB.Net, but I've seen it done in C++ and I'm sure it could be done in Scala with minmal effort.

    I love all these "whip up an applet in Blub, abstractify the SKU-fu, wrap gently with HTML and retrain all 2000 managers in the company to tick hundreds of boxes on a restful app, then run over to the printer to make sure it's in Postscript mode" suggestions. These people should never be let anywhere near existing production code.

    Don't get me wrong, the original system sounds horrendous -- as someone pointed out, it's pretty much bound to be CodeThulhu. Which is a good reason to run away from it. I don't know what the lease term might be, but I'd assume from standard amortization practice that it's around three years. You can throw an awful lot of three-years-later value at the problem if you just concentrate on the hardware, which is what these managers are used to hearing anyway.

    I hate myself for saying this, but that is almost certainly the correct decision in this case. Perhaps enough Big Iron will crush the thing like the giant bug it is.

    On the other hand...

    On the other hand, there is such a thing as scalability. I don't really care how dreadful the original app is: it works. That is All Ye Know, and All Ye Need To Know. I do care about the scalability issue. The thought of "just throwing hardware" at the problem when you start with twenty-odd servers, 2000 stores (with known requirements), and a big bag o' 400 new stores (with new requirements) coming on line brings me out in a cold sweat.

    The great thing about enterprise scalability is that it doesn't. Scale, I mean. With that much hardware in the system, you can define the current trend in requirements as much as you want, and the whole thing will collapse when you add something as simple as a single network connection.

    I'd model the heck out of it before I let 400 new stores come anywhere near my prized mission-critical VB5 company jewel...

  • iMalc (unregistered) in reply to SomeCoder
    SomeCoder:
    At first glance, I agree with the rest of the posts about this. He should have just done the work to generate a hash from the params, create the tif, wrap an if around it and call it a day.

    However, being that this is probably a big ball of mud of VB5 code.... I too would be tempted to either say "Rewrite the damn thing" or "Throw more servers at the damn thing".

    Having dealt with some stuff like this before, I can understand why he just chose the hardware route shrug

    Sure, one could take the lazy route, but some day someone will come along that CAN be bothered working with old VB5 code, and will increase the performance by a factor of 50 or so (hashing the data that goes INTO the tiff creation). Then when suddenly one PC is enough, and the customer is overjoyed, who is going to look a bit stupid for the increased server maintenance and power costs that would have would probably paid for the time spent optimising it since then? Throwing more hardware at it is the cowards approach. Sheesh, it's only a couple of years back that I dirtied my hands with VB3 on a legacy program!

  • (cs) in reply to ObiWayneKenobi
    ObiWayneKenobi:
    Shial:
    The answer to why he did it in this manner is at the end of the article:
    enterprise-level VB5 code.

    That's an oxymoron, isn't it?

    I don't know, but it certainly does peg the idiot-o-meter.

  • Anon (unregistered) in reply to iMalc

    [quote user="iMalc"][quote user="SomeCoder"] Sheesh, it's only a couple of years back that I dirtied my hands with VB3 on a legacy program![/quote]

    Yuck! You'll never get the stink of VB3 off of you now!

  • Capt. Obvious (unregistered) in reply to Shial
    Shial:
    yes, but would you want to poke around in some b@#@#@ spawn of codethulu or just suggest they sacrifice some more hardware to it while you run away?
    Salaried, sacrifice hardware. By the job, well, what's 90% of the difference in server costs?
  • (cs) in reply to my name is missing

    If the hardware cost is $4000, you owe approximately $1000.

    New hardware $4000

    Your pay: $50/hr x 10 hours per day x 5 days per week x 2 weeks = $5000

    This doesn't take into account any lost revenue from delays in the production of profit-generating code.

    Sometimes... It actually is more cost-effective to "throw hardware at the problem"

    In this case, a $1 fix of removing the hash generation will probably increase performance to an extent that they will get more life from the servers.

  • (cs)

    I feel so sorry for the guy... how he must feel after "pouring" through the code...

  • crap crap crap (unregistered)

    Hey, has anyone noticed that he could've just hashed the parameters that went into the creation of the tiff, instead of the tiff file itself? you know, things like the SKU, the price, maybe the date and the store or region.

    Seems like that would be a pretty simple fix just to get back to aptent.

  • (cs)

    Wow, it was running on "21st century hardware?" My god, that's positively ancient!!! Waittaminute...what century is this again?

    And ya gotta hate those "viscous circles" - they're really slippery! :-)

  • john (unregistered)

    Uh.

    You could hash THE DATA used to generate the TIFF, and use THAT as an index, avoiding the whole TIFF-generation process.

    This is a stopgap. I would then rewrite the system from scratch in C. If there were no really-freaking-awesome Oracle engineers around, or if everyone currently employed and involved could give a nod that, yes, they're a little shaky about Oracle and kind of hobble through it, I'd migrate the whole thing to MySQL (i.e. something I know at least, and that's easier to hobble through).

    Finally I'd do a parallel deploy, slowly deploying it out to sites next to the existing system, and just testing with test signs, then real signs, then do all work completely on the new (tested!) system and remove capability at each site one at a time to use the old system. Once they're all gone, the support for the legacy system (hardware) gets pulled; along the way, it gets trimmed, so its operating cost decreases.

    Alternately, I could have a Web based interface (PHP) to do this from, which renders the signs to LaTeX, pdflatex, and e-mails a URL to the PDF to each remote site (or otherwise triggers something).

    Alternately, each site could check a URL each hour. Signs would be downloaded, with meta-data telling when they were to be issued; the remote site would decide when to print them, automatically. Yes, deploying scheduled data for automated time-managed batch printing.

    There are a billion non-invasive ways to handle this, and better ways to do this.

  • Wedge (unregistered)

    This is dumb. Even before you have generated the image you have all of the unique information for the image, just create a hash using that. You could even keep a boolean flag which kept track of whether the image had been generated already or not.

  • Franz Kafka (unregistered) in reply to smbarbour
    smbarbour:
    If the hardware cost is $4000, you owe approximately $1000.

    New hardware $4000

    Your pay: $50/hr x 10 hours per day x 5 days per week x 2 weeks = $5000

    This doesn't take into account any lost revenue from delays in the production of profit-generating code.

    Sometimes... It actually is more cost-effective to "throw hardware at the problem"

    You left out the cost of operating the server - the rule of thumb I've heard is that 3 years of operation costs as much as the server, so your ROI for the specific problem is 1 year, with better scaling for the next time you need more cap.

  • Franz Kafka (unregistered) in reply to And now I are one!
    And now I are one!:
    "Being professional" does not mean "protecting management from the consequences of their decisions" or even "doing what I want anyway even after management said no." Those are called "being dysfunctional" instead. There are risks, paybacks, and costs involved with any course of action in this scenario; the role of the engineer is to estimate them, and the role of management is to choose among them. "Shiny" attracts engineers, "cost effective" and "low risk" attract managers, and these are not the same. Throwing $50,000 worth of servers at a problem in exchange for three years of not having to worry about it is certainly more management attention unit-effective and quite possibly more cost-effective than throwing $10,000 worth of engineer at a problem just to size it.

    Funny, my experience is that it's the CTO with a industry rag that goes for shiny, while the engineers just want it to work and not take all their time.

  • rc (unregistered)

    nice pun at the end. Viscous cycle. heh.

    Course it wasn't intended, was it :P

  • (cs)

    trwtf is "the viscous cycle"

  • (cs)

    Viscous cycle? Well whoever designed this was certainly thick, but I doubt they were a fluid.

    Anyway the obvious solution is to just use those signs where each digit is a separate sheet, so you just stick the appropriate numbers in where the sign goes.

  • Casey (unregistered)

    Wait a second. I think the original submitter is just fishing for free solutions.

  • Another Gary Williams (unregistered) in reply to OldPeter
    OldPeter:
    So why send TIFFs at all? Why not send just the small codes to the stores and leave it to some client application there to render the final signs.

    Why bother with tifs, Our apps sends pure ASCII text to Zebras Label printers. The Zebras generate all the image code internally and then print. I assume his printers are similarly capable.

  • (cs) in reply to crap crap crap
    crap crap crap:
    Hey, has anyone noticed that he could've just hashed the parameters that went into the creation of the tiff, instead of the tiff file itself? you know, things like the SKU, the price, maybe the date and the store or region.

    Seems like that would be a pretty simple fix just to get back to aptent.

    Oh, you are so clever. Yet it seems that you never had to deal with a poorly-designed System from Hell.

    Sure, it would be a trivial change to make if the code went like:

    SKU = getSKU(); Date = getDate(); ... Image = createTIFF(SKU, Date, ...);

    But it's far more likely that input parameters were scattered all over the generation routine, like:

    drawHeader();

    if(SomeObscureCondition) { drawSomeMoreStuff(); drawPriceInDollars(round(getPrice())); drawEvenMoreStuff(); drawPriceInCents(); ... }

    And all of this - naturally! - would be stuffed in a single N*10000-line procedure, pixel output and data fetching all mixed up and dependent on each other.

    Not so simple now, is it, Mr Smart Guy?

  • Hamstray (unregistered)

    Chances are high that the 400 store competitor they planned on acquiring has their own system which is not so fucked up. Why not just plan on adopting that one instead?

  • Wolfgang (unregistered)

    Just hash the input data using two different hash functions and store both hashes, Two hashes, to reduce the likeliness of a collision minimal.

    Another thing to improve: Why generate the TIFFs locally? You could as well send out just the datasets and let the computers in the store do the hard work. This lifts a lot of load of the main servers and saves bandwidth.

    Next step: Generate PostScript instead of TIFF. Most good Laserprinters happily take PostScript data for input (the newest models even PDF).

  • hangfire (unregistered) in reply to Anon
    Anon:
    WTF indeed. He could possibly imagine creating a hash from the row that produced the TIFF before the TIFF was created and using that. Presumably the same data going into the TIFF generator produces the same result, so just hash the data going in.

    Damn, after I wrote up this post, I hashed it and discovered you'd made a post saying exactly the same thing

  • IByte (unregistered) in reply to Code Dependent
    Code Dependent:
    Gotta watch out for those viscous cycles. They can get sticky.
    Anatomy of a viscous cycle: arcane system runs slow as molasses, making "fixing" it by sweeping problems under the rug an interesting option, making even slower molasses...

Leave a comment on “Throw Some Hardware at it!”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article