• (cs) in reply to maht
    maht:
    I wonder what will be considered your things that are really exciting now but will sound ridiculous or banal in 10 years.
    Using keyboards and mice, maybe?
  • (cs)

    And I simply thought the "Transcoder" was a person. Like those old Monks who would translate the bible by candle light. I suspect they had Compuserve accounts (way back then).

  • w00t (unregistered) in reply to Dan Gardner
    Dan Gardner:
    bling:
    If by "extremely robust" you mean that the application has no way of knowing whether the message was actually delivered, yes.

    Of course the application can know whether the message was delivered. Given a properly configured MTA, the absence of a bounce will indicate that the message was delivered. It may not be immediate but that applies to any system involving submitting a message to a queue.

    This is what the standard has to say: " Delivery SMTP systems MAY reject ("bounce") such messages rather than deliver them."

    It doesn't say too much about what that bounce entails. Different MTAs, or differenly configured MTAs, can reject messages at different stages of the process.

    The code snippet supplied doesn't contain any error handling, so it will miss any errors that occur during the initial spooling at the local MTA (such as a rejection because the message is too large). If something goes wrong on the local MTA, it might give and error message, or it might send back a message; that bounce message might contain a reference that can be correllated back to the message that was sent. A typical bounce message is pretty useless even to human readers, and often doesn't include full headers from the sent message. And results may vary from the local spooling MTA and the remote MTA.

    Even if no bounce is received, and you'd need to keep polling your own mailbox to look at those, the message may still not be received. In fact, the bounces may be bouncing because your own mailbox is full, or other such mishaps.

    So, you'd need total control over both your local MTA, and the receiving MTA, so you can be 100% sure what their behavior is (no silently dropping e-mails!), what format their Delivery Status Notifications will be in (no changing the configuration!), and your own mailbox shouldn't fill up (we ARE talking about multi-megabyte messages here).

    Now, the very concept of an asynchronous message isn't flat-out wrong. In fact, it's a great idea, if implemented correctly. That's why there are correct implementations of this idea, invariably called something like 'enterprise message business queue broker talk bus'.

  • ignome (unregistered)

    Does anybody realize the link between SMTP and failures they observed? SMTP is not assured-delivery. It is practically impossible to detect a failure and correlate back to specific message. Bounce messages even if they arrive are pretty useless here - I don't know anyone that attempted to read and parse bounce messages. So if something went wrong somewhere, specific multimedia file will never be transcoded.

  • That's Me! (unregistered) in reply to w00t
    w00t:
    This is what the standard has to say: " Delivery SMTP systems MAY reject ("bounce") such messages rather than deliver them."

    It doesn't say too much about what that bounce entails. Different MTAs, or differenly configured MTAs, can reject messages at different stages of the process.

    And? Isn't it likely that this all happens on the same network on (an) MTA(s) completely under the control of these developers? If anything, the WTF here would be that their MTA is f*&%ing up... MTA's are almost build-and-forget.

    w00t:
    The code snippet supplied doesn't contain any error handling, so it will miss any errors that occur during the initial spooling at the local MTA (such as a rejection because the message is too large).
    You don't have enough info here to make this assessment. Do you know what error-handling is or isn't in DeliveryAgent()?
    w00t:
    A typical bounce message is pretty useless even to human readers, and often doesn't include full headers from the sent message. And results may vary from the local spooling MTA and the remote MTA.
    You know that these are fully-configurable on most MTA's that I'm aware of... You can even use processing on the MTA side to change the formatting of different bounces based on anything from intended recipient to message content. IIRC, the old bat-book explains how to do most of that.

    I'm gonna stop replying to this cause the rest is mostly just nonsense. Is SMTP the best way to deal with this? Who knows... with more reliable MTA's... maybe. Is it a giant WTF worth all the "on a table" gibberish? Not likely. Remember, your defenition of best != the next guy's. Maybe the original build requirement was "get me something by the end of the day that'll work", or "what can we do that's fast & cheap?" (which is a whole other WTF we've all had to deal with).

    Anyway, I'm not meaning to be too hard on you... I'm just tired of dealing with similar logic on a day-to-day basis. Just because you can over-engineer something to never fail, doesn't mean it's always the best choice to do so.

  • leeh (unregistered) in reply to dkf

    I think its far more enterprisey to deliver it to a MQ-Series/Tibco type system. Of course, if you can't trust your email systems to deliver messages, then you truly are enterprisey. This is a good lightweight solution, until you throw in point'n'click crap email system administrators.

    I say it would have been a huge WTF if they had reinvented the wheel and implemented their own queueing system, and paired it with making direct socket connections to the transcoding server.

  • Jim (unregistered)

    This reminded me of a "Web 2.0" version of a transcoding service. It was cited as a case study of using Amazon Web Services to scale out on-the-fly, using Ruby on Rails and other new tech.

    Wonder if they use email too?

    http://info.rightscale.com/content/case_studies

  • Bill (unregistered) in reply to John Doe
    John Doe:
    FredSaw:
    Locutus:
    EXACTLY! How hard would it be to implement RFC 1149 ??
    I have to admire Joe's perceptive, insightful comment on the proposal.
    Why? He obviously missed the date at the top of the proposal, so didn't grab its intention. It's been a tradition at the IETF to submit RFC's at April 1st, and this is one of my favourites :)

    To make it more robust, err, enterprisey, I'd suggest to use RFC 2549.

    No - if you want REAL enterprisey - then I would suggest Binary Lexical Octet Ad-hoc Transport. This is a classic piece on the usage of XML.

  • Andrew (unregistered) in reply to FredSaw
    FredSaw:
    Locutus:
    EXACTLY! How hard would it be to implement RFC 1149 ??
    I have to admire Joe's perceptive, insightful comment on the proposal.

    I like the legal notice under Joe's comment better. Anyone who takes medical advice from programmers deserves a funeral home discount.

    ... These posts are not intended to substitute for medical, tax, legal, investment, accounting, or other professional advice. ...

  • Pitabred (unregistered) in reply to rast
    rast:
    totiredtowork:
    Reminds me of one time when I requested a customer to email me an excel document. He then: 1) Printed the excel sheet. 2) Scanned the printed document. 3) Emailed me the jpeg.
    totiredtowork:
    Reminds me of one time when I requested a customer to email me an excel document. He then: 1) Printed the excel sheet. 2) Scanned the printed document. 3) Emailed me the jpeg.

    The real WTF is that he did it twice!

    Did what twice? Used a homonym of "too" in his username, rather than the proper spelling of the word?

    Captcha: smile. It's a joke ;)

  • (cs) in reply to Pitabred
    Pitabred:
    Did what twice? Used a homonym of "too" in his username, rather than the proper spelling of the word?
    No. He only did that once.
  • DW (unregistered)

    This is the kind of pragmatism money can't buy - when you're done realizing the cost of a Websphere license and drinking buzzword cool-aid, you'll realize that the Internet has had an extremely robust reliable messaging serving available since the dawn of time.

    Seriously, when is the last time you had a legitimate e-mail silently dropped? I see nothing wrong with the code above. The guy deserves a bonus.

  • Evan (unregistered)

    This is so eerie! I just learned today that one of my consulting Customers is purchasing an "e-commerce system" that will take orders online and send them to their in-house "line of business" system for processing. According to the vendor, each time an order is placed on the web site, an email will be sent from the web server to an address of my Customer's choosing. This email will have an XML file attached. Someone needs to open the email, save the XML file to disk, and do an "Import new order" in the "line of business" application such that the order can be processed. sigh

    I listened to the presentation and kept thinking to myself "I should write this up for DailyWTF..."

    Guess I don't have to.

  • mikko (unregistered) in reply to java.lang.Chris;
    java.lang.Chris;:
    foo:
    Some how I thought Java would be better when I left Perl behind... now I see that it wasn't Perl that was screwed up.

    The (alleged) WTF here is that SMTP is being used as a delivery method for processing requests, a sort of asynchronous remote procedure call. How you can interprete that as something you can beat Java with is beyond me. So just to counter your wild assertion, here's mine. Every large system written in Perl that I've encountered has been a mass of unstructured spaghetti, which is only understood (if at all) by the original author. Therefore Perl sucks for anything more than a one-shot throwaway hack.

    The possibilities resolve to three:

    1. You have not seen very many large systems.

      • very possibly you have just graduated trade school.

    2. You do not understand how to program in Perl.

      • or C or C++ or Fortran or Pascal or Basic or ...

    3. You program by the "cut and paste Java samples from Google searches" method.

      • see comment #1, see comment #2 (leave off "in Perl")

    I do hope that after you have had a chance to work in the industry for more that a month or two, you will realize that just because you don't know what to do when confronted by something beyond your comprehension does not mean that all others are so limited in capability...

  • w00t (unregistered) in reply to That's Me!
    That's Me!:
    Anyway, I'm not meaning to be too hard on you... I'm just tired of dealing with similar logic on a day-to-day basis. Just because you can over-engineer something to never fail, doesn't mean it's always the best choice to do so.

    And then there's just using SMTP for huge media files, which is in it's very own class of stupid. If anything it's underengineered.

    Consider the wide range of alternatives. FTP, SSH/SFTP, rsync, HTTP post are all pretty much synchronous and have clear, defined ways to deal with errors.

    SMTP was never meant to send binary files, isn't even 8-bit safe, in the real world it doesn't handle big (media) files, and doesn't have clearly defined error handling. It's not robust.

    You can just as well argue "hey, maybe using DNS tunneling was the best way of doing it!" - simply because an application can be abused beyond its original purposes doesn't make it close to a good idea. "Get me something by the end of the day" is a "requirement" that leads to WTFs, not a valid excuse for them.

    Oh, as for the 'you can't know about errorhandling' - notice the absense of any 'throw' declaration, and the absence of 'try'/'catch'.

    Besides, it's pretty clear this setup got submitted as a WTF precisely because it was unfit for its purpose - it didn't work!

  • gary (unregistered) in reply to w00t

    As was stated before:

    "it appears that only the request is sent via SMTP. The actual file upload happens later. "

    the xml request that is being sent to the transcoder only seems to include info that a new media file has been uploaded and where it is located.

    Not really a WTF in my opinion.

  • leah (unregistered) in reply to gary

    This is not a WTF as pointed out before. The media file upload is done elsewhere.

    The problem Toti has is that all of the employees got laid off 3 weeks before christmas last year without pay so he doesn't have anybody with knowledge of the codebase to maintain it for him.

  • Djigy lrgy (unregistered)

    It is almost kind of a WTF, because the administrators must now correctly configure and keep track of more services. I wonder if the system documentation says so anywhere.

  • Liinux (unregistered) in reply to FredSaw

    RFC 1149 really has been implemented, see this.

  • Matthew (unregistered) in reply to Dan Gardner
    Dan Gardner:
    bling:
    If by "extremely robust" you mean that the application has no way of knowing whether the message was actually delivered, yes.

    Of course the application can know whether the message was delivered. Given a properly configured MTA, the absence of a bounce will indicate that the message was delivered.

    Absence of a bounce?? You mean the developers have to implement functionality in the sender to receive bounceback messages and then try to match it to the original request?

    It may not be immediate but that applies to any system involving submitting a message to a queue.

    At least with HTTP could know immediately whether or not it made it into the queue at all. AND you don't have to impelement bounceback processing... which could get rather complex because you would have to parse MTA specific messages.

    I'm going to make note of your name and be sure never to work with you.

    -matthew

  • Sin Tax (unregistered) in reply to toby

    Why, of course if you want to put XML into a SOAP call, you have to encode it! Otherwise, how could you pass it as a string?

    (Never mind that the data needs to be parsed twice, once to remove the SOAP envelope, and then again to parse the actual data.)

    A former colleague of mine did exactly this some years ago. He was viewed by himself and several others as a quite intelligent Java programmer, so I suppose it was the right thing to do. I didn't understand it back then either. Still don't, actually.

    Sin Tax

    (captha: cognac - yeah I could use one right now.)

  • (cs) in reply to sf
    sf:
    real_aardvark:
    java.lang.Chris;:
    foo:
    Some how I thought Java would be better when I left Perl behind... now I see that it wasn't Perl that was screwed up.

    The (alleged) WTF here is that SMTP is being used as a delivery method for processing requests, a sort of asynchronous remote procedure call. How you can interprete that as something you can beat Java with is beyond me. So just to counter your wild assertion, here's mine. Every large system written in Perl that I've encountered has been a mass of unstructured spaghetti, which is only understood (if at all) by the original author. Therefore Perl sucks for anything more than a one-shot throwaway hack.

    Ooh look -- language flames wars, just when you least expect them.
    And then you proceed to regal us with your unsubstantiated opinions. Please take these old and tired arguments to another forum. To everyone else: PLEASE DON'T FEED THE TROLLS!
    Sincere apologies. I do so hate to bruise your tender ego, or other parts.

    Any chance you could explain what "Unregistered" means?

    Just asking, trollie ...

  • (cs) in reply to That's Me!
    That's Me!:
    Anyway, I'm not meaning to be too hard on you... I'm just tired of dealing with similar logic on a day-to-day basis. Just because you can over-engineer something to never fail, doesn't mean it's always the best choice to do so.
    On the other hand: if it costs you tuppence to get it right, given that the infrastructure is already there (as in the OP), wouldn't that be the best choice?

    Just a thought.

    Me, I'm lurve sendmail. Such a wonderfully thought-out and ahem infinitely configurable protocol.

    %"()&£---*&(%=X

    (This probably means something in Sendmail configuration.)

    Having worked with the only real hardware fault-tolerant system in history -- Stratus -- I would accept your aversion to "(some)things that never fail."

    I do not, however, accept your presumption that an attempt to approach this limit is "over-engineering."

    If anything: it's the only bit of engineering that allows all of us to describe software as an engineering discipline. With tolerances. Measurable. Y'know?

  • Mr B (unregistered)

    Ok so I did WTF when I first saw this.

    Then I read the comments, I re-read the code, had a walk around the kitchen while I searched for some nicotine gum, and then thought to myself "The WTF here is a TRUE WTF for once because its the WTF I would say if I found another developer had so stupidly broken the implicit contract i.e. NOT to make more ***** work for me".

    Maybe it was the original coder, maybe it was the original spec or maybe it was management who didnt want to pay the price, but someone, somewhere decided that they need not love, honour or obey (or even specify maybe!) the requirement that the system should be "RELIABLE"....hence Toti's WTF at seeing email used as a delivery method in a system where its obviously totally inappropriate. Poor Toti!

    Emails is GREAT if you dont give a flying French faggot whether your message is actually delivered to your recipient(s)...but if you do (and I love the French)...well something a little more.....robust is called far.

    Its a WTF.

  • Funky Monkey (unregistered) in reply to Dan Gardner
    Dan Gardner:
    What is wrong with using SMTP for passing messages between servers? It is extremely robust - if the Transcoding Server was not available for some reason, the message should have been queued and delivered when the server became available again. It can support authentication (TLS) and handles transport failures gracefully, all without any additional application code. I see no point in re-inventing the wheel.

    Unless they were using Microsoft sExchange as their MTA (which would be a WTF) I can't see the problem here.

    That would be a WTF - but not for the reasons you think - it's a WTF because (until very recently) Exchange isn't an MTA. It doesn't include one. The IIS SMTP Service is what Exchange uses as an MTA, and it's /plenty/ robust.

    Exchange 2007 breaks this slightly, by bundling an MTA (something that funnily enough, Exchange 5.5 did - with the Internet Mail Service, which was also fairly robust), but the behaviour of E2007's MTA at the protocol level clearly indicate it's based on the IIS6 smtpsvc codebase.

  • (cs) in reply to toby
    toby:
    OK, at least they built the XML correctly! Here's the C# code I "inherited" to create a web request for an internal app. At least they actually converted the quotes and angle brackets... 
    public string BuildRegistration(Config config, IEmployee Employee, string PlanXml)
{
    StringBuilder content = new StringBuilder();

    content.Append("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n");

    content.Append("<!DOCTYPE InitechRegistration PUBLIC '/dtd/InitechRegistration.dtd'  'http://www.initech.com/dtd/InitechRegistration.dtd'>\n");

    content.Append("<InitechRegistration Environment=\"").Append(config.Environment.ToUpper()).Append("\" Status=\"RUN\" Client_ID=\"").Append(config.SponsorCode).Append("\" Subcode=\"").Append(Employee.BenefitCode).Append("\" EnrollmentYear=\"").Append(config.PlanYear).Append("\" SessionURL=\"http://www.initech.com\">\n");

    content.Append("  <Employee Person_ID=\"").Append(Employee.EmployeeID).Append("\" CurrentOptionLevel=\"").Append(Employee.BenefitGroupCode).Append("\" ZipCode=\"").Append(Employee.ZipCode).Append("\" />\n");

    content.Append("  <PlanEligibility CostStructure=\"NON-UNITIZED\">  \n");

    content.Append(PlanXml).Append("\n");

    content.Append("  </PlanEligibility>\n");

    content.Append("</InitechRegistration>\n");

    content = content.Replace("<", "<");
    
    content = content.Replace(">", ">");
    
    content = content.Replace("\"", """);

    return content.ToString();
}

    OK. Thanks for playing; but do you really, seriously think that anybody is going to read this crap?

  • (cs) in reply to Funky Monkey
    Funky Monkey:
    Dan Gardner:
    What is wrong with using SMTP for passing messages between servers? It is extremely robust - if the Transcoding Server was not available for some reason, the message should have been queued and delivered when the server became available again. It can support authentication (TLS) and handles transport failures gracefully, all without any additional application code. I see no point in re-inventing the wheel.

    Unless they were using Microsoft sExchange as their MTA (which would be a WTF) I can't see the problem here.

    That would be a WTF - but not for the reasons you think - it's a WTF because (until very recently) Exchange isn't an MTA. It doesn't include one. The IIS SMTP Service is what Exchange uses as an MTA, and it's /plenty/ robust.

    Exchange 2007 breaks this slightly, by bundling an MTA (something that funnily enough, Exchange 5.5 did - with the Internet Mail Service, which was also fairly robust), but the behaviour of E2007's MTA at the protocol level clearly indicate it's based on the IIS6 smtpsvc codebase.

    No.

    It's plain idiocy.

    Two servers, one protocol, and a Ring to bind them.

    There is no. Repeat, no. Repeat for emphasis, no reason to handle messaging this way.

    Only a fucking moron would choose to use SMTP as a messaging system between two servers.

    Only a (possibly different) fucking moron would choose to defend such a decision.

    The various RFC and SMTP variant comments are sad, sad, reflections on the state of software engineering today.

    Jesus Java Fucking Christ..

    I would far rather see this serialised through some horrid Java interface, or else see it synchronised at a hardware level via rsync or some such, or even sent over a TCP connection via XML (enterprisey!) with an ack.

    The "solution" in the OP is clearly doomed. Let's all stop arguing about how this or that RFC might potentially make it ever so slightly more reliable.

    It's doomed. Doomed, I tell you.

    And I think that was the point of the post.

  • (cs) in reply to leeh
    leeh:
    I think its far more enterprisey to deliver it to a MQ-Series/Tibco type system. Of course, if you can't trust your email systems to deliver messages, then you truly are enterprisey. This is a good lightweight solution, until you throw in point'n'click crap email system administrators.

    I say it would have been a huge WTF if they had reinvented the wheel and implemented their own queueing system, and paired it with making direct socket connections to the transcoding server.

    Huge, huge, huge.

    Round about a thousand lines of C++, Java, C#, take your pick.

    And, the sad thing is, it would actually work. To requirements. As opposed to some piece of Sysadmin-mandated email shit that wasn't even designed to come near to requirements.

    Do they pay you minimum wage in your job? Because, if they do, you're stiffing them on basic competence levels.

  • Some C programmer (unregistered) in reply to vt_mruhlin

    Agreed that SMTP is a reasonable messaging system. However, if the transcoding server listens on port 25, you lose half the benefits of STMP messaging. Much better to have a real mail server on port 25, so it can queue requests if the transcoder is temporarily down.

  • Brad (unregistered) in reply to bling

    Part of the e smtp specification that is widely implemented is called delivery status notification (dsn) so indeed it is perfectly likely that someone could verify delivery within a time period.

  • ajk (unregistered) in reply to DW
    DW:
    Seriously, when is the last time you had a legitimate e-mail silently dropped? I see nothing wrong with the code above. The guy deserves a bonus.

    LOL!

  • Gambler (unregistered)

    ZOMG. I can't believe people are defending using SMTP here.

    The correct lightweight solution would be to have the upload server write the files to a shared directory and the transcoding server to periodically poll if any files are present.

  • wedonthavethewholepicture (unregistered) in reply to real_aardvark
    real_aardvark:
    Only a fucking moron would choose to use SMTP as a messaging system between two servers.

    Unless of course it's the only way to get around some corporate ICT rules that we haven't been told about.

    Perhaps the two machines are on separate networks and no direct communication from the 'Upload Server' to the 'Transcoding Server' is allowed?

    Perhaps only certain software is allowed ?

    We don't know.

  • Tom Melly (unregistered)

    I make no comment of the WTF-ishness of the solution, but it's worth noting that Bugzilla uses email to move bugs between different Bugzilla systems, even if they are on the same server.

    I assume that this is because it's flexible and (generally) universally available.

  • nomdeplume (unregistered) in reply to Gambler
    Gambler:
    The correct lightweight solution would be to have the upload server write the files to a shared directory and the transcoding server to periodically poll if any files are present.
    Totally agree with this: clearly the transcoder has access to the location of the files, so this is simply one less point of failure. If you can't simply use "a file exists!" to test for the appearance of a new file, then the little request XML thing could be dumped into a queue directory on the same filesystem, which would be polled by the transcoder.
    ZOMG. I can't believe people are defending using SMTP here.
    OTOH, I don't think SMTP is that bad a choice, particularly if SMTP is used for other purposes within this transcoder thing and therefore the service needs to be maintained anyway. This is assuming that the servers involved are reasonably well controlled, i.e. messages won't randomly be eaten by a spam filter (but having a spam filter inspecting such messages would be a WTF in itself), but you'd need that guarantee for any other solution to be reasonably sane, anyway. There's no reason why properly configured SMTP servers would start dropping messages any more than a properly configured web service would.
  • nomdeplume (unregistered) in reply to real_aardvark
    real_aardvark:
    Me, I'm lurve sendmail. Such a wonderfully thought-out and *ahem* infinitely configurable protocol.

    %"()&£---*&(%=X

    Sendmail isn't a protocol, it's a program which implements SMTP. Yes, it has a horrible configuration file syntax. For a saner alternative, try Postfix or qmail.

    SMTP doesn't seem like that inelegant a solution here, but it really depends on how the transcoding server was implemented. The real WTF here is that we don't know how the story ended. What caused the transcoder to hang all the time? It's not as if x% of your messages will vanish into thin air because they're being transported via SMTP.

  • (cs) in reply to mikko
    mikko:
    java.lang.Chris;:
    foo:
    Some how I thought Java would be better when I left Perl behind... now I see that it wasn't Perl that was screwed up.

    The (alleged) WTF here is that SMTP is being used as a delivery method for processing requests, a sort of asynchronous remote procedure call. How you can interprete that as something you can beat Java with is beyond me. So just to counter your wild assertion, here's mine. Every large system written in Perl that I've encountered has been a mass of unstructured spaghetti, which is only understood (if at all) by the original author. Therefore Perl sucks for anything more than a one-shot throwaway hack.

    The possibilities resolve to three:

    1. You have not seen very many large systems.

      • very possibly you have just graduated trade school.

    2. You do not understand how to program in Perl.

      • or C or C++ or Fortran or Pascal or Basic or ...

    3. You program by the "cut and paste Java samples from Google searches" method.

      • see comment #1, see comment #2 (leave off "in Perl")

    I do hope that after you have had a chance to work in the industry for more that a month or two, you will realize that just because you don't know what to do when confronted by something beyond your comprehension does not mean that all others are so limited in capability...

    I've been programming commercially for 12 years, and first learned C eight years before that. I've designed and been lead developer on systems with 6 figure lines of code, and worked on other systems with seven figures lines of code. That's option one covered.

    I contracted for much of the late 1990's, working in a mixture of C (for performance) and Perl (for convenience). However, when Java started to perform well, I switched from Perl, sick of fixing the spaghetti other people often wrote. As systems have bcome more powerful, and the Sun JVM more optimised, I've gradually stopped using C and C++. I'm pretty much guru level on Java and C, competent at C++ and would have once considered myself guru level at Perl. I've also maintained some FORTRAN 77 code, and written a fair bit of assembler. Overall, I used Perl every working day for five or six years, so that's option two taken care of.

    As for understanding how to program, I work as a senior developer, designing large scale Java systems as well as the accompanying database schemas. I have written frameworks for everything from asynchronous messaging to MVC web publishing.

    In short, confident I'm likely to win any pissing contest against you ...

  • John B (unregistered)

    Forget faxing, wooden tables, & SMTP. The most reliable transmission would be to print each request on standard paper, seal it in an envelope, and add it to the canvas mail bag. Once a week, the station master would hang the bag on the hook above the track, and wait for the daily steamer to come thundering by. At the end of the week, when the steamer comes by the other way, the station master waits by the receiving hook for the report of how many messages were delivered.

    Points of failure can be minimized by creating cron jobs to shoo away large cows along the route, and performance monitoring to gauge when there is too much whiskey back in the caboose.

  • Michael.H (unregistered)

    The real WTF here is some of your comments lol

  • whatevuh (unregistered) in reply to java.lang.Chris;

    good for you. go get a kitkat and give (us) a break.

    a bigger wtf will come around when somebody decides they also need to use the system and the SMTP trans-coding server lives for much longer that it was probably ever meant to. i'm sure the l33t d00d who wrote this is quite chuffed with himself.

  • Logan (unregistered) in reply to Mexi-Fry

    Ah yes, HTTP makes it easier to detect the errors. That I agree with 100%. Detecting errors with SMTP is ugly and non-trivial.

    Now for the next question: once you have detected that the HTTP server isn't available at this precise moment, how do you handle that error? You still need the work done. But you're just client code and perhaps you don't have a queuing system handy to retry this stuff. So you either try once, dutifully stash the error in a log file that nobody checks, or you try again with some timeout and then dutifully stash the error message in a log file that nobody checks.

    Basically, in the real world, when an HTTP failure happens, you know about, but there is often no plan to deal with it.

    Meanwhile, if you use SMTP (and if your SMTP server isn't crap), it will take ownership of the responsibility for delivering the message and then the sender of the message can stop owning it.

    Yes, it's imperfect (as evidenced by the fact that a developer sometimes had to give the system a smack to get it unstuck), but it's often better than forcing the queuing logic into client code.

  • mokus (unregistered) in reply to real_aardvark
    real_aardvark:
    Only a fucking moron would choose to use SMTP as a messaging system between two servers.

    Such hate! "Only a fucking moron" would get this worked up over something they know so little about - you've seen what, 20 lines of code here? And you suddenly know that because SMTP is involved, SMTP cannot but be the cause of an inevitable catastrophic failure...

    Believe it or not, there are times and places where it is perfectly acceptable to produce a system reliable to fewer than 5 nines. Believe it or not, sometimes the Simple Message Transfer Protocol is a viable way to Simply Transfer Messages. And, horror of horrors, it might even work!

    Some people need to learn a few simple phrases along the lines of "I am not the only person smarter than peat moss," or "the first thing I see that I don't like is not necessarily the cause of all evil in the universe." Maybe even "people I disagree with might not all be complete fucking morons." Because, really, it doesn't reflect badly on them at all when you call them that in public.

    BTW - any statement that begins "there is no reason to..." is almost certainly false. Even if you add the implied "good."

  • (cs)

    As a former Sendmail administrator for a VBC, I feel confident I know the real WTF here. Or, rather, WTFs.

    1. Applications which send SMTP messages do not, generally, send the messages via SMTP. They make a single attempt to send the message to the SMTP server, and disregard any error messages. I've only seen two exceptions to this, out of hundreds of applications. Note that, in my experience, far more application messages are lost because the nearest email server had a momentary load issue and the application failed to try the next MX or queue the message for another attempt than are lost due to message bouncing.

    2. The worst problem with SMTP for message passing in general is that it frequently requires multiple hops to get to the destination. Even with fairly optimal routes, the total network cost is almost always at least twice the cost of a protocol which can deliver it directly (like, say, FTP, which was specifically designed to transfer files.)

    3. Delivery time is at the mercy of the email system load. An email problem on any system it needs to go through can substantially delay the message, even if it's not a problem that causes the message to bounce.

    4. The majority of programs I've seen which receive automated messages are only marginally better than my grandmother at reading email. For example, I've seen half a dozen that would core dump if they received an email with no 'From:' header - despite the fact that they never did anything with the From: header other than parsing it.

    5. A message queue can be implemented for any protocol which indicates delivery status. It doesn't require explicit permission from the protocol.

    6. In general, writing an application to parse the email is about as difficult as it is to write an application to receive the email through either a telnet or ssh connection.

    7. TRWTF is that the email server is Sendmail using background DeliveryMode. I could explain why that's a WTF, but since nobody's likely to read this, I'll save my fingers to explain it elsewhere.

Leave a comment on “Old Fashioned Web Services”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article