"If at first you don't succeed, try, try again."

We have all encountered situations where we need to attempt an operation, with full knowledge that the operation might very well fail. And if it does, we should retry it. Usually after a delay, usually with a maximum number of retries.

Today's anonymous submitter inherited a site-scraping application. Already, we're in dangerous territory- site-scraping is inherently fragile, dangerous code. It's more dangerous, of course, when it's written by the culprit behind this:

public void ScrapeASite()
{
    try
    {
        //Some setup work
        var doc = HtmlWeb().Load(url); //a synchronous call
        //use the returned data to do stuff
    }
    catch(Exception e)
    {
                   //recurse, and hope the next request for the same domain 
                   //gets a different host from the load balancer
                    try
                    {
                        Console.WriteLine(DateTime.Now + " " + BAD SERVER!!! " + e.Message);
                    }
                    catch(Exception)
                    { }
                    //Rinse Repeat
                    ScrapeASite();
    }
}

The Load method getting called is synchronous, which certainly makes this code simpler, even if it's probably not the best idea. If what goes wrong is a timeout, you might be waiting quite awhile as your program hangs.

And when something does go wrong, we handle the exception.

I'm first delighted to see that Console.WriteLine is wrapped in a try/catch. We can't be too careful, I suppose. We'll just swallow that exception, if it happens, and then we call ScrapeASite. Recursively.

This is C#. The .NET runtime does support tail call optimizations, but the compiled Intermediate Language needs to contain certain op-codes to enable it. The F# complier emits those op-codes. The C# compiler does not.

So if the site goes down, or the network is out, or what have you, we start sending request after request by piling frame after frame up on the call stack. The good news is that this method probably only uses a relatively small amount of space on the call stack, which means you can do this many many times before getting a StackOverflowException.

Perhaps the person responsible for this code should try, try, try a little harder to catch their own mistakes.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!