In the bowels of a business unit, a director got a great deal on a third party software package. He bought it, without talking to corporate IT, and then was upset when it couldn’t gracefully integrate with any of the corporate IT assets. Eager to throw good money after bad, the director hired his nephew’s consultancy to build an integration tool to make his new toy work.

A few months later, the users complained about performance, and somehow, fixing this thing became Jeff’s problem. The process was simple enough: slurp enterprise data out of a text file, and pass the data on to the third-party tool. It didn’t take Jeff long to figure out why it performed poorly:

Private Sub ProcessFile()

    ' prepare to do stuff

    Do Until blnLastTime = True
        Set fileReader = fso.OpenTextFile(strFileName)
         If fileReader.AtEndOfStream = True Then
            blnLastTime = True
            strTextLine = fileReader.ReadLine
         End If

        ' actually do stuff
        Delete_Line (strFileName)

End Sub

Private Sub Delete_Line(strFile)

    Set fileReader = fso.OpenTextFile(strFile)
    If fso.FileExists(strFile & "2") Then
        fso.DeleteFile (strFile & "2")
    End If
    Set fileWriter = fso.CreateTextFile(strFile & 2)
    If fileReader.AtEndOfStream = False Then
    End If
    If fileReader.AtEndOfStream = False Then
            strLine = fileReader.ReadAll
            fileWriter.Write (strLine)
    End If

    fso.DeleteFile strFile, True
    fso.CopyFile strFile & "2", strFile, True
    fso.DeleteFile strFile & 2, True
End Sub

Start by opening a file “foo.txt”. Read a single line from the file. Send it to the third party app. Close the file. Open “foo.txt” again. Open another file, called “foo.txt2”. Read the first line from “foo.txt”, again. Throw that away. Read the remainder of “foo.txt”, and write it to “foo.txt2”. Copy “foo.txt2” back over “foo.txt”. Now, go back to the top of the loop and read a single line from “foo.txt” again.

So, for a 10,000 line file, this would perform 30,000 file open operations, write nearly 50 million lines, delete 20,000 files, and perform 10,000 copy operations. It didn’t take Jeff very long to rewrite this to simply read the file, one line at a time. The runtime dropped from a few hours to less than a minute.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!