• grumble (unregistered)

    I love that a perfect explanation was given very early on about float representation by Simon Bradley but has been totally glossed over by a bunch of people that think they're programmers. That's my WTF for the day.

    captcha: stinky - mmmmmmm

  • (cs) in reply to PLasmab

    I think .AwayFromZero should be the default, too... but Banker's Rounding does have a justification.

  • Charles Perreault (unregistered) in reply to Look at me! I'm on the internets!
    Look at me! I'm on the internets!:
    Bankers rounding has nothing to do with it. How could bankers rounding have anything to do with it if he never calls the Round function?

    39.999 is actually stored internally as 39.9949...

    3999.49... + .5 = 3999.99... int(39.9999...) = 39.99.

    I agree about the banking rounding function, it has nothing to do with it. But the internal representation of 39.999 (if VB uses IEEE754) would be :

    0x421ffefa in hexadecimal for only 32-bits (float)

    If we print back that number we obtain : 39.999000549316406 which is not 39.9949, even with single-precision 32-bits floating point. In double precision, the number would be even closer to 39.999. Your explanation don't work, unless VB don't use the cpu's FPU and registers to hold its floating point values.

    Here is the C code to get those results: #include <stdio.h>

    int main() { float d = 39.999; int i = (int)&d; printf("0x%08x %2.15f\n", *i, d); return 0; }

    I don't have VB installed, I work under Solaris, and so cannot see what's going on with the posted code. I don't have time right now, but if anyone post a snippet of C / C++ code that exhibits the same odd behaviour as the original VB code, I will gladly find the problem.

  • Charles Perreault (unregistered) in reply to grumble
    grumble:
    I love that a perfect explanation was given very early on about float representation by Simon Bradley but has been totally glossed over by a bunch of people that think they're programmers. That's my WTF for the day.

    captcha: stinky - mmmmmmm

    I did not see the Simon Bradley explanation on my first reading so I went on with two (now useless) posts, but I totally agree with you.

  • (cs) in reply to Charles Perreault
    #include <iostream>
    
    int main() {
        std::cout << (int)(39.995 * 100 + 0.5) << std::endl;
    }
  • Rohit (unregistered)

    It is a just a side effect of how IEEE floats/doubles are represented. The general formula for a double( not counting the zeros , nans and infinities ) for instance is V=(-1)S * 2 ** (E-1023) * (1.F) S being the sign bit E the exponent bits and F the mantissa bits.Any compiler / interpreter would first have to convert the given base 10 float into this format.While converting the digits to base 2 to the left of the decimal point is easy, the digits to the right often end up having a large summation of the form a(n) * 2-n where a(n) is 0 or 1 and where n varies from 1 to m and m> number of bits of the field F(52 for double) .Hence the float ends up being represented inaccurately and any further operations on it would obviously be in error as well.As an example you could try firing up some interpreter and inputting various floats and observing the output. On my system in python:

    39.995 39.994999999999997 39.995 * 10**2 3999.4999999999995

    Why that rounding scheme doesnt work is now fairly obvious.

  • morphemass (unregistered)

    Interesting thread. Given the S/N ratio it looks like it would make a VERY good interview question.

  • wade b (unregistered)

    It is definitely NOT the bankers rounding issue. Test it with: 37.995, 38.995, 39.995 to see for yourself.

    It is loss of precision as explained previously.

    The answer is: tell your programmer to use the Currency datatype instead of Double.

    Also store those values in the DBMS as DECIMAL or the equivalent.

    Once that is complete, you must still account for the bankers rounding issue with the VB round function (write your own, as previously described)

  • emmbee (unregistered)

    It's the usual one.

    IEEE 754 doesn't store decimal digits (i.e. sum(10^-i)*10^exp), but binary ones (i.e. sum(2^-i)*2^exp).

    There is no exact representation of either 39995 or 11995 that way, so they're approximated. If they're approximated above (11995.1...), the function works, because casting to an int will round down. If they're approximated below (3999.49..), it won't.

    It's the same in all languages when you deal with IEEE 754. The fact that it's VB is irrelevant.

  • newt0311 (unregistered) in reply to

    indeed. for financial apps, one should be using arbitrary precision libraries.

  • Rohit (unregistered) in reply to emmbee

    39995 and 11995 are exactly represented , so are 3999.5 and 1199.5 but 399.95 and 199.95 arent in the available mantissa bits.

  • Crotchety Old Guy (unregistered) in reply to wade b
    wade b:
    It is definitely NOT the bankers rounding issue. Test it with: 37.995, 38.995, 39.995 to see for yourself.

    It is loss of precision as explained previously.

    I vote for loss of precision, too.

    It can't be bankers rounding, cuz his example says that 11.995 rounds to 12.00. Bankers rounding (with two digits precision) would have rounded down to 11.99 because the digit before the 5 is odd, and there are no trailing non-zero digits.

  • AdT (unregistered) in reply to grumble
    grumble:
    I love that a perfect explanation was given very early on about float representation by Simon Bradley but has been totally glossed over by a bunch of people that think they're programmers. That's my WTF for the day.

    captcha: stinky - mmmmmmm

    Don't worry - the forum is always flooded by comments that have nothing to do with the WTF (like those concerning banker's rounding with Round() when the posted code actually uses Int()) or suggest "improvements" that are actually worse, e.g. "return INT(number + (factor * 0.5))" which "rounds" 0 to 50 when digits=2.

    My favorite WTFish comment so far is, though: "it's a generally known fact that == operation fails in all programming languages, if the operands are of type float. == works only with integers."

    And although Simon Bradley came to the rescue and noted that 39.995 is actually represented as the binary equivalent of ca. 39.994999, I have no illusions that everyone who even took the trouble to read this comment understood the implications.

  • linepro (unregistered)

    Int, Fix Functions

    Returns the integer portion of a number.

    Syntax

    Int(number)

    Fix(number)

    The required number argument is a Double or any valid numeric expression. If number contains Null, Null is returned.

    Remarks

    Both Int and Fix remove the fractional part of number and return the resulting integer value.

    He could have used Round().

    The real WTF is he can't even read the MSDN help

  • AdT (unregistered) in reply to deathbydesignflaw
    deathbydesignflaw:
    "CINT(number + (factor * 0.5))"

    is real close.

    Yes, roundTo(0, 2) = 50 is "real close" indeed.

    deathbydesignflaw:
    The 'easiest' I think is to do CINT(number + (.5 / factor)).

    So that roundTo(3.899, 2) = 3. Brillant, deathbydesignflaw, just brillant!

  • RobertB (unregistered)

    Since nobody will admit to being a VB programmer, I'll add my 2c, for I have no shame. I fired up VB6 and tried it.

    I put a breakpoint here:

    temp = number * factor + 0.5

    Going to the Immediate pane, I check the value of temp after this operation:

    ?temp
     4000 

    Then, I find out what the value really is.

    ?temp-4000
    -4.54747350886464E-13 

    As noted before, the original multiplication resulted in a non-exact floating point number in the neighborhood of 3999.9999999..., so Int() truncated it just like it's "supposed" to.

    Oddly enough, I saw this CodeSOD while debugging a rounding error... in a financial program.

    captcha: dreadlocks. I hate it when Sybase aborts my queries, too.

  • Vlad Patryshev (unregistered)

    I do not see any problems with the code. Maybe the problem is with the programmers that did not learn enough math at school, and are unable to understand this simple stuff? My condolences then. You should become managers, and not bother about understanding other people's code.

  • JackB (unregistered)

    Så what does ^ mean in VB is it an exponent operator or is it XOR like java. I don't get this code if it is XOR.

  • Gorshkov (unregistered)

    Wow.

    It is amazing to me how few people here - presumably, those feeling full of themselves for being able to laugh at other people's silly mistakes - have absolutely no conception of numerical methods.

    The lot of you should be ashamed of yourselves. Kudos for the person very, very early on (although not early enough on to stop the rest of these silly postings) who pointed out the IEEE format for floating point numbers.

  • Your Name (unregistered) in reply to JackB
    JackB:
    Så what does ^ mean in VB is it an exponent operator or is it XOR like java. I don't get this code if it is XOR.
    It is the exponent operator in VB.
  • MrBester (unregistered) in reply to Otto

    Yes, ^ is the exponent operator in VB...

    Otto:
    <snip/> -Adding .5 at this point is supposed to do the round operation. If it already had .5 or more in the fractional part, this would cause the integer to round up. This is where the failure happens, of course. -Finally, it truncates off the fractional part (that's what Int() does), and divides by the factor to move the decimal back up.

    The failure is that it's doing this not to actual decimal numbers, but to floating point numbers. As has been pointed out already, 39.995 cannot be represented exactly in that limited space, and you actually get 39.994998931884766. When you do the math on that, you get: 39.9949989 * 100 = 3999.49989 3999.49989 + 0.5 = 3999.99989 Int(3999.999) = 3999 3999 / 100 = 39.99

    <snip/>

    If you're doing your math in base 10, use a different representation. Visual Basic has a Decimal data type for this very reason (which I think uses a BCD format internally). Most other languages have something similar.

    But if you don't know about the Decimal data type (for whatever reason), what you could have done is run the calculations with ++digits, adding 5 instead of 0.5 and Int() it after the division: 39.9949989 * 1000 = 39994.9989 39994.9989 + 5 = 39999.9989 39999.9989 / 1000 = 39.9999989 Int(39.9999989) = 40

    12.995 * 1000 = 12995 12995 + 5 = 13000 13000 / 1000 = 13 Int(13) = 13

    captcha: stinky. And how.

  • wade b (unregistered) in reply to grumble
    I love that a perfect explanation was given very early on about float representation by Simon Bradley but has been totally glossed over by a bunch of people that think they're programmers. That's my WTF for the day.

    Ya think so? Those are good explanations and I believe they apply but also look a little deeper at the vb code.

    There's another bug with this statement: Int(temp) / factor

    The Int function truncates so Int(99.5) becomes 99.

    That is the biggest of SEVERAL problems with the current routine.

    Toss your insults somewhere else, I've got 10 years of real working experience but don't feel the need to dis-respect others.

    My WTF for the day.

  • MrBester (unregistered) in reply to MrBester

    Dammit all to hell, <slap target="self"> Int() truncates...

    Boy, do I feel stupid now

  • (cs)

    <sighs> There are a number of things every system designer should know. They should be taught in grade school or something.

    Never use binary floating point for money calculations. Decimal fixed point is the only way to do it. That's what COBOL and PL/I are for, folks; use them! God knows they're not good for much else.

    In Java, use BigDecimal with the BigDecimal(String) constructor. In SQL, use the Decimal data type.

  • Bert (unregistered) in reply to hpeg
    hpeg:
    Which then makes people do:
    public float round(float f, int prec) {
        BigDecimal big = new BigDecimal(f);
        big.setScale(2, BigDecimal.ROUND_HALF_UP);
        return big.floatValue();
    }

    Sorry, hpeg. You better write a unit test for that code. You just returned the same exact value. BigDecimal is immutable, so you can not round big. You have big create a new, rounded BigDecimal that contains the result. The correct code (with debugs) is:

    public float round(float f, int prec) {
    	BigDecimal big = new BigDecimal(f);
    		System.out.println("big=" + big);
    	BigDecimal tmp = big.setScale(2, BigDecimal.ROUND_HALF_UP);
    		System.out.println("big=" + big);
    		System.out.println("tmp=" + tmp);
    	float ret = tmp.floatValue();
    		// using BigDecimal to unveil true value
    		System.out.println("ret=" + new BigDecimal(ret));
    	return ret;
    }

    Of course, since you are starting with a floating point and converting it back to a floating point... round(39.995F, 2) still gives you 39.99xxxxxx,

    Here is the debug output big=39.994998931884765625 big=39.994998931884765625 tmp=39.99 ret=39.990001678466796875

    I hate floating point numbers!

    P.S. I was tempted to use a StringBuffer for my string concatenations. ;-)

  • wade b (unregistered) in reply to newfweiler
    Never use binary floating point for money calculations. Decimal fixed point is the only way to do it.

    I was with you up until:

    That's what COBOL and PL/I are for, folks; use them! God knows they're not good for much else.

    Arrgggh!

    I'd sooner drop down to ASM and use BCD instructions there (are they still supported on X86, crap I haven't used that stuff since 486...)

    Or roll your own using strings - I know that sounds funny but it's basically ASCII-encoded 8-bit BCD.

    Each digit is represented with one byte using it's ASCII char equivalent. Then you iterate over this buffer and do your math ops (basically).

    Sounds crazy but I've seen a good working implementation in a C algo book I had (by Binstock & Rex, don't remember title).

  • Danger (unregistered) in reply to tuomas aaltio

    Not Ruby :-)

  • (cs)

    Floating point, as has been noted, causes problems in places due to inexact representations. One solution: BCD instead of floating point. The old Vax processors had it as a native type; I've seen business 4gls that use it as their native real type. 50 sig figs with 10 decimal places, never a rounding issue or inexact result.

  • Decimator (unregistered) in reply to newfweiler

    The other option is to use decimal floating point. By using a representation with a decimal exponent instead of a binary exponent all finite decimal number can be represented exactly. Granted the rounding errors are always present, but the representational issues in many financial applications (tax rates, fractional prices, etc) are gone.

    Decimal floating point will be part of IEEE 745R. For more information than you ever wanted to know see http://www2.hursley.ibm.com/decimal/

  • bramster (unregistered) in reply to Cotillion
    Cotillion:
    I think the problem is that he's mixing floats and doubles. I had a similar thing happen in C#. The 0.5 is instantiated as a float and then cast to a double. Sometimes, funny things can happen when casting between the two.

    Unless I don't understand VB (possible) then it has nothing to do with baker's rounding. Casting to an int should truncate, not round.

    I think it's got to do with the digits as integer. Any number above 32.768 is going to rounded wrongly.

  • Kiss me, I'm Polish (unregistered)

    Nobody pointed out yet that there's no such thing as $0.001, since that would make less than a cent. But the joke has never really been funny. I like the fact that some people here actually know what they're talking about. I almost had forgotten how the floats were stored.

    Captcha = boobies. No, actually I lied about the captcha.

  • Fan the flames! (unregistered) in reply to Kiss me, I'm Polish

    $0.001 = 1 mill

    http://en.wikipedia.org/wiki/Mill_(currency)

  • Eric (unregistered) in reply to wade b

    There's another bug with this statement: Int(temp) / factor The Int function truncates so Int(99.5) becomes 99.

    I'm not following. That int() is quite intentional and is meant to truncate the number to the expected number of decimal places. It's not truncating "99.5" to "99", it's truncating "9950.6" to "9950", then dividing by 100 to get "99.50".

  • BBT (unregistered) in reply to Kiss me, I'm Polish

    $0.001 = 1/1000 of a cent, according to Verizon Wireless.

  • snoofle (unregistered) in reply to Eric

    Works as coded.

  • TSK (unregistered)

    The code is bad and the comments are even worse.

    a) As many people already pointed out: Never use doubles. The point is that binary fractions in decimal representation always ends with 5 or 0. So fractional input data which does not end with these both ciphers will be inevitably mangled. Because something which ended by five divided by two always returns a five you get the unfortunate result that the exact decimal representation of n fractional bits is n decimal digits long ! So a correct decimal representation of a double has in the worst case 53 decimal digits ! Worse, while the processor offers flags for internal rounding (-Infinity, towards zero, away from zero, +Infinity) they are inaccessible from almost all languages. So you can't even choose if your implementation choses 3.9999.... or 4.0000; indefinite behaviour.

    b) The effective range of the double. Only 16 digits are stored, so if a number which is greater than 2^54 is stored in x and you try to count to 2^55, you can add one to x endlessly because it will be ignored (it is too small).

    c) You can't use 10 ^ x as factor. Most languages transform that into e^(x*ln(10)) which is given back mostly by 1 ulps precision. But the power function accumulates much more significant digits (multiplying two numbers adds the number of digits) than given back by the e function; hence the result has a very large relative error.

    So bluntly spoken: Each use of double is garbage. GARBAGE. And inventing these "corrections" of adding 0.50001 etc. are plain dumb because they almost always give the correct result and give therefore a secure feeling.

    Either use BCD or arbitrary precision binary. The last one is very fast for adding, subtracting etc., but has an decisive disadvantage for binary -> decimal conversion because you must use time expensive division for converting integers. BCD is not so fast for adding and much slower for multiplication, but very easy and fast to convert.

  • TSK (unregistered)

    (Hopely self-evident) correction: Each use of double for financial calculation is garbage.

  • ChiefCrazyTalk (unregistered) in reply to anonymous
    anonymous:
    VB uses banker's rounding http://en.wikipedia.org/wiki/Banker%27s_rounding , a well understood WTF. There's even a Microsoft KB article, can't find the id at the moment.

    You are correct, but its not a WTF. I remember running into this circa 1997 for the first time (using Delphi, not VB). The real WTF (tm) is that programmers dont know how the Real World (tm) works, and the real world outside of technology uses Bankers Rounding.

  • (cs) in reply to wade b
    wade b:
    There's another bug with this statement: Int(temp) / factor

    The Int function truncates so Int(99.5) becomes 99.

    That is the biggest of SEVERAL problems with the current routine.

    That's what you want it to do.

    number = 0.995 digits = 2

    factor = (10 ^ digits) = 100 temp = (number * factor + 0.5) = 99.5 + 0.5 = 100.0 (if exact) roundTo = (Int(temp) / factor) = 100 / 100 = 1.00

    Thus, it is what you want.

    Now make the number 0.997. So temp is 100.2. You don't want this .2, so you truncate it and get the 1.00 you expected.

  • xcor057 (unregistered) in reply to Anon

    sixidecimal?

  • (cs) in reply to Kit
    Kit:
    Ah, I know what it is! ... But it's (for lack of a better word on the spot) slightly ironical. ^^

    Perhaps a better word is "ironic": a shorter adjective with exactly the same meaning <s>

  • Tom (unregistered) in reply to wade b

    There's another bug with this statement: Int(temp) / factor

    The Int function truncates so Int(99.5) becomes 99.

    Ok wade, take it down a notch. It's supposed to truncate. That's what the 0.5 increment was for. After being incremented by 0.5, 99.5 becomes 100.0, Int() or which is 100, as expected. The problem here isn't truncation. It's the inability of floating point types to store certain decimal numbers with absolute precision.

  • Jon W (unregistered)

    Being a game physics, collision and graphics coder, I have had my run-ins with floating point.

    However, floating point is not "inherently imprecise" any more than integers are. The rules for floating point math are very well defined, and if you KNOW THE RULES then you can use it safely and efficiently, even using the "==" comparison operator.

    The WTF is, of course, as has been mentioned, using floats/doubles for money. Back when I did ERP/financial accounting systems, we knew that, and used Decimal, Numeric, and similar data types. In fact, IBM-type programmers have known this all the way since the '60s. The real WTF is people working as "programmers" who never passed Numerical Analysis 101 (or even knew that there were colleges giving those classes).

    I'm posting this mostly because I want to kill dead the rumors that floating point is inherently "unreliable" and that "you can't use == with floating point" -- those statements are un-true, and borne out of not understanding what floating point numbers are, and what they do. What they do, is very appropriate for certain scientific calculations, but really bad for baseically integral things like money.

  • Scottford (unregistered) in reply to TSK
    TSK:
    The code is bad and the comments are even worse.

    a) As many people already pointed out: Never use doubles. The point is that binary fractions in decimal representation always ends with 5 or 0. So fractional input data which does not end with these both ciphers will be inevitably mangled. Because something which ended by five divided by two always returns a five you get the unfortunate result that the exact decimal representation of n fractional bits is n decimal digits long ! So a correct decimal representation of a double has in the worst case 53 decimal digits !

    Aren't there decimal numbers that are not exactly representable in binary floating point, no matter how many digits are used? If so wouldn't the reverse apply? (i.e. binary floating point numbers for which there is no exact decimal representation)

  • TSK (unregistered) in reply to Scottford
    Scottford:
    TSK:
    The code is bad and the comments are even worse.

    a) As many people already pointed out: Never use doubles. The point is that binary fractions in decimal representation always ends with 5 or 0. So fractional input data which does not end with these both ciphers will be inevitably mangled. Because something which ended by five divided by two always returns a five you get the unfortunate result that the exact decimal representation of n fractional bits is n decimal digits long ! So a correct decimal representation of a double has in the worst case 53 decimal digits !

    Aren't there decimal numbers that are not exactly representable in binary floating point, no matter how many digits are used? If so wouldn't the reverse apply? (i.e. binary floating point numbers for which there is no exact decimal representation)

    Yes, there are decimal numbers which cannot be represented; all numbers which haven't a common denominator of base 2: 1/3, 1/5, 1/7, 1/9 and their multiples (if they are not canceled as integers). No, there is always an exact (but sometimes very long) decimal representation of binary numbers because base 10 consists of 2*5 and therefore binary numbers can be exactly displayed. A short note: The length of the fraction grows even more large than n digits for n bits if the number gets smaller with the same amount of bits.

  • (cs) in reply to MrBester
    MrBester:
    But if you don't know about the Decimal data type (for whatever reason), what you could have done is run the calculations with ++digits, adding 5 instead of 0.5 and Int() it after the division...
    Tried it, didn't work.

    Making temp a decimal instead of a double is all that is necessary to make the supplied code work. Nothing else "needs" to be switch to a decimal. (Note that this was tried out in VB2005. Which version of VB was the original code intended for? The OP doesn't say.)

    Honestly though, why does everyone say it's a financial thing? Where in the OP was finances mentioned? Are you just assuming because it's rounding to 2 places that it MUST be dollars and cents?

    When I saw the code I thought, "What's the WTF? The way the processor handles floats?" 'cause the code looks WTF free to me, just an honest and understandable misunderstanding.

  • (cs) in reply to Scottford
    Scottford:
    Aren't there decimal numbers that are not exactly representable in binary floating point, no matter how many digits are used? If so wouldn't the reverse apply? (i.e. binary floating point numbers for which there is no exact decimal representation)

    Yes; in fact, there are infinitely many.

    In C, nextafter(double x, double y) and friends are defined in math.h; for any x, let x < y and z = nextafter(x,y); if x, y, and z are representable, all of r in (x,z) are not representable.

    But this is all because IEEE floating-point is a finite space. Real numbers are an infinite space. So, no, there are no numbers that are representable in IEEE floating-point that are not representable in decimal.

  • Josh (unregistered)

    39.995 is actually stored as 39.994999999999997 in a floating point number, so when rounding to 2 decimal places it actually rounds down. The joys of floating point numbers!!!

    Actually the worst part is that different systems will store the same number slightly differently when storing as a float. For instance, Sybase, SQL Server, VB, etc, may store them ever so slightly differently, so that when comparing the 'same' number generated from VB or SQL Server, they don't equal each other (their bits are different). Fun fun.

  • TSK (unregistered) in reply to Josh
    Josh:
    39.995 is actually stored as 39.994999999999997 in a floating point number, so when rounding to 2 decimal places it actually rounds down. The joys of floating point numbers!!!

    To be precise: 39.995 is stored as 39.99499999999999744204615126363933086395263671875 :oD

  • PseudoNoise (unregistered)

    This thread is awesome.

    First, there was Simon Bradley's post which nails the problem, but was an actual engineer-type response so it seems to have been ignored.

    Next, there is Derrick Pallas's post, which notifies me of a libc function I've never seen, "nextafter". My excuse is that I've been working in fixed point for the last 8 years.

    I have a feeling that nextafter won't even solve this problem, hitting a corner case for numbers which are just below XX.5 such that nextafter +.5 will push them over the XX.00 mark rather than leaving them at XX.99999.

    But it's a great function to know about--I've never poked around in the floating-point-twiddling part of libc before. Cool stuff.

Leave a comment on “Round and Round”

Log In or post as a guest

Replying to comment #:

« Return to Article