Comment 9 for bug 1261048

Revision history for this message
In , Bjbraams-2 (bjbraams-2) wrote :

Created attachment 91706
Calc spreadsheet to demonstrate rounding errors for large argument values

This report concerns LibreOffice Calc; functions roundup, rounddown, trunc, ceiling, floor, int, even and odd.

A minimal set of instructions to reproduce the core of the problem follows. Open a new blank spreadsheet. In the A1 cell enter "=power(2,50)+1" (without the quotes) and in the B1 cell enter "=rounddown(a1,0)-a1" (without the quotes). Expected result: B1=0. Actual result: B1=5.

More generally, let y=power(2,50)+1. I expect that rounding y to integer will return y exactly, for any of the functions roundup, rounddown, trunc, ceiling, floor and int. I also expect that even(y)=y+1 and odd(y)=y.

Actual results: functions roundup(y,0), rounddown(y,0), trunc(y,0), ceiling(y,1), floor(y,1) and int(y) all return y+5. Moreover, even(y)=y and odd(y)=y+6 according to Calc.

(Note that the second argument to roundup, rounddown and trunc indicates number of places behind the decimal symbol and the second argument to ceiling and floor indicates the unit.)

There are similar problems with these rounding functions when rounding to some number (>0) of places behind the decimal symbol.

The attached spreadsheet illustrates the issue. I have highlighted in yellow the cells where the result is different from what is expected. Row 5 shows the precise results described here; other rows show similar calculations for values y=power(2,x)+1 for other values of x.