Mathematics-Online lexicon: Rounding

	[home] [lexicon] [problems] [tests] [courses] [auxiliaries] [notes] [staff]
	Mathematics-Online lexicon:
	Rounding

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

overview

Standard rounding approximates a real number with the nearest even floating point number R. If $\vert x\vert$ lies within the admissible floating point range, then

R $\displaystyle x - x = \delta x,\quad \vert\delta\vert \le$ eps $\displaystyle = \beta^{1-p}/2,$

where $\beta$ is the base and the number of digits of the floating point representation. The constant eps is called the machine precision and equals

$\displaystyle 2^{1-53}/2=2^{-53}$

for double precision.

see also:

Keyword: Floating-point arithmetic

[Annotations] [Examples]

automatically generated 3/ 8/2007