mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-02 16:13:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

Remove long integer output.

Browse source
This commit is contained in:
Georg Brandl 2008-08-10 12:16:45 +00:00
parent 3d1c7dec4a
commit bae1b94d9e
1 changed files with 6 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

12
Doc/tutorial/floatingpoint.rst
View file

@ -173,24 +173,24 @@ and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
the best value for *N* is 56:: the best value for *N* is 56::
>>> 2**52 >>> 2**52
4503599627370496L 4503599627370496
>>> 2**53 >>> 2**53
9007199254740992L 9007199254740992
>>> 2**56/10 >>> 2**56/10
7205759403792793L 7205759403792794.0
That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The
best possible value for *J* is then that quotient rounded:: best possible value for *J* is then that quotient rounded::
>>> q, r = divmod(2**56, 10) >>> q, r = divmod(2**56, 10)
>>> r >>> r
6L 6
Since the remainder is more than half of 10, the best approximation is obtained Since the remainder is more than half of 10, the best approximation is obtained
by rounding up:: by rounding up::
>>> q+1 >>> q+1
7205759403792794L 7205759403792794
Therefore the best possible approximation to 1/10 in 754 double precision is Therefore the best possible approximation to 1/10 in 754 double precision is
that over 2\*\*56, or :: that over 2\*\*56, or ::
@ -211,7 +211,7 @@ If we multiply that fraction by 10\*\*30, we can see the (truncated) value of
its 30 most significant decimal digits:: its 30 most significant decimal digits::
>>> 7205759403792794 * 10**30 / 2**56 >>> 7205759403792794 * 10**30 / 2**56
100000000000000005551115123125L 100000000000000005551115123125
meaning that the exact number stored in the computer is approximately equal to meaning that the exact number stored in the computer is approximately equal to
the decimal value 0.100000000000000005551115123125. Rounding that to 17 the decimal value 0.100000000000000005551115123125. Rounding that to 17