language-servers/roc
1
0
Fork 0
mirror of https://github.com/roc-lang/roc.git synced 2025-08-04 12:18:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

Don't say floats have less precision than Dec

Browse source 
It's more accurate to say they have issues with base-10.
In fact, very small floats can have *more* digits of precision
than Dec can!
This commit is contained in:
Richard Feldman 2023-09-17 19:55:18 -04:00
parent cbbf04cb85
commit 83587eca22
No known key found for this signature in database
GPG key ID: F1F21AA5B1D9E43B
1 changed files with 22 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

30
crates/compiler/builtins/roc/Num.roc
View file

@ -470,8 +470,8 @@ F32 : Num (FloatingPoint Binary32)
## are, because [Dec] is represented in memory as base-10. In contrast, [F64] and [F32]
## are [base-2](https://en.wikipedia.org/wiki/Binary_number) in memory, which can lead to decimal
## precision loss even when doing addition and subtraction. For example, when
## using [F64], adding 0.1 and 0.2 returns 0.30000000000000004,
## whereas when using [Dec], 0.1 + 0.2 == 0.3.
## using [F64], `0.1 + 0.2` returns 0.30000000000000004,
## whereas when using [Dec], `0.1 + 0.2` returns 0.3.
##
## Under the hood, a [Dec] is an [I128], and operations on it perform
## [base-10 fixed-point arithmetic](https://en.wikipedia.org/wiki/Fixed-point_arithmetic)
@ -481,16 +481,30 @@ F32 : Num (FloatingPoint Binary32)
## quintillion, along with 18 decimal places. (To be precise, it can store
## numbers between `-170_141_183_460_469_231_731.687303715884105728`
## and `170_141_183_460_469_231_731.687303715884105727`.) Why 18
## decimal places? It's the highest number of decimal places where you can still
## convert any [U64] to a [Dec] without losing information.
## decimal places? It's the highest number of decimal places where you can
## still convert any [U64] to a [Dec] without losing information.
##
## There are some use cases where [F64] and [F32] can be better choices than [Dec]
## despite their lower precision. For example, in graphical applications they
## can be a better choice for representing coordinates because they take up
## despite their issues with base-10 numbers. For example, in graphical applications
## they can be a better choice for representing coordinates because they take up
## less memory, certain relevant calculations run faster (see performance
## details, below), and decimal precision loss isn't as big a concern when
## details, below), and base-10 generally isn't as big a concern when
## dealing with screen coordinates as it is when dealing with currency.
##
## Another scenario where [F64] can be a better choice than [Dec] is when representing
## extremely small numbers. The smallest positive [F64] that can be represented without precision
## loss is 2^(−1074), which is about 5 * 10^(-324). Here is that number next to the smallest
## [Dec] that can be represented:
##
## * Smallest [F64]: 0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005
## * Smallest [Dec]: 0.000000000000000001
##
## This is because [floating-point](https://en.wikipedia.org/wiki/Floating-point_arithmetic) numbers
## like [F64] can gain more digits of precision after the `.` when they aren't using as many digits
## before the `.` - and this applies the most if the digit before `.` is `0`. So whereas [Dec] always
## has 18 digits after the `.`, the number of digits after the `.` that [F32] can [F64] can represent
## without precision loss depends on what comes before it.
##
## ## Performance Details
##
## CPUs have dedicated instructions for many [F32] and [F64] operations, but none for [Dec].
@ -510,7 +524,7 @@ F32 : Num (FloatingPoint Binary32)
## Keep in mind that arithmetic instructions are basically [the fastest thing a CPU does](http://norvig.com/21-days.html#answers),
## so (for example) a network request that takes 10 milliseconds to complete would go on this
## list as about 10000000x. So these performance differences might be more or less noticeable than
## the precision differences depending on the use case.
## the base-10 representation differences depending on the use case.
Dec : Num (FloatingPoint Decimal)
## Euler's number (e)