Troubleshooting Data Types
The following is a list of common problems that can occur when working with intrinsic data types.
Floating-point expressions not comparing as equal
When you work with floating-point numbers (Single, Double, Real, and Quad), keep in mind that they are stored as binary fractions. This means they can hold an exact representation of a quantity only when it is a binary fraction (of the form k / (2 ^ n) where k and n are integers). For example, 0.5 = 1/2 and 0.1875 = 3/16, thus these represent precise values. In contrast, 0.3 = 3/10 and 0.15 = 3/20, thus these represent approximations.
As a consequence, you cannot rely on exact values resulting from floating-point operations. In particular, you cannot assume that any two given values will be considered equal, even if mathematically they should be.
To compare floating-point quantities, do the following:
- Determine an acceptable maximum tolerance for the given comparison such that any amount within this tolerance is considered equal.
- Place a Tol directly after the end of the comparison expression, and follow that with the chosen tolerance amount, which can be a numeric literal or constant defined by a Const statement.
For more information, see Tol Clause.