IEEE 754 Floating-Point Errors and Alternatives in Java

In July 2011, a grade-processing error in NEIS, the next-generation National Education Information System of South Korea, changed the end-of-semester class ranks of about 29,000 students at 823 high schools nationwide. It happened just before the early-admission season for universities, so the impact was large.

NEIS computed a semester total by adding written exam scores and performance assessment scores. The ranks of students with the same total were decided by a tie-breaking rule that each school chose from 45 options (Yeongnam Ilbo article, in Korean). For example, at a school that gives priority to written exam scores, student A with 65 on the written exam and 25 on performance assessments ranks ahead of student B with 60 and 30, even though both have a total of 90. Detecting ties is an important step in computing ranks; schools even had to choose a tie-breaking rule in advance.

NEIS was programmed to display totals to 16 decimal places, and some values had a stray '1' appearing irregularly in those decimal places. This is the same pattern as 1.03 - 0.42 producing 0.6100000000000001 at the beginning of this article. For example, if two students' totals are computed as 90 and 90.00000000000001, the values are not equal, so a direct comparison treats the two students as not tied and sorts the one with the error as having a higher score. As a result, students who should have had the same total were not detected as tied, or the ranks among tied students were reversed (notice from the Ulsan Metropolitan Office of Education, in Korean). The Ministry of Education, Science and Technology explained that the bug was caused by not correcting this calculation error (Kyunghyang Shinmun article, in Korean).

jshell shows that adding the same numbers in a different order can produce a different sum. The following is not NEIS’s actual formula, only an illustration of the principle.

Both sums add the same three numbers and should be equal, but with double they produce different values and are not detected as a tie.

According to the Ministry’s count, the ranks of 29,007 students at 823 high schools changed, and for 2,416 students at 350 of those schools, the rank grade changed as well (Jeju Ilbo article, in Korean). Schools had to recompute grades with an error-correction program and reissue report cards (Kyunghyang Shinmun article).

The results of an inspection the Ministry’s special task force announced in September of the same year identify the type in which the error occurred. According to the report, while the old NEIS programs were being redeveloped for the next-generation NEIS, the team failed to anticipate arithmetic errors with floating-point (Double) data that arise from the characteristics of the newly installed database (DB2), and this caused the grading errors (Electronic Times article, in Korean). In other words, grades were computed with the DB2 DOUBLE type. The DOUBLE type in Db2 for Linux, UNIX, and Windows is 64 bits with the same range as IEEE 754 binary64, so it has the same kind of error as Java’s double. Some programs were missing the 'error correction code' that was supposed to fix that error, and the ranks were reversed in those programs (Yonhap News article, in Korean).

At the time, network developer Lee Kyung-moon pointed out in a DailySecu article (in Korean) that the problem was using floating point where exact numeric representation was required. He argued that rather than correcting the error, the design should have switched to integer processing or a type that supports arbitrary precision. Integer minor units and BigDecimal, introduced later in this article, are exactly those two options. The incident shows that handling values with decimal arithmetic, such as grades, in floating point can break comparisons such as tie detection.

The fact that 0.1 cannot be represented as a finite binary fraction has also cost lives. On February 25, 1991, during the Gulf War, an Iraqi Scud missile struck a US Army barracks in Dhahran, Saudi Arabia, killing 28 American soldiers and injuring 99 (US Army article). A Patriot missile battery was deployed at the base to intercept such attacks, but it did not even attempt an interception.

Chapter 1 of the Korean book Software Errors in History (Acorn Publishing, 2014), titled "28 Lives Taken by an Error of 0.000000095," explains the cause of the incident based on the US GAO’s investigation report. The Patriot system counted time in tenths of a second to predict where a target would appear, and converted the count to seconds by multiplying it by 0.1 stored in a 24-bit fixed-point register. Because 0.1 is an infinitely repeating binary fraction, everything beyond 24 bits was truncated, and an error of about 0.000000095 seconds accumulated every tenth of a second. After 20 hours of continuous operation, the computed time was 0.0687 seconds behind the actual time; after about 100 hours, as with the battery at the time of the incident, it was 0.3433 seconds behind. By the GAO report’s calculation, this timing error shifted the range gate where the system looked for the target by 687 meters. The tracking radar searched an area that far from the missile’s actual position. The radar found no target there, so no interceptor was launched, and the Scud hit the barracks.

This system used fixed-point arithmetic, not floating point like double. But the principle is the same as in this article: 0.1 could not fit into a finite number of binary digits, and the error accumulated.

The standard way to handle decimal fractions exactly in Java is java.math.BigDecimal. Why You Should Never Use Float and Double for Monetary Calculations also recommends BigDecimal instead of floating-point types for monetary calculations.

There is one catch. If you pass a double to the constructor, the rounding error introduced when the decimal fraction was stored in the double carries over into the BigDecimal. As shown earlier, new BigDecimal(0.1) holds the approximation stored in the double, not 0.1. To avoid this error, use the constructor that takes a string, or BigDecimal.valueOf(). BigDecimal.valueOf(0.1) builds the value from the string "0.1" returned by Double.toString(0.1), so it is 0.1. However, valueOf() cannot remove an error that already exists in a value computed as a double. BigDecimal.valueOf(0.1 + 0.2) is 0.30000000000000004.

Using BigDecimal does not decide the rounding policy for you. A division whose decimal expansion does not terminate, such as 1 divided by 3, throws an exception unless you specify the scale and the rounding mode.

java.math.RoundingMode, used here, is an enum for choosing how to round: whether to round 0.5 up (HALF_UP), to the nearest even number (HALF_EVEN), or to always discard the fraction (DOWN). For example, rounding 2.5 and 3.5 to integers gives 3 and 4 with HALF_UP, 2 and 4 with HALF_EVEN, and 2 and 3 with DOWN. What BigDecimal eliminates is the error that comes from binary representation. How many decimal places to keep and how to round them is a rule the team must decide and follow according to the needs of the domain.

Joshua Bloch makes the same recommendation in Item 60 of Effective Java, 3rd edition, "Avoid float and double if exact answers are required." The item suggests int and long as well as BigDecimal; the section on choosing between them below covers which to pick.

Handling amounts as integers in the smallest unit, such as won or cents, is also common. The API of the payment service Stripe is an example. The amount attribute of the Charge object is an integer, described as follows.

This is especially practical for the Korean won, where amounts below 1 won have little value, as long as it does not conflict with business rules. For the same reason, existing systems in Korea often define won amounts as integer columns in the database. So even if the application calculates with BigDecimal, there is at least one conversion to an integer right before the value is stored. An example from the Woowa Brothers tech blog post Writing Test Code with Spock (in Korean) shows this approach. It uses BigDecimal for rounding, but the amount parameter amount and the return type are long.

The amount itself stays an integer, and right after the rate calculation, which produces a fraction, the method returns to an integer through a BigDecimal with an explicit rounding mode.

However, once a floating-point type is mixed in, as in amount * rate * 0.01, the calculation becomes floating-point arithmetic. The long value is converted to float and the multiplication is done in float, losing precision, so errors can occur even though the amount parameter and the return type are integers.

The first call multiplies by 100%, so it should return 16,777,217 unchanged, but the result is 1 won short. A float has 24 bits of precision (a 23-bit fraction plus the implicit leading bit), so it cannot represent every integer above 2²⁴ (16,777,216); 16_777_217L becomes 16,777,216 the moment it is converted to float. The second call should return 335,545, which is 50% of 671,089 won (335,544.5 won) rounded, but it is 1 won short. The amount is less than 2²⁴, but the product with the rate, 33,554,450, exceeds 2²⁴ and is stored in a float as 33,554,448. Even if the inputs stay integers, a single floating-point step in the middle of the calculation brings the error back.

If you modify the quoted example to move the rate into BigDecimal as well, the result is 16,777,217 as expected.

The longValueExact() call on the last line throws an ArithmeticException if the result exceeds the range of long. In that case, longValue() silently returns only the low-order 64 bits, so a wrong amount could be stored.

Even with integer minor units, eliminating errors requires keeping every intermediate value as an integer or a BigDecimal.

Java Money is an API for handling monetary amounts together with currencies, standardized as JSR 354. It represents an amount and its currency together with the MonetaryAmount interface. The JCP proposal considered including it in Java SE 9, but it was never added to the JDK. To use it, you add the API (javax.money) and its reference implementation, Moneta, as separate dependencies.

The Money implementation calculates with BigDecimal internally, so 1.03 - 0.42, which gave 0.6100000000000001 with double at the beginning of this article, comes out as exactly 0.61.

Arithmetic between amounts in different currencies throws javax.money.MonetaryException. The API stops the mistake of mixing amounts in different currencies with an exception at run time.

Java Money does not replace the two approaches above; it sits on top of them. Two of the main MonetaryAmount implementations in Moneta map directly to the two numeric representations discussed in this article.

Because they share an interface, you can swap Money for FastMoney if speed becomes a problem. Even with a domain type, you still need to decide on the numeric representation, and the trade-offs from the previous section reappear as the choice of implementation.

In my view, whether to adopt Java Money depends on whether you handle multiple currencies. With a single currency, plain BigDecimal or long values are enough; in a system that mixes currencies, having the API catch currency mismatches is worth the extra dependency. For more usage examples, see Java Money and the Currency API on Baeldung.

Java Money is not the only library that wraps numeric representations in domain types.

Even if none of these libraries fits, a type you build within your own system can play the same role. In that case too, decide whether to hold the number as a BigDecimal or an integer by the criteria described in this article. And make sure float or double cannot slip into intermediate calculations.