Error Range Plots for x86 FPU Error Data


These plots were with gnuplot 4.6 and data gathered using the included utility. First run gather-data.bat to generate data files. Then run plot.bat to make the plots. Error ranges are plotted in red. Red indicates the 64-bit precision result differs from the correct result by >= 1.0 ulp. When 53-bit precision (C type double) and 24-bit precision (C type float) is used, the error ranges become narrower but to not disappear.

Here are the results from Intel Core i7-2600 and AMD A6-3650. The AMD transcendental instructions have behavior similar to those found in Intel processors, though the non-trig AMD transcendental instructions are slightly more accurate.