codersnotes

Learning To Wrangle Half-Floats September 10th, 2016

You all know what floating-point arithmetic is, so I won't bore you by covering that. The IEEE standard originally defined two variants, the 32-bit single-precision format and the 64-bit double-precision format.

But that's not all you can do. If you understand the principles behind it, you can make your own floating-point format at any precision. The most popular small-float format is the 16-bit half-precision format. Popularized by Nvidia and ILM, this is supported in hardware by most GPUs.

The half-float format is great because it's good enough for many cases, while only being half the space of the standard 32-bit format. It's not just the space either -- the PS3 GPU, for example, would often run twice as fast when using halfs. (Interestingly enough, this usually wasn't due to the precision difference, but to restrictions on register file access. The smaller data access allowed the compiler to better schedule the instructions.)

There's a downside to this flexibility though. With regular FP, you can usually just throw it in there and not have to worry about precision. That's no longer true for half-floats. Every time you use them you now have to worry about whether it's suitable for the current case. And, as you may discover here, the results can be surprising.

The format is very simple; it's basically the same as the 32-bit version but with less bits:

Sign	Exponent	Mantissa
1 bit	5 bits	10 bits

The Wikipedia page has a good detailed explanation of it, but the trouble with just running the numbers on it is that you don't really get a good feel for understanding it. We need a better way of grasping the fundamentals.

To get a good visualization of half-float, the most useful property we can use is the fact that they're only 16-bit. This means that there's only 65536 of them. You know, that's not actually that many. So, why not just list them all out? I did just that. That's the great thing about computers today, data doesn't seem as big as it used to be. Once you have all the data in front of you at once, it's much easier to get a grip on it.

This text file has come in very useful for me on several occasions, and I'd recommend keeping a copy of it around for any time you're doing graphics work. Let's see what we can discover from this. We'll start by making a simple table, showing off the ranges at which the precision changes:

Exponent	Starts at	Step between each number
0	0	1/16777216
1	1/16384	1/16777216
2	1/8192	1/8388608
3	1/4096	1/4194304
4	1/2048	1/2097152
5	1/1024	1/1048576
6	1/512	1/524288
7	1/256	1/262144
8	1/128	1/131072
9	1/64	1/65536
10	1/32	1/32768
11	1/16	1/16384
12	1/8	1/8192
13	1/4	1/4096
14	1/2	1/2048
15	1	1/1024
16	2	1/512
17	4	1/256
18	8	1/128
19	16	1/64
20	32	1/32
21	64	1/16
22	128	1/8
23	256	1/4
24	512	1/2
25	1024	1
26	2048	2
27	4096	4
28	8192	8
29	16384	16
30	32768	32
31	infinity/nans

There's quite a few surprises nestled away in here. Perhaps the most shocking is the extreme precision loss at the high end. After 32768.0, you're stepping over 32 integers at a time! Even as low as 1024.0, you're still stepping 1.0 each time. Just to ram that point home, numbers higher than 1024 lose all fractional precision.

The maximum half-float possible is only 65504. That's not very big for many applications. And even at that range, you're only accurate to +/- 32.

Thinking of storing UV co-ordinates at half precision? Think again. At the 1.0 range our halfs are only accurate to 1/1024. For a 4096x4096 texture that means they're only accurate to every 4 pixels.

Trying to store a displacement map at half-precision? If it's in the 0-1 range, you're effectively only getting the same accuracy as a 10-bit format. That might be OK for a simple effect, but don't try it for a heightfield.

To summarize, while half-floats are great and you should use them whenever possible, you have to check your range first. How much precision do you require? It's easy to assume that a floating-point format will just magically give you everything you need, but it's not always so. Once you get outside the 0-1 range, half-floats lose their appeal for many cases.

© 2022 Richard Mitton. All rights reserved.