40
Should you normalize RGB values by 255 or 256? (30fps.net)
submitted 3 months ago by to c/programming@programming.dev
22 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 months ago* (last edited 3 months ago) (1 child)

So, it's basically saying that anything less than 1/256 (0.00390625f) is 0x0.

Anything equal to or over 0.99609375 (255/256) should be 0xff.

Divide by the count (256), not the max index (255). I'm not sure the 0.5 offset is really needed.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    The offset is needed because this quantization dividing by 256 uses truncation whereas by 255 uses rounding.

    So a value of 245 (for example) could be anything between 245.00000 and 245.99999. Therefore, when dequantizing, we set the value to 245.5, which is in the middle. Between 245.00000 and 245.99999.

  • permalink
  • fedilink
  • source
  • parent