40
Should you normalize RGB values by 255 or 256? (30fps.net)
submitted 3 months ago by to c/programming@programming.dev
22 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 months ago* (2 children)

Did you read the article or just the title?

I'm honestly surprised this comment is the one that shows at the top.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 3 months ago* (last edited 3 months ago) (1 child)

    So, it's basically saying that anything less than 1/256 (0.00390625f) is 0x0.

    Anything equal to or over 0.99609375 (255/256) should be 0xff.

    Divide by the count (256), not the max index (255). I'm not sure the 0.5 offset is really needed.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    The offset is needed because this quantization dividing by 256 uses truncation whereas by 255 uses rounding.

    So a value of 245 (for example) could be anything between 245.00000 and 245.99999. Therefore, when dequantizing, we set the value to 245.5, which is in the middle. Between 245.00000 and 245.99999.

  • permalink
  • fedilink
  • source
  • parent