40
Should you normalize RGB values by 255 or 256? (30fps.net)
submitted 3 months ago by to c/programming@programming.dev
22 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 16 points 3 months ago* (last edited 3 months ago) (7 children)

This is a stupid debate. Of course you should use 255. Everybody uses 255.

The standard algorithm means that if you go from int(0-255) to float, black stays black and white stays white. The silly new one turns the extremes very slightly grey.

This needs stomping on, because inventing a conflicting standard where there's currently only one is really really stupid.

If you actually want an even distribution of random colours (who on earth wants that?) then instead of generating random floats, generate random int(0-255). It's not hard. It really isn't.

I bet you that float values that round to 0 come up in float colour data way, way, way more frequently than values that round to 0.5/256, nullifying the silly mathematical point that you might get 0 less often. No. No you won't. Not in real data.

  • permalink
  • fedilink
  • source
  • hideshow 7 child comments
    •
  • [–] 4 points 3 months ago (6 children)

    If you’re generating random numbers you apply the correction to your random numbers: [0, 1) is a bin for zero, and [255,256) maps to 255. If you are sampling as such, you are responsible for applying the correction as part of your quantization.

    256 never comes up, because it cannot in the 8bit quantization.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 6 points 3 months ago (5 children)

    Yes, indeed. If you're randomly generating colours and you want to evenly cover the {0,1,...,255} integer space, then generate a random integer in that range directly, or truncate a random float in [0,256). Generating a random float in [0,1) and converting it afterwards, then complaining you're not getting enough 0 or 255 is silly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 3 months ago (4 children)

    The random number generation is not the source of the complaint. That is just made to easily prove the point in the article.

    The argument is not to use another method when randomly sampling from [0,1). The argument is that f32 color channels are already in the [0,1) range, and following the alternate method results in a "fair-er" representation of the range.

    The plot that was generated with the random sampling is just to show that the standard method is not "fair".

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 3 months ago (3 children)

    I don't think we're going to agree on any points whatsoever, I'm afraid.

    I think their method is pointless, that introducing new formats that are indistinguishable from well established ones is actively harmful, that their reasoning is flawed and raises mainly irrelevant points, and their only valid point is that fractions with denominator 256 are stored exactly, but then if the whole point of converting your colours to floats is to do interesting transformations of the colour space that you can't do with integers, that's all gone as soon as you start manipulating the floats, and all your carefully designed exactness went down the pan. If you want exact, keep it to the integers 0-255.

    Neither[-0.5/255) nor [0,0.5/255) are some sort of disadvantaged and discriminated against group of humans that need more representation and I don't understand why on earth anyone would care even a tiny little bit if they were under represented nor why you would describe it as unfair.

    The whole thing is really, genuinely deficient in benefits and has obvious drawbacks.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 3 months ago (2 children)

    I'm trying to understand where this rage and hatred is coming from. This isn't some canonical interchange format here.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 months ago (1 child)

    It breaks convention to solve an entirely nonexistent problem.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    If you read the conclusion, it is very clear that you should not ever break convention. You should not apply this if it would affect any other program.

    That is, if your input is in u8, your output should be in u8 too. Otherwise you are "exposing" your different quantization.

    I don't think this post is telling you that you should use the 256 method. I think it is pretty clear that it is just explaining it, not advocating for it. At various points of the article it is written something like "I struggle to see a situation where it would be beneficial".

  • permalink
  • fedilink
  • source
  • parent