That's not strictly true, the voltage of the battery changes as it discharges, so you technically need to integrate the area under IV curve.

Yes but a fully charged battery will also have a voltage higher than its nominal voltage which mitigates that quite a bit. Also as I said in my origional comment, the rated capacity is only ever a rough measurement anyways so while volts×amphours=watthours isn't perfect, it's close enough.

or anything that let's us compare across voltage

I guess the part I'm missing is, when are you ever really comparing across voltage. Generally when you're shopping for batteries it's for a specific application and that application will have a specific voltage requirement. So you're only ever really going to be comparing batteries of the same voltage anyways. It isn't like you can just toss a 48V battery in something that requires a 6V battery.

Although I do agree that a voltage agnostic capacity measurement would be nice. My personal preference would be watt hours.