To explain a bit more, most of the information in an image is actually stuff we would never notice. The very specific way a few of a pixels slightly deviate from a perfect gradient, for example.

The basic idea of a jpeg is to convert an image to a kind of frequency space, where each pixel corresponds to a specific wave-like pattern. It's reversable, preserving the information. and fast conversion to do, thanks to FFT. Even without, it would only be quadratic time. Then, the algorithm crops it in that new space. (IIRC jpeg actually has a few extra features, as well)