It really depends on what security level you want out of a phone

It does, but that's exactly my point. GrapheneOS will provide you essentially no more security than any other alternative Android operating system, should it have to operate on a Fairphone with all those features not supported by a Fairphone stripped away.

Unless Fairphone adds more hardware security features that are standard on most other phones, and highly supported on Pixels, installing a heavily crippled GrapheneOS on a Fairphone would get you essentially none of the benefits of GrapheneOS in the first place.