Biggest issue is that Qualcomm, Mediatek and anx others don't use mainline kernels. They all have their ancient forks with garbage code stapled on to run Android. That means you're either using that ancient code through a translation layer (Halium + libhybris, used in UB Touch, Droidian) or try and write drivers that can be merged upstream (postmarketOS).
RISC-V doesn't have any CPUs with similar performance to ARM, still work in progress and not guarantee the manufacturers will care any more about mainline Linux.