They detected allozymes (differences in proteins) by electrophoresis in the 1970's.

This could tell the difference between species and maybe if they were lucky large family groups. It wasn't as exact as using DNA.