I am not naive to think that we can end factory farming tomorrow. Like you said, it's something that will happen gradually (but not too slow I hope). I am not aware of any animal product for which there aren't already alternatives. Do you have examples?
As for the bivalve discussion, while they do not have brains, the do have a very limited nervous system that can react. For example, oysters close up in the presence of predators like crabs. I think it is an individual decision of where to draw the line in this circumstance. I do not eat bivalves.
As for the sustainability aspect, I do not think they would be more sustainable than plant based alternatives like soy and lentils, especially given the existing infrastructure. To harvest bivalves in the quantities needed if you removed all other animal for sources would likely be devastating for the ecosystem. Meanwhile, 83% of agricultural land use goes to animal feed. Yet animals account only for 37% of consumed protein and 18% of consumed calories. That means that if we convert all the land usage dedicated for animal feed to human crops, we would have more than plenty for all humanity without even using all the currently used land.
Source for the above percentages. Direct pdf link to the paper above.