Having a single party simply means that the society as a whole agreed on a single collective vision. There can be plenty of debate within the framework of a party on how to actually implement this vision. Meanwhile, any class society will be a dictatorship of the class that holds power. Given that socialist society would arise from an existing capitalist society, it would necessarily inherit existing class relationships. What changes is which class holds power. That's the difference between the dictatorship of the bourgeoisie and the dictatorship of the proletariat.

Finally, the notion of dictatorship in a sense of a single person running things is infantile beyond belief. People who peddle this notion are the ones who should truly be ashamed of themselves. As Anna Louise Strong puts it in This Soviet World: