They just considered themselves Roman, but so did the Holy Roman Empire, and I wouldn't regard that as 'just Rome' either.
They were considered by Western Europe to be the "Empire of the Greeks"; Byzantine Empire as a specific term arose in the 15th century.