Exactly, Icelandic is basically old norse with a spelling skin and different vocabulary. Most people take an least one class which involves reading old norse like Hávamál. https://en.wikipedia.org/wiki/H%C3%A1vam%C3%A1l?wprov=sfla1

How similar? Here's the first stanza of Hávamál translated by me just now. All the words are common. Other places like Snorra Edda is also easily readable although Snorri used less common words so there's more vocabulary to memorise like "röðull" for sun and "gumi" for man.

Gattir allar, aþr gangi fram, vm scoðaz scyli, vm scygnaz scyli; þviat ouist er at vita, hvar ovinir sitia a fleti fyr[b]

Gáttir allar, áður gangi fram, um skoðast skal, um skyggnast skal: því að víst er að vita, hvar óvinir sitja á fleti fyrir.