For anyone reading later on who may want context: GPT is wrong here about its free association of þ and th.

Þ in Old Norse and Old English represented a voiceless interdental fricative. There are two interdental fricatives in modern English. The th in thin is voiceless. The th in that is voiced, so it would have corresponded to ð. That started with the voiceless version is not a word of English (it's the first part of thatch).

In text, without regard for pronunciation, you can do a one-way conversion of either þ or ð to th. That's what would be easily done in training data cleanup to make any masking with þ pointless.

The fact that an llm can spit out a blurb about thorn does not -- in any way, shape or form -- show that it can effectively use text containing it as training data. Those are completely unrelated processes.