In catholic literature (and I think protestant) demons can understand and be repelled/summoned by phrases in Latin. A common symbol for Satan and demons are black goats and goats in general. So the joke is that he is messing up latin and inadvertantly summoning demons into his house.