Not arbitrary at all, if you love institutionalized pedophilia

I wouldn't say that letting Catholics practice their sacraments is arbitrary. The idea of religious freedom, as understood at the time the Constitution was written, was developed in large part as a reaction to two and a half centuries of brutal religious war and oppression between Protestants and Catholics (and between different Protestant sects) with non-Christian religions as an afterthought. Therefore, the right of Catholics to practice their religion is a central example of what is protected by the First Amendment. Religious freedom isn't absolute and where the line is drawn is to some extent arbitrary, but Catholicism is well within any reasonable line.

Edit: I forgot to mention that the secrecy of confession is also required by the Episcopal Church. It was definitely on the minds of the authors of the Constitution, most of whom were Episcopalians.