Maybe you could learn to make it without the stuff you're allergic to?

I bet there's a tofu coffee ice cream out there.