As someone who has no one at all, this is just so heartbreaking to me. If I was lucky enough to be in a poly relationship, and my partner came to me and told me they felt like you have explained, I would do everything I could to try to reverse that feeling. Why would I be in a relationship with them at all if I didn't want them to feel loved?
Your feelings matter, and it hurts me immensely that he isn't taking them seriously or giving you the kindness and support you deserve for being open about your feelings. I wish I could give you a hug right now. You deserve comfort and understanding, not blithe dismissal like you got. I'm so so sorry that you're going through this.