Why is this happening?

Anything is possible. Since you have so many negative feelings, you probably should call this thing off. You sound mature enough to know what to say and how, and still be kind. I understand how you feel about loud people, as I have the same problem and try to avoid them. You can't very well avoid someone you're having a relationship with. It doesn't sound as though the two of you are meant for each other, and that you may need someone who is more mental than the other way. Maybe you know what I'm getting at. Anyway, good luck to you.