Interesting similarity there. I majored in music (woodwinds) and taught private lessons & played in ensembles. The teaching was one-on-one and I was a musician so almost expected to be "weird" so that worked out. For a while. I didn't mind the orchestral ensemble stuff, which can be as loud as any rock band but the sound is so "pure" that it doesn't seem harsh (a bunch of good trained musicians playing in tune and clear tone doesn't hurt either). I tried playing in a couple bands but I couldn't stand the chaotic loud noise.
After a few years (about 12 I think) I gave that up and became a machinist, where the unusually high ability to hear nuanced audible differences with high resolution is an advantage you wouldn't think even existed in that field, i.e. my acute hearing has helped me be a better machinist. Early on I tried explaining this to others, but whether from the oddity of it or my lack of ability to articulate it, nobody ever "got it" so I shut my yapper. It's a pretty solitary activity and very detail-oriented, so a pretty good fit for my abilities and faults.
Anyways, after I got out of music I pretty much never looked back. I haven't even taken my "pride and joy" clarinets out of the case for over 20 years. But I picked up playing drums and even though I suck at it, it's still fun and the volume doesn't bother me. I only play at home along with recorded music or solo. The idea of playing with others seems fun as a concept, but I know better than to believe I might enjoy it in reality. I never do it nor seek it out and since I'm old now, I don't care what other people think - one of the things that comes naturally with age. So I just entertain myself as best I can and things have kinda worked out.
So answer to the O.P. is "yes". For me it's not volume which bothers me, but a wide variety of sounds really grates on my brain like I'm walking on glass while others, whether at low or quite high volumes depending, is curiously soothing. What I find soothing vs. what is annoying is all over the map for me so I can't make much sense of it, and just try to go with the flow... that and mostly stay busy at home by myself when I'm not at work.
I have a hypothesis that harmonic/formant singing (like the Gyuto monk choir and others) might be soothing to anyone's ears. I'm certain that's not correct because brains are so different and many people probably liken that sound to brain splinters instead, but when I first started really paying attention to my sound sensitivity, that kind of vocalization was one of the most soothing sounds to my mind for whatever reason. Kind of magical. In the meantime, playing cool things on a drum will have to do.
I am aware that "I could have been a contender", i.e. my bare I.Q. should have resulted in a higher type of profession in life, but I'm satisfied where I have settled. Another thing that comes with old age if you're lucky is an appreciation of the good things in life, or maybe more like an appreciation of the problems you were spared from. If I was a bumper-sticker kind of person, mine would probably just say "meh".
p.s. I've had pretty strong tinnitis my whole life. Sometimes it's seriously annoying but most of the day I guess I'm focusing elsewhere enough that I don't notice it. Anyone else?