So... many "normal" people have a right to be total creeps when they are angry because they know how it fits in to the NT world. I never act on my anger because I don't know how to do that. I often will set a boundary around the offending behavior then cut a person from my life if they continue to be abusive. I use the three strikes rule. Ignore or violate my boundary three times, then that person is out of my life. I always used to feel "meh" because I had so many of them. It was a long term low level depression. I have only started lifting that through removing them from my life.
It sounds like you deal with things in quite a good way. Not that anger is necessarily a bad thing. It depends on how it is expressed of course.
I learned about boundaries in a conscious way only in my forties - very late

I think you've done the right thing limiting those people who are boundary busters. Some people just won't be told. It's like they can't help themselves, they have to do the thing you've said you won't accept.