Is anyone 100% right all the time? No. And are there times where he's actually mad before I pointed it out, Yes there is and he's admitted it. This happens pretty often with him, so thus I get use to the pattern. He had times where he doesn't understand his current emotions that he's displaying and even with his friends, they point it out and he admits that he didn't realize what he was doing/how he was actually feeling till later.
Can I fully put myself into the mindset with someone with Asperger's 100%? No I can't, that's impossible, it's about the same as asking someone with Asperger's to fully put and understand the mindset of someone of NT.
I digress, I understand your points, and yes prehaps there are times where he isn't mad at all and I assume he is from other things or words he uses. Once he explains that he's not, then I just move on with the conversation and that's it.
You're welcome.