I can sort of relate, sometimes my mind will be elsewhere and then someone will distract me and start talking to me but then I cannot focus on what they are saying and get upset by it even so I just repeat 'I don't know', 'I don't know' etc. If they keep talking it makes things even worse, my head gets too cluttered and I get annoyed with them and the situation and just have to get away from it. But even worse when it does happen it is usually at work and I get accused of being rude to them. But often it happens when they try to make small talk with me or ask me questions about myself that I do not like answering so hence I get stuck because I am no good at small talk and it's hard to get out of answering questions.