Your conversational philosophy is a lot like mine. That's a relief - I was thinking I was the odd one out.
I don't always talk to people because often times they talk about things I have no interest in, or that actually make me a little uncomfortable or angry. Reactions inside my head can range from "this again? Bleh, I don't care." To "If you DON'T STOP talking in the next MINUTE I'm going to rip my eyes out and THROW them at you JUST TO MAKE YOU SHUT UP!!!"
However, I've become so good at acting that I can usually feign interest convincingly (although thinking of it, even then I seem to use "stock" responses that seem appropriate). I know I shouldn't be passive aggressive, but if you tell people the truth about your feelings, they want to interrogate you and discuss them or just write you off as rude, and it's so much easier to nod your head and agree. Anything for a quiet life. :/
Hypocritically, if I get onto one of the things I like talking about, I'll go on a mini rant/lecture about it whether they like it or not. xD
I don't feel like I lack social SKILLS - I can act as normally as any NT, and am less shy than most, especially since I got married (and was no longer auditioning women as potential mates to impress) - I'm just lack social ENTHUSIASM. Being alone is a wonderful, rare, and spiritually refreshing occasion I wish I could do everyday.