Many Aspies mark the whole NT population as "extreme social creatures with no interests other than socialising". But, in reality, they don't all live their lives like that.
I'm a sociable person and I like people, but I can be shy too, but I tend to force myself to attend social situations (not that I don't like social situations) because if I don't I feel like I'm being autistic. But many NTs avoid social situations too, even if they seem extroverted. So it doesn't always mean you're autistic just because you would rather stay home or do your own thing than go to a party.
My husband's daughter is an extroverted NT, and I've met her boyfriend a few times, who also seems like an extroverted NT. I know he's an NT (I can usually instinctively sense when someone may not be an NT, especially in males, even if they're highly social and seem normal, but with him I can instinctively tell he's not autistic or other ND type at all). But he often avoids social gatherings and my husband's daughter usually turns up to family gatherings alone without him. She doesn't mind. I once asked her if he was shy and she said he's not shy but just social gatherings aren't really his thing so often makes excuses to avoid going, preferring to pursue his hobby instead, which is fishing.
So you don't have to be autistic to not jump at the invite of a party. And I need to stop being embarrassed about my shyness and just sometimes openly admit that I'm not the life and soul of a party and that sometimes I don't always have to attend a party if I don't want to. I do enjoy socialising though, but at the same time I do get fidgety and restless at parties. I prefer to go to parties where you don't need to wear smart or "dressy" clothes. I'm going to openly admit that I don't like going out for work dos at Christmas or any other time of the year.
Saying no to a party doesn't mean you're autistic, or it won't make people immediately go "oh, he/she must be autistic because they don't always attend social gatherings".