l was afraid to go because l knew l was going to get lost. Sure enough, l was anxious at xmas party due to what everyone stated here, (noise, eg.) then left party and was lost for an hour and a half. Next year if still working there, l will simply state, sorry, can't go, l always get lost. Also l was feeling pressure to drive someone who l defintely don't like. l adulted up and asked her not to put her hands on me finally. There was a lot of baggage around this party. lol.
I like "sorry, I can't go, I have a prior commitment". (Your prior commitment is your sofa and your favorite TV show, but they don't have to know that!)
One thing I've learned over the years is that pretty consistently, if you give a reason that they can poke holes in at all, they will. So if you say you're doing X, they'll ask if you can reschedule X. Or try to find some way around it. "I have a prior commitment" is both solid, and doesn't give enough information for them to try to wiggle around it.