NT woman(46) newly dating ND man (39). Misread text and showed up at his house in announced. Said it was ok, now has almost completely withdrawn

Hi. Things were coming along very well. We met on a dating app a few months ago and have been actively dating and talking daily since. Until last week. He is ND and had ADHD. He was very diligent about contacting me daily, via text mostly, phone or FaceTime. I also reached out to him. We have been on 4-5 formal dates and have hung out at his house 2-3 times. Since it’s so early (2.5) we haven’t had any serious relationship talks, still feeling each other out. From the beginning he said that he wanted us to pace the dating (which I happily agreed to) and he also emphasized the importance of building friendship. We have similar taste in sneakers, music, clothes, both sober, spiritual and religious, love the same movies, tv shows, and have a plethora of other similar interest. I felt like I met my perfect match. He communicates very directly, so any time he had an issue, he would very bluntly address it. He’s also very sarcastic and loves to joke. At first this was off putting because I’m a little overly sensitive, but now I understand he picks on me because he likes me. I am diagnosed with OCD and GAD and on meds and in cognitive behavioral therapy for it. I tend to overthink and ruminate a great deal. Last week , he sent a string of slightly vague texts asking if I worked the next day , how long I’d be up and then mentioned that he was going to ask me to “pull up” ( come over) but he was working and wasn’t sure how long he would be up after that. I replied was he almost finished with his work and that I could have been halfway there by now but it was his call. He replied no he wasn’t almost done, to which I texted well… I’ll see you soon. He replied with a “ha ha” about 10 minutes later and I erroneously took that as an invite. I did not call or clarify in text and he never said don’t come so I took that as an invite. Big mistake. At this point, it’s 11:15 pm. I’ve been over to his house late before, so it wouldn’t be uncommon. I headed over and when I got there I texted him “knock knock”. I walked to the front door and called him. When he came to the door he was shocked/surprised and said “you’re here? I didn’t say come over.” I was so embarrassed I almost cried. I said omg I’m so sorry I feel so stupid, let me go home. He replied, no you don’t have to come in”. He then goes into all the what ifs, what if he had had company and I took it the wrong way, what if he wanted to clean the house before I came, what if he just wanted to work alone.? I knew he was working so I had even brought my own work as to not disturb him. He said you could have called and we could have settled this in a 3 minute call. I attempted to leave 2 more times but he insisted I stay. We talked for 2 hours. Things seemed fine. He assured me it was ok, I made a mistake and now I know. He said next time I better show up in a trench naked underneath lol. He hugged me and kissed me on the forehead and walked me to the door, watched me get into my car. He told me to text when I got home and he would stay up until. I texted him and went to sleep. While I was there, he told me he was doing work that was due 2-3 weeks ago, so when he didn’t text me good morning the next day, I didn’t think much of it. We are both up late and id slept most of the day. I texted him around 3pm to no reply. I texted again a couple hours later to share some new sneakers, still nothing. I let it go. The next day he didn’t text, nor the day after that. By the fourth day I was a little panicked. I texted him to say I hope he was ok, I hadn’t heard from him in a few days and that I was just checking in. Then I followed up with another apology for the pop up, letting him know again that I would never just show up at his house. He finally responded the next evening via text, arresting the apology “no worries, it’s all good it happens. Then he asks me to send him my address when I get a minute. This caused me to panic even further because he borrowed something from me and I fear he is trying to send it back and cut ties with me. How do ND men deal with being caught off guard and surprised? I’m choosing to do no contact and wait for him to initiate any further communication as I was unable to reach him via phone. I’ve never dated an ND man before and I’m still learning. I’m hoping that his withdrawal is just temporary and that we can resume dating because I like him very much. Any insight would be appreciated. Thanks!