That is an important observation as once again, it is reflective of who people unfamiliar with people with disabilities are, it is not reflective of people with disabilities themselves. Now clearly people with disabilities are just people themselves so they may feel awkward in a social situation just like anyone else. But in the context we are considering, one person has a characteristic called impairment, and the other person once again is unfamiliar with that characteristic so they bring awkwardness to the social situation.
The good news is that familiarity with people who are different from yourself, whatever your characteristics are and whatever their characteristics are will lead to a break down of the unfamiliarity leading to awkwardness. So you can change! You don't have to be awkward.
Just remember if you could make the same statement as the title of this post, that is a reflection of who you are, not a reflection of who a person with a disability is.