"I'm actually in the middle of writing a fantasy project where the protagonist starts out with the handicap of being an intersexed slave in 800BC."

Oh, cool! Will you be posting this online, or is this a pro-fic project?

"I could well see a society with, say, telepathy, where those who cannot speak mind-to-mind are considered the disabled ones"

Yes, exactly. It's like with mental illness: we don't consider most people mentally ill, because they don't fall outside the human norm, but from an extraterrestrial perspective, all of humanity might be mentally ill.

"I don't know if you've read the Vorkosigan books"

By coincidence, I just took out a Vorkosigan audiobook from the library. I'll look forward to reading it now.