Okay, this has come up several times recently, so in honor of the approach of Intersex Awareness Day, on the 26th, a central thread to talk about it. Content warning: long wall of text. Tl,dr at the end.
Disclosure: I am not intersex. I have two relatives (one by marriage) who are. I am close enough friends to four other intersex people that they've disclosed significant medical details about their condition, and I volunteer with a support group for intersex kids. I keep myself current with medical research and with political & social issues important to the intersex community. (If there's anyone here who is intersex or has better firsthand knowledge, please chime in. I'll try to be a good ally.)
Definitions: A person is intersex when they have sexual organs that developed in some way different from typical male or typical female. There are many, many possible differences, and so many intersex conditions. The medical term was, for a long time, hermaphroditism, even though this is a different use of the term from the way biologists use it. It is now considered offensive to call anyone intersex a hermaphrodite — offensive not quite on the level of nigger, but approaching it. (This is why the SOL tag drives me to fits — there are stories with true hermaphrodites, and it's appropriate for those, but there's nothing else for tagging intersex characters and readers want to be warned about that.) The prefered medical technical term is Differences of Sexual Development, or DSD. Most intersex people use the term intersex.
Numbers: Because of the stigma intersex conditions have, hard numbers are hard to come by. Estimates vary significantly. The best we have is that around 1 in 200, or 0.5%, are born with some sort of intersex condition. The two largest types are having an unusually enlarged clitoris and having a urethra that does not exit the end of a penis. These are relatively minor, and there is continued debate over whether to classify them as intersex conditions. Even if we put those aside, all other conditions come to around 1 in 1000, or 0.1% of the population. In the United States, that's over 30,000 people.
This is not a trivial number of people. Trivializing them as insignificant is not respectful.
A lot of energy gets expended, often generating more heat than light, debating whether certain people are "really male" or "really female". One commonly asserted guideline is, ignoring the fringe cases of chromosome duplication, whether the 23rd chromosome pair is XX or XY. This is a horrible, HORRIBLE guideline. Here's why:
Sexual development is not controlled at the chromosome level, but at the gene level. As of three years ago (the last time I deep-dived this), we've identified at least a dozen genes responsible for different aspects of how sexual organs develop, a handful of which are the most important. Not all of them are in the 23rd pair, not even of that most important handful. (Or usually there — that genes jump chromosomes is yet another complication.) IF all of the relevant genes are pulling in synch AND a person develops as "typical male" or "typical female", THEN and only then is XY/XX a reasonable shorthand for what sex, genetically, someone is. This does not hold true for a large proportion of intersex conditions.
To give one broad example: Early on, embryos grow proto-sex-organs that later, in the fetus, can (assuming everything works as designed) develop as male or female. So there's a cliteropenis that can either stay small and become a clitoris or grow large and become a penis. There are proto-gonads that can develop into either ovaries or testes. There's a proto-uterus that can either grow into a uterus + fallopian tubes or wither away. (Yes, all you dudes out there: at one point, every single one of you started growing a uterus. Most of you stopped well before you were born, but you had one.)
The genes that control most of the development of internal sex organs are different from the ones that control development of external genitalia. Which means you can have typical male internal anatomy and typical female external anatomy (with testes that never migrated out of the body cavity into a scrotum) — regardless of whether the person is XY or XX. The most common cause of this is Androgen Insensitivity Syndrome (AIS). Complete AIS results in external anatomy that is typical female, Partial AIS in anatomy that is somewhere between typical male and typical female. (FWIW, which is not very much, most but by no means all people with AIS are XY.)
tl,dr: Biological sex is not a binary. It is clear for most people, the overwhelming majority in fact. But between them, there is a continuum of anatomical and genetic possibilities, and insisting on binary terminology for people in that continuum is not rational.