By defining gender as binary, you are saying that there are only two different genders, male and female. While that is normally what society thinks as the only genders, this is not necessarily accurate. We cannot label gender as binary when there are so many people out there who do not see it that way. By saying that gender is binary we as a society only see male and female. The reality is that there are many other genders than just male or female. If we go outside the realm of binary gender then you are risking going against the social norm and not fitting in.

Third gender is defined as recognizing more than just male and female genders. Australia, Germany, India, Japan, Nepal, New Zealand, Pakistan, Philippines and Thailand all recognize third genders. They have the option to choose indeterminate gender on birth certificates.

The Kathoey are what the third sex is called in Thailand. They are born males who identify as female.  The Thai are extremely tolerant towards the kathoeys. They recognize them as a separate gender and some even see them as woman or a different category to woman. The kathoeys all recognize that they may have male genitals but none of them see themselves as men.

https://en.wikipedia.org/wiki/Third_gender