After reading the whole Library of Alexandria here in this thread, this is what i think right now:
Btw: If i got something completely wrong, feel free to correct me :).
The first thing i have to say is that i will concentrate on humans in the following.
One thing i realized pretty fast is that the definitions, which were and are used, are not the same for all of us and furthermore are often blurry, which leads to different answers.
So if you were to ask me, how to define "sex" now i would go by the function someone has in sexual reproduction, which i would categorize into a binary system, but why? I wouldnt group people in terms of their chromosomes in categories (if you go by that there would be certainly more), but in terms of their function in sexual reproduction, so there are different karyotypes in the same group. One group fulfills the male part and the other fulfills the female part in reproduction. So what if you cant fullfill a part in reproduction? The answer is simple you dont have a sex. At first, it sounds disrespectfull and strange but here is why: I would use the term sex only to describe which is your part in the reproduction. Nothing more, but also nothing less. It is just a word to describe something, as there are millions of words to describe all different things that exist and if it wasnt the term sex that is discribing what i was writing about, there would be simply another word to describe it. I elaborate on this further with an example (it may not be perfect but it does the job :D). There a all sorts of bottles, some are larger, some have a smaller opening, some are green and so on, but they are all categorized into the same group by the word bottle, just by the simple reason that they all have the same function. The same with the male and female sex. If you have no sex then you wouldnt be a bottle but as the bottle, you would be a container (In terms that the bottle got one function more, for example a lid or something) just as a non-sex person is still human just has no sex, same as there are people with cancer and no cancer (simply put).
Furthermore, you could also ask what in case of a true hermaphroditism? You would be simply both cause you could fulfil both functions. So what i am also trying to express it that the sex i try to describe here is nothing that is more an part of you than the description of your hair, it is simply a description of reality.
On top of that what about people that lose their part in reproduction due to age, accidents, surgery...? They dont have a sex, because they lose those function and so the description doesnt fit anymore and you cant just mess up a word because the whole point of having a word is so that it describe some things and doesnt describe other things. You could make the word account for whatever you want but the problem is that you blur out the word so badly, you cant tell what it means anymore.
You could also define the word slightly different and call all people who once where male/female in sense of their reproduction for ever male/female, but it kinda looks to me that that is the part in which the word gender comes into play and i would just refer to them in a biological way as "once male" or "once female".
What about other aspects of gender differences? These are causally conditioned by your karyotype, which also determines your sex (role in reproduction) and if you got a sex. So you could select differneces by sex but since there are not just 1 karyotype in the male and female group the more accurate version would be by splitting it up in differences of karyotypes.
Quickly reminder all just theorizing in my head. Nothing fix, nothing 100% perfectly planned out thing.
One thing i also thought about is sex in terms of history. Why? Because i thought wouldnt it be interesting thing to know how it all started?
I did no research in that area, but the following is what i came up with myself again (dont take it too strictly):
Since humans developed from instinct driven animals to a conscious creature, sexual reproduction was something that just happend instinctively and by time got passed down by language. So people needed some wording to describe the world and the first thing you describe is what you see in some manner. Back then they didnt know about chromosomes, DNA and so on, they were more fixated on the external human. So they saw and knew: Genitalia 1 and Genitalia 2 is needed to reproduce (and maybe they named them). They came up with names for humans who bear certain genitalia and also differentiate in external looks beside genitalia, so they were able to communicate and voilà male and female is born. Back then they didnt thought about things like gender or are there more than 2 sexes?, because they "only" saw the external and some aspects where more important than others, the primary reproductive organ for example was superior, because it carries with some other aspects the most meaning in surviving as a race. So you could also have men breasts if you just got a penis and would still be male. Also worth to consider is the low life expectancy which probably never led to unfertility because of age, the high death rate of accidents (rare infertility because of accidents) and the fact they didnt know as much about medicine as we know today. Reproducing isnt just something that works everytime, and im certain back then they knew, so they probably thought of people who were infertile since birth not as a new sex or something like that but as people who "had bad luck and had a reproduction failure". To put it simply once categorized a sex, forever this sex. With time more and more knowledge was gathered and more and more questions arised. But the sole reason why i am writing this is to show that i think (that is important), sex was in the beginning never something hyper-personal like gender is today, but started merely as a descriptive tool of mankind.
And to have a word to describe the reproduction function is nothing bad to have.
You can have names for people that have the same karyotype but this is just by definition something else.
And then there is gender in which i wont dive into :D.
But i think, this is it. So by this definition the answer is no.