Not sure I have much to add to this (EDIT: I say as I type out several paragraphs like an idiot), but yeah, as far as I understand it vingans and nandans aren't really neat analogues to modern trans identities, partly due to pronoun usage it seems, but also because in Orlanthi society, gendered identity is cultic, and by extension sorta professional. Vingans commonly appear to take on stereotypically male traits by DOING stuff men tend to do. And presumably nandans the opposite, though they are criminally underrepresented. In stateless, labor-divided societies like the Orlanthi, work and ritual practice is identity, much moreso than in a modern capitalist society. (This is not to say that ALL work, or even necessarily most work, is gendered, but some duties, especially religious ones, appear to be.)
To me, at least, this sorta builds upon the Orlanthi identity as a much more collectivist one than modern society, where the self is conceived of less as an expression of some inner "true" self, which sorta exists independently of other people around, but more as a bundle of professional and personal relations through labor and social obligations. Identity is necessarily interrelational (and therefore conditional and dynamic), rather than "expressed" (or emanated) out of a (commonly presumed) stable self-concept, the way we tend to use it in modern (colloquial and mainstream) parlance. IF that makes sense.
This also relates to how some women seem to fall in and out of a vingan identity for specific purposes (and perhaps nandans or even helerings do too, though they don't appear to have specific analogues to the avenger quests of vingans). So someone performing duties that are typical of a man and initiating into Vinga can be a vingan for a while, but then presumably stop doing that and do something else, effectively becoming someone else identity-wise, and that's fine too. (I'm not an expert on how this exactly would work with regards to switching cults and retribution, but hopefully I'm not completely off the mark when we're talking about mutually friendly cults). Hell, we know from Jeff that women can join the cult of Orlanth no problem, so there's even more nuance and complecity here.
The counter to this idea of Orlanthi identity being more of an amalgamation of ongoing social relations than an expression of inner self-understanding is admittedly the notion of fixed Runic affiliations that sorta just... exist in your soul or something.
I say all of the above with an anthropological lens, mind you, and an inescapable cishet one too. All of this "isn't it neat" is irrelevant if the very premise just really bums you out in an out-of-universe sense, as a player interacting with other players.
To me, it's interesting to see how, for all intents and purposes, the Orlanthi devised a different, but apparently functional system to deal with diverging ways of being a person than we have in the modern west, BUT if this kind of concept, creative though it may well be, ends up alienating trans or non-binary players (who are pretty likely to be "modern Westerners") then... yeah, it's probably a problem.
(On the further topic of Runes, I'll admit I'm not a huge fan of people basically getting a glorified Myers-Briggs test at the onset of puberty that will define their lifepath, but then I approach this setting less as a gamer and more as a reader. At the very least, I would have liked to see periodic opportunities to effectively re-test one's runic affinities throughout life. "Oh, godspeaker, I have felt a change in my heart, I am not who I was yesterday, I seek help to see the change in me so that I may once again have sure footing on the path of my life", or something along those lines, to make it less deterministic and essentialist, but maybe that's dumb.).
tl;dr: I have a fondness for identity systems that are different from the ones most of us are typically familiar with, not just with different categories but also in underlying logic. However, if the basic logic of this system ends up frequently making trans or non-binary people feeling unwelcome or uncomfortable (ie. it ends up being "transphobia with extra steps"), then that should be adressed so that it stops doing that. I'm not gonna die on a hill that pushes trans folks away.
I dunno, just typing out why I think the gender system of the Orlanthi is interesting, but also possibly more difficult to grok than we might think at first glance, while keeping in mind what y'all have said above.