D&D went through a similar quandary when it added, and later removed, the Comeliness Attribute. The problem, of course, is that both physical appearance and personality attractiveness tend to overlap each other in subtle ways that are difficult to draw boxes around. If we try to say, for instance, that a physically attractive female who has a horrible and irascible personality has no charismatic influence over people, we are confounded by the fact that somehow she still does indeed have a charismatic affect on others - and even might have people seemingly irrationally bending obsequiously to her every whim, whether they actually enjoy doing so or not (perhaps such followers would feel less abused - or not abused at all - if she also had a high charisma: food for thought). One example of such a character might be the role that Meryl Streep played in The Devil Wears Prada (and here we can get into trouble again with those blurred lines, because Streep is older in that film, and although she has an element of being physically attractive, I would say that the efficacy of her physical appearance has more to do with the way in which she carries herself than with raw physical appeal; and yet she severely struggles with truly connecting with others throughout the entire film - is that what we would expect from someone whose influence comes from their "charisma?"). On the other hand, it may be a little easier for us to understand the effect that someone has through their personal charm or animal magnetism even though they may not appear to be very physically attractive - yet, it does seem that there must be some rudimentary physical attractiveness for their charm to "leverage" - otherwise, they effect can be quite the opposite!
Further complications arise when once considers physical attractiveness as it relates to mate selection. Now, physical attractiveness is not simply a simple attribute to be considered as an effect that can be objectively measured when checking for someone's reaction. Instead, it becomes a game in itself that involves the self-perception of the person who would be affected by the attractiveness of someone else. For example, a woman sees a man who has a certain level of attractiveness, but, she is also attractive, and based other interactions she has had before with other men, she might consider herself to be able to do better, even though he has a certain level of attractiveness. So, in such a case as this, we might model a check for reaction on the man's score for attractiveness (is this a composite of physical attractiveness and charisma - this is still undecided for now) and then account for the woman's confidence in her ability to potentially do better by a simple attribute bonus subtraction or by an attribute vs. attribute chart comparison (using some statistical curve). But even then, we hit upon yet another problem: how do we measure the woman's confidence? Would it be some composite of her overall appeal (charisma and physical appearance) modified by her intelligence/power/(wisdom?)...?
My point, overall, is that it is both a subtle and complex question.
PS - I also forgot to mention the effect of financial and physical power (even intellectual power) that factors into attractiveness/appeal. Streep's character was a successful fashion mogul, and this certain had an effect on her followers. Women can often find men who are physically powerful to be more physically attractive. And let's not forget the appeal that Leonard Nimoy had in his role as Spock: he had more female fan mail that William Shatner, and yet, in a within the constraints of a gaming system, we might easily be led to think that Spock would have less charisma or even physical attractiveness than Kirk.