The answer is to look at Game Theory. Men are programmed to breed and spread genes. They are also programmed to defeat competition. They are always worried that they may be providing for the offspring of another male if their female is promiscuous. If they find females who do not appear to be likely to have sex with other males then these are ideal partners for spreading his genes. Remember that for most of primate history the females were not always necessarily willing partners.