I hear where you're coming from, but that hasn't been my life experience of men. (As I mentioned, I think women are different in this regard.) I can't see into anyone else's brain so I don't want to speak for anyone's experience. I've just been out a long time, and have not talked to others who are set up the way you describe.
At the time I came out, saying you were bisexual for months or years before you admitted you were gay was almost mandatory, so I don't have direct experience with bisexuality being a more difficult coming out, but like I say, I can only speak from my own past and the people I've known along the way, and I'm sure my own further inferences and assumptions from that sample (as humans are wont to do). I'm not negating anyone else's reality. And I personally don't count behavioral bisexuality (i.e., having sex with women) as bisexuality if the person is truly only physically aroused by men, but who cares about my definition honestly. I'm just a person with opinions.
I wish straight men were more prone to bisexual behavior because it would make wedding receptions and business conventions a lot more fun and less frustrating.