So as someone who is sometimes attracted to "straight" men (among many other types I have), I think I'd explain it this way:
I spent a big chunk of my 20s in the Army on active duty. In part, that was about problems I had with my own sense of masculinity. I wanted to prove to myself that I and that being gay didn't make me any less of one. Another part of that was wanting to turn myself into the kind of man that I'd be attracted to. I was very much into hypermasculinity (some of my first JO material was the WWF) and I wanted to be that big kinda intimidating guy who doesn't set off anyone's gaydar. About the same time, I was noticing that a lot of gay guys who, for instance, get into the leather scene, may be able to dress reasonably convincingly as bikers, but don't exactly act the part; as is sometimes said with some cruelty, "He looks good, but he opens his mouth and his purse falls out."
Meanwhile, I would have all these straight guys back in my unit (and later in the jobs I held after getting out) who were rubbing off on me. You are who you associate with, and for some very formative years I was hanging out with a bunch of crazy straight infantrymen.
I want to emphasize that a lot of this had to do with my own hangups about my orientation. I'm a bit ashamed of how I could be pretty cruel myself regarding stuff like drag or effeminacy (and I've since come to have very great respect for the skill and talent that goes into putting together a drag persona).
So anyways, of course these straight guys around me would have looked fantastic to my eyes. They were everything I wanted in a man at the time. Except as far as I could tell they weren't into me at all, and I didn't dare say anything that might have given away my orientation. This was the age of Don't Ask, Don't Tell.
I think, too, being stuck in the the closet throughout my time in the Army also had the effect of making me wonder if I'd be accepted if the other guys knew who I really was. That's one of the things that sucks about the closet: that fear of rejection if they knew the real you really fucking sucks.
So when I did happen to run into someone who came across as "straight" but nevertheless was interested in sex with me, that not only really was an attractive quality, but it also was a way of getting some validation: "See? This straight guy likes me well enough to give my his cock." It was a tiny bit of proof that maybe if ever I did come completely out, it wouldn't be as bad as I feared.
In retrospect, there's something very humiliating and debasing about that sort of thing. It can make you desperate to please men who likely really don't deserve it, just to keep alive the faint hope that maybe, if I can catch him alone at just the right moment, he'll give me the validation that I need to see myself as more of a "real man". I want to be just one of the guys, but I also want it to have that sexual twist added in, or else it's not a full acceptance of the whole me.
If that makes any sense at all.