I think it actually falls into another kind of "human" reaction -- not believing things that we haven't experienced ourselves. Some people on this thread think that wanting a threesome for added size is very rare, because it is outside of their personal sphere. However, I have been with many dozen couples who want an outside man to join them. Other guys here have as well. More often than not, cock size plays a role. That is our experience. And I assure you, that desire in couples is way more common than your 'panhandler phenomenon."
Take a look at my poll about sharing wives and girl friends, in the ("Relationships, Discrimination and Jealousy..." forum). Over 110 people answered that they either fantasized about, or actually shared their partners (in a ratio of about 2:1). I'm not making any attempt to be statistically accurate, just that a significant number chose to answer the poll positively just on this site. In terms of the people who take the time to read and answer polls here, that's not that rare at all.