If I had to guess... Peter is not the first guy Roger allowed access to his body for money. He's inarguably straight (and way out of Peter's league on the attractiveness scale), probably had as much sex as he wanted, and did not find the situation remotely arousing.
Also if you get sports massages, etc., on the reg, the sensation of having someone's hands on you is not as tittilating as it might be if you are not used to that kind of contact. Peter tried his damndest to get a rise out of him, though, didn't he. In one of the videos, Roger tells him flatly, "If you want to see me hard, get a girl here." But that would end up being a different kind of content than Peter wanted.