My fuckbuddy is married but in a sexless relationship with his wife, who like you, loves and loves his family very much.
If anything, Im happy for him, as he has his emotional needs met by them.
For him, Im just a physical outlet, a release for him, we have sex then move on. Mind you, he considers himself "straight". He is also very much an Alpha Dominant Top Male. So I respect his choices and to be honest, I dont care what he thinks or views of himself. When we have sex, he does love to control, dominate, humuliate and make me feel submissive and feminine, it probably helps him justify the sex we have, ie like as if he is with a woman, without being with one, which he would consider cheating.
I love feeling this way with him lol, so its a very much win win for both of us.
The bottom line, is that after the sex, we are both satisfied and the fact that we keep hooking up means we are comfortable with everything that we do and want.