I think he's incredibly hot with and without the beard, with scruff, and everything in between.
Also from what I've heard it's common enough for some doctors to not focus on getting serious with their personal relationships until later. They're just too busy with extended medical schooling during a time in their life with a lot of other people are pairing off. Doctor Mike has his public career in addition to medical on top of everything else so would be even more busy. I wouldn't read more into it than that.