It is not transphobic to disagree with this statement. Trans men are trans men and trans women are trans women.
Trans men are, indeed, biological women who have taken exogenous male hormones and gotten surgeries to masculinize their phenotypes. That doesn't change their DNA which is, and always will be, that of women.
I disagree that 'man' and 'woman' are just "genders." They are biological terms that distinguish life stages of human males and females (i.e the adult life stage). That's all. Being a man or a woman is not a "social construct." To give credence to this flawed ideology is to actually reinforce gender stereotypes.
Transphobia is to deny someone basic human rights such as access to healthcare, housing, education, employment etc., just because they're trans. I am 100% against that. Transphobia is not acknowledging scientific fact, and insinuating that it is, is exactly what is turning so many people off.