I hate shaving, and usually have a stubble. And I've always lived, studied and worked in highly "propah" environments.

All my life I've had dirty looks thrown my way, from men as well as women. Whether in India (I'm from there originally) or in the US.

IMHO, it's just a manifestation of implicit hatred towards "assumed toxic" masculinity.