What about the rights of sperms?

Long ago I entered into a heated discussion with someone. As per him all sperms are destined to one place. If sperms end up elsewhere then it’s a sin. So I said but then an women has only one egg, what about the sperms that didn’t get chance? And what about the time when your wife is pregnant? Sperms are useless at that time.
-Well that’s why men are allowed to have more one wife.

World is full of many such people. Unfortunately they are in majority in many places and end up making law about it.