Apologies, I'm really not trying to put words in your mouth, I'm trying to figure out what exactly you're saying. Because if you think one needs a categorical stance against all violence to believe in human rights, almost no one can coherently believe in human rights. Which would be fine for you to believe, but I'm not sure if that's what you're asserting. Is that what you're saying?
but googling UN's human rights declaration, under article 3 (right there at the top of the list!) it says "security of person", which translates to you have the right of not being physically assaulted.
That is a good place to start if you've never read about human rights before. But that article is intentionally vague, and comes into conflict with this article:
Article 29.
(1) Everyone has duties to the community in which alone the free and full development of his personality is possible. (2) In the exercise of his rights and freedoms, everyone shall be subject only to such limitations as are determined by law solely for the purpose of securing due recognition and respect for the rights and freedoms of others and of meeting the just requirements of morality, public order and the general welfare in a democratic society. (3) These rights and freedoms may in no case be exercised contrary to the purposes and principles of the United Nations.
It would be pretty straightforward to argue that Nazism directly conflicts with Sections 1, 2, and 3 of Article 29. As signatories like the US, Egypt, Cuba, Haiti, El Salvador, the UK, or Turkey (among others) would probably assert about the political ideologies they've violently suppressed since signing. How coherently a state can assert this is up to you. But many of the signatories of that Declaration have violently suppressed political ideologies. In my view, many of those suppressed political ideologies are far less dangerous than Nazism as well. But again, it's up to you to decide if you think liberation theology, or democracy, or capitalism, or a country's right of self-determination, or other ideologies that these countries have violently suppressed are more or less dangerous than Nazism.
You can't justify punching someone for the sake of preserving the right of people to not be punched, that's just insert slur foolish .
Nazis advocate racial superiority and genocide, not punching people. Punching people may be incidental in Nazism, but it isn't the core of Nazism. Arguing for the use of violence against violent aggressors seems pretty straightforward to me, especially if you only have to argue for violence at the level of one punch to the head of one Nazi.