There exist scenarios under which two people can both be happier with the outcome than they would have been otherwise.

Ok, so economic growth is measured in happiness. Good to know.

comparative advantage violates physics

The problem is is how cost and wealth is measured. Economic growth is basically an accounting trick.

What counts as wealth? Everything that shows up in some accounting ledger declaring this stuff is property of someone else. If it doesn't show up as someones property in some ledger it isn't wealth and doesn't count towards economic growth.

Imagine a river system that supports the life of millions of people in a subsistence economy. That isn't wealth. But give out some mineral rights, drill for oil, build some chemical plants, destroy the river ecology and the basis of life for those millions of people, then those plants and the sold oil show up on ledgers, while to the cost to those millions of people doesn't, and the economy has grown.

Or a different example: Quality time spend at home with your family to maintain a working relationship and raise your kids properly doesn't show up on any ledgers. But when both parents have to work two jobs to make ends meet, those billable hours do show up on ledgers, while the neglected kids and broken marriage don't, and the economy has grown. Even better, during the divorce, much of the little wealth the family has managed to gather without showing up on some ledgers falls to lawyer fees and the like, and the economy has grown.

Any economic growth is just a result of not putting externalities on the ledger, by grabbing natural or social resources into private hand and saying this wealth didn't exist before.

What does change is the kind of wealth people have. Todays kids may have game consoles and night sky lamps to help them sleep. But they don't have a real night sky and large areas to roam outside anymore.

We may have forgotten what hunger is, but we do have impoverished diets that make us sick in exchange.

And in every one of those shifts of "kind of wealth", a little more absolute wealth ends up in fewer and fewer hands.

So how does this relate to comparative advantage? In a global economy when the players are private fortunes, local comparative advantages becomes increasingly irrelevant. When real wealth and resources and effort are spend and wasted to facilitate growth in fictional wealth in a few bank accounts, because that is how economic advantage is measured, overall economic numbers become irrelevant, because they don't reflect the wealth of societies, but of a few people.