The following is my understanding of it, but bear in mind it's been a while since I researched it so things might've changed. Also I'm not an expert, so take my words with a grain of salt:

Over-the-ear headphones are perfectly safe because it provides an air cushion between your ear and the source of the sound. As long as you don't have the volume on ear-blasting levels, you should be a-okay.

On-the-ear headphones are less safe, but are still mostly okay, especially if they are used sparingly, say, while listening to music at the gym.

Earbuds/in-ear speakers are pretty much the worst for your ears. They can be okay as long as you're listening at low volume for short periods of time, but they have the highest chance of causing you ear trouble later, because there is literally no real air space between the speaker and your ear. Think of it like putting your ear flush against a speaker, blaring music, and expecting to be fine. It's just not gonna happen long-term.