Great question! Locke's argument here is that while you're sleeping you actually don't exist at all. Obviously what this means is a little confusing, someone can clearly observe you sleeping and even film you sleeping and show you afterwards to prove that you "existed" while sleeping. But the concept of what it means for you to exist is a little more complicated than that. Certainly you wouldn't argue that you exist when you're dead because your corpse hasn't completely rotted away. So are you really yourself in a state of non-consciousness like sleeping? It's a difficult idea to wrestle with.

As far as being a new you when you wake up, the memory theory idea of the self says when you wake up as long as you remember being you before you fell asleep you're still the same you, you just weren't you while you were asleep.

If this is something you find interesting I'd recommend reading chapter XXVII of Locke's An Essay Concerning Human Understanding titled: Of Identity and Diversity. Locke's English is a little hard to follow and it's kind of dense but pretty interesting.

Memory Theory in some form or another is a really widely accepted identity theory among philosophers but some really great philosophers have other ideas as well. If you're interested in something a little different you could check out David Hume's A Treatise of Human Nature this is a big book and there's a specific chapter I think towards the end that is relevant to identity theory where he essentially argues there is no concrete "self." It's worth checking out but I can't recall exactly which chapter it is.