This is covered largely in the history section of Wikipedia’s entry on newlines. Basically there are two lineages of operating systems leading to modern-day Windows on the one hand, and Unix-like systems on the other.
Windows descends from MS-DOS (because initially it was implemented on top of DOS), which itself inherits much of its behaviour from CP/M. CP/M inherited its line-endings from DEC systems, which used CR+LF because that’s the character sequence required to move the cursor to the start of the next line on ASR-33 teletypes (among others), which were common teletypes used with DEC systems. On most teletypes, CR and LF do just what their names imply: carriage return returns the carriage (carrying the paper) to the right, so the hammers or type head are above the left of the page (or equivalently, it returns the type head to the left of the page, depending on which part of the assembly is mobile), and line feed feeds the page one line up. The order was important: carriage return takes some time to execute, so starting it first meant that the line feed would happen in parallel to the carriage return, and by the time the line feed was processed, the carriage had a decent chance of having finished, so the next character could be processed safely (otherwise it ended up smeared across part of the page as the carriage finished flying back).
Unix was inspired by Multics, whose developers chose LF as the line-ending character, relying on device drivers to translate that to whatever character sequence was required on actual devices. (I recall discussions on this topic where the idea was floated that the Multics developers did this in order to save disk space, but I’m not sure that’s accurate. Another possible reason is that relying on the device driver to handle this meant that each driver could adjust the timing as necessary, without the system having to care about it — CR+LF in particular was chosen partly for timing reasons.)
Some systems used other conventions (see this table); many systems, including all of Apple’s computers before OS X, used a single CR, and obviously non-ASCII systems had their own line-ending characters (this includes IBM mainframes and 8-bit Atari home computers).