The basic idea is that it's a chain of blocks, in a fairly literal (CS) sense. The genesis block is the first block in the chain, and it contains a bunch of data and information and junk. The block after the genesis block then contains the hash of the genesis block as part of the header, which is how the "chain" bit is formed. Each block contains a reference to the previous block, meaning you can start from the end and validate that each block is meant to be on the end of the previous block.

Hashing is extremely important to bitcoin. In a very abstract sense, a hash function is anything which takes some block of data, and returns a number. If you put the same data in, you get the same number. If you put different data in, you get a different number. A cryptographic hash, like bitcoin uses, has the additional constraint that even tiny changes in the input data makes for huge changes in the output number, such that you can't guess what data would make a particular number. This is vital for how bitcoin works.

The second important part of the block header is the nonce. This is just a number, but it's a part of the block you can change without changing anything about the block itself. A valid block on the blockchain is one where the hash of the entire block, including the previous block hash, and the nonce, has more leading zeros in it than the difficulty factor. Because you can't guess what data will create a given hash, there's essentially no choice but to sit there tweaking the nonce until you get a hash that happens to have the right number of zeroes. At this point you can tell the rest of the network you've found a block, and they go and look at your block, make sure it hashes right, make sure the previous block hash matches their chain, and then they stick it on the end and start working on the next block in the chain.

Repeat ad infinitum.