I did Strang's linear algebra course[0] (link goes to Youtube playlist of lectures) several years after graduating and recommend it highly. I was looking for refresher but I gained a deeper understanding of several important concepts; in particular it's fair to say I barely understood, or perhaps even misunderstood, SVD until Strang. If you're not sure if you need something like that, I suggest doing something like testing yourself on this video[1] or the MIT problem sets[2] it's easy to tell yourself that you "know" linear algebra when it would be closer to the truth to say that you used to know linear algebra, but can't answer even basic questions today. After Strang, Golub's book on Matrix Computations is also really incredible.[3]
So grateful something mentioned this man. Strang is just amazing. When I first moved to Cambridge, his corresponding book was a relatively expensive prospect, and I was rather serious about 18.06, so ascertaining its book was very important to me. He was gracious enough to gift me a copy that I still have and cherish to this day. Some real moments of thrilling discovery happened for me, it was exhilarating, though trite as they may be! Like in implementing a program for general inversion of any MxN matrix, one would typically perform the Gaussian elimination (going down) and then the Jordan (going back up) and finally divide by the scalars in the pivot columns. But, as it turns out, it's a much simpler program if you do Gauss elimination, literally rotate all matrices by 180 degrees, do Gauss elimination again, then rotate everything back, and then address the non-unit pivot columns. src: https://github.com/wittedhaddock/AlgebraicCircumscriptions/b...
??! is there such a thing ? only nonsingular square matrices can be inverted. a general mxn matrix may have a “left inverse” or a “right inverse”, but i don’t think your code is computing that.
There is such a thing as a generalized inverse, in which you can invert non-square or non-full rank matrices and the generalized inverse meets many (but not all) of the properties of an inverse. The tough part is that the agreed upon set of properties does not create a unique solution for a generalized inverse like it does for the inverse, so there are multiple possibilities when someone says generalized inverse. However, the most popular is probably the Moore-Penrose inverse: https://en.wikipedia.org/wiki/Moore–Penrose_inverse
I like Strang and have worked through some of his books but when I was first learning Linear Algebra, I preferred learning from Dr. Aviv Censor's videos on YouTube[0] from when he was teaching at Technion. It's a few dozen videos where he spends a lot of time going through each component of an introductory course on Linear Algebra.
/aside As someone who's also refreshing maths: What's the point of investing in studying hard, if I end up forgetting? Is there some aspect of maths I should focus on, that is an enduring investment?
My feelings is that getting an intuitive understanding is key, so that it becomes part of myself... but I find proofs don't give me this (they show me that it is true, but not why it is that it is true - if that makes sense). Derivations seem important, because then I can re-derive when I forget (though, I must recall the "tricks" of the derivation, and also know the operations used). General skills can persist, if they get ongoing exercise (e.g. methodicalness, care with definitions, close reading, organization to cope with complex and multi-layered problems).
Finally - and awfully - I now think mathematics is like a language, not so much in the sense of communication, but by being full of special cases, exceptions, "abuse of notation". Becoming fluent in a language takes much practice, and may be impossible without special talent. But once achieved, is never fully forgotten, and quickly regained.
The payoff is indeed learning the intuition, but also learning how to prove things and learning how to learn math. The content itself is only as important as whatever field you're in, or whatever application you're studying that requires that specific mathematics. But the "thinking like a mathematician" and knowing how to learn new math is the valuable skill you'll gain.
Another great "intermediate" textbook (in my opinion) is Trefethen's Numerical Linear Algebra [1]. Much more readable than Golub's I would say, which is more like a reference than a textbook.
Strang just published a new book called "Linear Algebra and Learning From Data" [1] which I only just started but find to be quite enjoyable so far. It's simultaneously conversational but also quite terse (similar to "All of Statistics"). There are many advanced and very contemporary applications covered in the book, with a focus towards machine learning.
When taking Lin Alg in undergrad, I attended the first 3 classes. My friend then showed me the youtube series from Strang; I never went to class again and learned it all.
Speaking frankly, it's ambitious to complete even one of these courses in a single summer. Most students would be taking one or two of these in 12 - 16 weeks while doing nothing else but being a student. Accomplishing the same in about eight weeks from video lectures will be more difficult. I would strongly urge the author to slow down, choose a single course they have the background for and work from there.
I can't tell if the author has done a real analysis course before, but if they haven't that's the one they should choose next. If they already have that under their belt they should go to probability or complex variables. I don't see the utility of re-doing calculus or linear algebra if the author is already strong in both.
I appreciate the enthusiasm for math that's evident here, that's great! But if you're learning math the "right" way - by actively engaging with the material - there's only so much of it you can learn at once.
> I can't tell if the author has done a real analysis course before, but if they haven't that's the one they should choose next... I don't see the utility of re-doing calculus or linear algebra if the author is already strong in both.
having (somehow) completed many of these requirements for my 18c degree, i would say that analysis is not necessary if your interest is actually applied math. There's a great line in rudin's preface that says that his approach is ~"pedagogically sound at the expense of being logically incorrect," and recommending analysis for somebody that's not looking to mainline a pure math degree to me feels "pedagogically unsound (but logically correct)"[0].
I took analysis and i appreciated it, but i really loved the applied classes in my degree: 18.310, 18.311, 18.781 (theory of numbers) along with algebra 18.701/702. If you haven't taken a higher-level algebra class, it will let you know if analysis is right for you because you'll brush up against the edges of it without (what i consider to be, at least) its hallmark punishing density.
There are other great electives in math at mit, shop around the 18.4* classes and dial in by interest, most of them only require a prereq of 18.02/18.03/18.06 and you can sort of figure the rest out along the way.
Something to be aware of is that for a while 18.310 didn't have a dedicated instructor, so it really was all over the place. 18.311 was also somewhat hastily structured the semester i took it, but it is actually pretty good material.
You may find that after you've done all these classes that you are actually interested in pure math and at that point i would suggest looking at 18.100b (analysis), 18.700 (linear algebra), 18.100c(real analysis), 18.901(topology) and the rest of the "hard math classes," but i really do think that you'll find that the rationale for those classes doesn't click if you haven't taken a few classes like 310 or 701 first.
just my two cents! good luck, have fun!
[0]: this is the actual quote, it's in the preface rudin's principles of mathematical analysis 3rd ed. which is the 'textbook' for 100b.
> i would say that analysis is not necessary if your interest is actually applied math
If you start reading research papers in applied math, there’s a ton of measure theory and functional analysis there.
More generally, both introductory real analysis and introductory complex analysis are assumed basic foundational background for pretty much any kind of research mathematics, applied or otherwise.
I’m also not sure I would recommend trying to self-study them though. Some expert guidance/feedback is pretty helpful for someone starting out.
Scott Young is a publicist - his goal is to sell, sell, sell. Completing a world-renowned 4-year curriculum in 1 years time is impossible. (He also admitted to reading the solutions instead of doing the problem sets)
This is akin to saying that Math 55 is impossible and neglects that he didn’t claim to complete the requirements for an MIT degree but for a Computer Science Major. That’s two years of classes, not four, and if your aim is to pass the exams for them rather than learn everything to mastery doing it in one year seems possible if still incredibly hard.
> Most students would be taking one or two of these in 12 - 16 weeks while doing nothing else but being a student.
From my experience (at MIT in the 80s and 90s): The usual student takes 3 technical courses and one humanities course per term. From my own history: I took 18.03 in one term with two EECS courses and a humanities course, and then 18.04 the following term, also with two EECS courses and a humanities course.
I would guess that 3 math courses at one time (and taking nothing else seriously technical) would constitute a pretty full load for most people. Much beyond that probably impacts how much is being truly absorbed for the long term.
> From my experience (at MIT in the 80s and 90s): The usual student takes 3 technical courses and one humanities course per term.
My experience (in the 80s) was the same. There were a couple of semesters where I had to take 4 technical courses; I had no desire to repeat the experience. :-)
I agree with this comment and would supplement as follows:
whatever you intend to learn, consider how you're going to retain it. I'm quite sure that I've forgotten a lot more maths than I know. I'm not sure about the utility of forgotten maths. You may think it rehydrates well, but it doesn't. It won't be as hard to relearn but it'll be hard. So consider, for whatever you learn, are you going to be using it? Are you going to be building upon it? Is it just a tourist expedition? These are all fine reasons but they each have different implications.
I think most of us are highly invested in the value of easily-rehydratable knowledge, because an incredible amount of the first third of our lives is spent learning facts that we'll forget. If we admitted there might be a better way, we'd have to do the hard and thankless work of institutional overhaul in our schools.
This mistakes the purpose of school. It’s less learning than childcare for primary school, socialisation and teaching children to do apparently boring and meaningless work as directed by superiors, and ranking and proof of intelligence and conscientiousness at higher levels. We’ve known most people forget almost everything they learn at school for a very long time and spaced repetition and the forgetting curve date from Ebbinghaus (1885) while mastery learning is probably older than writing though the name dates from 1968 and the earliest unambiguous instance of the concept dates from the 1920s.
As someone who also went back to study math after not doing it for a long time, I would recommend doing a book that focusses on teaching how to prove things, using the math you already know. Most upper div math courses either assume you know how to prove things, or try and teach it in passing while also teaching the topic itself (classically linear algebra and calc 3 were taught this way, though now many places now have separate "proof" classes instead)
I would strongly recommend Chartrand and Zhang's "Intro to mathematical proofs", once you know how to prove things, every other more advanced math class you take will be much easier than it would have been otherwise.
[Edit] In particular, do the book, including the chapters on "proofs in calculus" and "proofs with real and complex numbers" before doing real analysis - you'll enjoy it much more that way! [/Edit]
I went through most of this book a few years ago, now about to finish my MS in math :)
Agreed. I'm trying to move from being a secondary mathe teacher to actually learning pure math, and proofs are the killer. I feel like I understand topics, but creating proofs kills me.
Technically speaking, no. Real analysis is pretty self-contained. You basically start out by constructing the reals from scratch and deducing continuity as a consequence of the completeness of the real field. You use a tiny bit of set theory to establish notation and define bounds, and then from there you go into limits, derivatives, integrals and maybe the Lebesgue measure. I wouldn't expect a real analysis course targeted at applied math majors to do much other than that.
The reason real analysis is useful is because it's (loosely) a deeper calculus course with proofs. Since probability theory becomes more proof-based (and ventures into measures), real analysis is good preparation for it.
> You basically start out by constructing the reals from scratch
Not necessarily -- a lot of books just take the existence of a unique set with certain properties as an axiom and call it R.
The main topics of basic real analysis IMO are differentiation, integration, (uniform) continuity, compactness, convergence, etc.; how to construct the reals from the rationals is a side point at most.
This could be personal bias as I just don't personally think that the exercise of constructing the real numbers is very interesting.
Analysis requires a certain level of mathematical maturity to motivate, follow, understand, and appreciate. This level is roughly acquired after having taken undergraduate coursework in ODEs, linear algebra, or abstract algebra.
I’ve been doing more machine learning for work, and I’ve realized a) how much calculus is involved in it and b) how much calculus I had forgotten since I took the courses as an undergraduate almost 30 years ago. I happen to still have my old calculus textbook (I couldn’t sell it back to the bookstore back then), so I dusted it off and started working through all the problems. I’m about halfway through right now, and man I get strange looks from people when I start working calculus problems. Good to know that I’m not alone, although I’m about halfway through the equivalent of 18.01 (1/10th of the poster’s coursework) after 6 months. It doesn’t look like there are that many sample exercises in the online course, though - that may make it harder to really absorb.
Awesome, it's heartening to see someone else feel the same way! And great note about the problems, maybe one of the classic textbooks would be a good addition for problems
It’s awesome to see more & more devs getting interested in math & stats as machine learning, optimization & sequential testing (what you folks call a/b test) becomes more mainstream.
The obvious benefits are, much of math literature is currently written for consumption by math grad students & profs, so all of this will get a much needed rewrite. I foresee a surge in material like Jeremy Kun’s popular “math for programmers” book that so many of you loved on HN. Much of math lit is imo unnecessarily dense & terse. Speaking as a phd student, I really appreciate atleast 1-2 numerical examples with answers, not just plain theory. Maybe if there was a Linear Algebra for programmers text with some code in python to accompany the theorems, it would be a huge hit among applied ML researchers & programmers. Graduate linear algebra is actually quite hard & very few phd students pass the linalg quals in their first attempt ( Grad linear algebra, not basic shit like matrix algebra, linear maps & decompositions). This material should imo be rewritten ( “dumbed down” ) so regular folk can also sample some of the magic.
In my previous work at an adtech corp, I built a whole bidding algorithm straight from Strang’s linear algebra book (the chapter on Lagrange optimizers). It’s still running, 5 years & counting. I once joked if they gave a nickel out of every dollar that algorithm made to Prof Strang, he’d be a multimillionaire.
The Coding The Matrix textbook is exactly the type of book you've described that is tailored for programmers. I've read only the first few chapters and found it a joy even if a little wordy in some parts.
I took a course in linear algebra when I was in grad school, took a graphics course (which is basically applied linear algebra), watched the 3 blue 1 brown linear algebra series on YouTube and have spent some time looking at machine learning algorithms (which seem to be basically applied linear algebra) and I still don't feel like I have any real intuitive sense of the topic. Whenever I read an algorithm description that includes a matrix multiplication, say, I understand exactly what they did, but I've never looked at a problem and thought to myself, "ah, that's a problem for linear algebra!"
it’s a good undergrad book. also on that page he says lights out game is a “linear algebra problem”. It’s actually not. incidentally I worked on Lights out for my thesis, and my advisor made a whole career out of lights out! That game relates to Fibonacci polynomials, studied in Combinatorics - https://www.unf.edu/~wkloster/fib.html
How would you compare coding the matrix to strang ?
I'm going to be starting out on one of these and coding-the-matrix is very python centric (which is super helpful and practical) versus MATLAB in Strang's book.
this is classic “mistaking the map for the territory”.
whether the code is in python or matlab matters little. the code is just a supplement to the main material which is still math. trying to understand that math by reading code is actively counterproductive and actually quite foolish. my professor gave a symposium talk on this topic, so i will use one of his examples -
At 0 minutes you have 0 dollars. suppose each minute you toss a coin. if it’s heads you make a dollar. if it’s tails you pay a dollar out of your pocket. What are the chances you’ll have $30 in an hour ?
This is the sort of classic problem that can be coded up in python or matlab under 5 minutes. it’s just a for loop & an if statement. It won’t tell you much about the actual probability because that number is so small your code will have to run for a long time to get a meaningful answer. Whereas it’s trivial if you just do the math by hand.
it just happens that i dont want to download and install matlab on my computer (which is company provided) and i'd rather work on python if there is an equally good course.
Good luck man. I did a similar kind of thing at one point. A lot of the material is extremely boring and it’s really easy to skim it and pretend you know it in principle. You’ll see... it’s one of those things. The unbridled enthusiasm won’t let you acknowledge it but you know ifs true :) So IMO you’ve got to find some way of doing some exams somewhere.
In the end the only way I could find enough motive to get through a whole applied maths syllabus was to book myself for exams so that I could panic about failing and study.
Suggestion: Set aside complex variables and do measure theory instead. For the Fourier and Laplace transforms, cover those via measure theory. For measure theory, H. Royden, Real Analysis and the first half of W. Rudin, Real and Complex Analysis.
Then take a course in graduate probability which is based on measure theory. Good authors are Loeve, Neveu, Breiman, Chung, among others. So, learn about the cases of convergence including almost sure convergence, a good version of the central limit theorem, proved carefully, the weak and strong laws of large numbers, ergodic theory, and martingale theory.
Learn stochastic differential equations.
Learn some potential theory.
Then do some work in optimization and optimization under uncertainty.
Personally, I think complex analysis is important. I really like Serge Lang's book "Complex Analysis." It is supposedly a graduate level text, but the first half of it covers the basics at a level suitable for upper division undergrad class. Ahlfor's book is also really really good, but a little harder to read in my opinion.
I didn't like either Royden or Rudin. I really like Folland's book "Real analysis: modern techniques and their applications." His Fourier analysis book is also pretty good IMO.
Yes, I've got a copy of Alfors. And at one time I started a course from a student of Alfors. And I have a book by Hille.
I just never could see much utility in complex variables: Complex valued functions of real variables. Sure. Complex valued measures? Sure. Fourier theory making a lot of use of complex numbers? Of course. A vector space where the field is the complex numbers? Certainly. Hermitian and unitary matrices with complex numbers? About have to like those due to the fundamental theorem of algebra, that is, roots of polynomials and, thus, complex eigenvalues. Functions of a complex variable? Never could see the utility.
Complex valued functions of a real variable are common and important but well covered in real analysis. Functions of a complex variable I've never seen in practice.
A good course in statistics, e.g., with sufficient statistics, needs graduate probability, and with a grad probability course can do elementary statistics easily in the footnotes. E.g., the proof I worked out for the Neyman-Pearson lemma is quite general but based on the Hahn decomposition based on the Radon-Nikodym theorem (Rudin gives von Neumann's cute proof) in measure theory. The grown up approach to conditional probability is based just on the Radon-Nikodym theorem of measure theory.
Can someone please explain how MIT course numbers work? I get that "18" means maths, but why do specific courses sometimes have two digits after the "." and sometimes three, and if the level of study is encoded in it, why does it seem to jump to starting with "6"?
Two numbers like 18.XX means that it is intended for non-math majors so 18.02 is the basic multivariable calculus that every MIT alum needs to graduate, 18.022 is multivariable calculus with an emphasis on theory and 18.023 is with an emphasis on application and usually taken by people considering math majors (though also fulfill the core requirement) The class numbering schema is by field: http://math.mit.edu/academics/classes.php
There is no strict system after the “.” however the 3-numeral course numbers tend to be advanced/grad classes. There are some smaller patterns within each major but nothing knits those together.
6.82x tend to be systems classes like networks or OS. But 17.828 isn’t like an intro to constitutional law and 6.42 isn’t Causes and Prevention of Software Project Failure.
This is misleading and false when it comes to Course 18. Math classes numbering has much more structure.
18.XYZ where Z >= 5 are grad classes.
X = 1 is Analysis
X = 2 is Mathematical Physics
X = 3 is Applied Mathematics
X = 4 is Theoretical Computer Science
X = 5 is Logic
X = 7 is Algebra
X = 9 is Topology
I studied a very similar curriculum over ten years ago during my Physics undergrad. While it’s not strictly “useful” in a day to day sense, nor do I remember much of the details of these courses or methods, I can tell you that studying applied math has served me very well during my career as an actuary.
Studying applied math gives you the skills of being able to approach problems in a logical, analytical, and above all PERSISTENT way (because let me tell you will you ever have some late nights banging your head on the desk trying to wrap your brain around these problems). These skills are crucial in any field where you need to create mathematical representations of a system - or break apart and understand someone else’a models.
Also - partial differential equations are just cool! They are like a mathematical rubick’s cube. Solving them will legitimately make you smarter.
Is there a place to find answers to textbook questions? I get that this is discouraged, because it means paying students can cheat. But for people who are self studying, there are many times you think you understand, but you're not sure and would like to see.
In particular, I'm hoping to self study the Structure and Interpretation of Classical Mechanics, but can't find the answers anywhere.
I think that music theory lends itself to a pace that can be dramatically increased and at the same time learnt successfully.
I don't think math works like this. I specifically decreased my pace when doing math courses that I hadn't done in undergraduate and I am glad that I took that approach. I now see any problem (given enough time) as more of a research question than just a box to tick.
> He taught me “the standard pace is for chumps” — that the system is designed so anyone can keep up. If you’re more driven than most people, you can do way more than anyone expects. And this applies to all of life — not just school
Yeah, it is, its more of a loose goal, I'm doing research and working on finishing 2 Master's alongside it, so we'll see what happens! I really just want to learn and grow, so if it takes months or if it takes years, I'm in!
This is a cool thing to do. I followed a slightly similar path in that my undergraduate degree did biochemistry up to 3rd year, but at the expense of third year pure math courses. (I had a kind of applied PDEs and modelling course but applied to biological problems like gene flow, disease modelling and population modelling.)
I ended up doing honours in bioinformatics, but then took a year to do third year pure math (and the honours course in lattice theory and topology). After this I ended up doing what I really wanted to, which was an MSc in pure mathematics.
I also did probably the first 1/3 of Strang's course (even though I had already done linear algebra formally in my original course). I think online math courses are a mixed bag and it could even be that it works better when you have an idea already of why you want to do it and the applications thereof. Applications need not imply applied math—you can apply mathematics to other branches of mathematics.
The more difficult thing for me is to independently find research topics and subtopics. Mentorship in mathematics is very important.
One question for all the self-learners here (I have immense admiration). Do these courses inform your day to day job in some way? Or is this for personal knowledge (even more impressive)? How do you find the damn time? Need some suggestions. My goals are infinite and my time is so short.
If learning is part of a well-rounded set of activities that you are doing in your life, then it will lead to several positive effects. One, learning new things, by itself, with nothing else, reduces cognitive decline and increase brain health. But optimal learning and brain health also derives from physical activity and sleep, so it's a small part of the puzzle.
EDIT: Brain health also derives from a healthy social life, I forgot to add that to the list involved. Having friends that are interested in learning the same things as you, and discussing it with them, help you to develop self-checking techniques.
Also, Alan Kay pointed out in a speech in the last few years that our economy is doing really poorly right now producing researchers, and people who understand research, and you really can't get there without a good education.
I'll go one further and suggest that 'Transfer', which is the application of understanding of one topic to another, is one of the prime sources of what we consider 'innovation' or 'paradigm shifts'.
I feel as if that should be enough proof that education is an end to itself, but I need to be really clear: continuing education doesn't need to have a goal, and I would strongly discourage you from adding one. I go to the gym all the time without goals. I go for walks everyday, without goals. Each of these items are basic maintenance. Learning new things _well_ is basic maintenance for your brain. If you want to do continuous learning then you should install Anki, get a notebook and learn Cornell notes, then go out and take a quick class, lecture, or study material.
If you are learning something new, the fastest way to learn and retain the material is to move to self-testing as quickly as possible. I can go over this in more detail, but perhaps the first class I would recommend would be "Learning How To Learn" (Coursera), which produced a serious attitude shift in me about learning and what I am capable of as a human.
I hope you get some good answers, but just consider that you'll never achieve satisfaction by having infinite goals (in any part of life). All the tips and self-discipline in the world might only take you from X to 10X, and somewhere down the line you'll have to learn to be okay with that. The alternative is to have the health of your life & the lives of those around you be consumed by ambition and its pursuit.
Great idea! I've used OCW to "re-take" a class I last had 25 years ago, plus a couple I never took, but never through to do a whole program.
A suggestion: 18.01 and 18.02 are typically taken in the first two semesters of freshman year (and many take 18.03 the first of sophomore year, or even the semester before). As you're well past the high schoolers who push up at MIT, I suggest 18.014 / 18.024 / 18.034 for more theory (those weren't available when I was a student).
It's funny: when I was an undergrad Math was quote popular because it had so few requirements and on your page it still looks that way (compare to course 6). I wonder if they same dynamic still applies?
It's amazing and wonderful you can get an MIT education from watching youtube videos of their lectures. I've used them to fill in gaps in my education. Highly recommended, and it's to MIT's great credit that they're doing this.
To really get an MIT education, you should team up with one or two other folks and work through the labs and problem sets together. There’s no substitute for the learning you get by explaining your understanding of a problem and hashing out how to approach it with someone else — and by building things!
I mean julia is much younger. The main strengths are that it's open source, it's a much nicer language, you can actually ship things in it (good luck doing that in matlab), package manager, etc etc. Matlab is legit not great.
[0]: https://www.youtube.com/watch?v=ZK3O402wf1c&list=PL49CF3715C...
[1]: https://www.youtube.com/watch?v=Cll03FUxjuk
[2]: https://ocw.mit.edu/courses/mathematics/18-650-statistics-fo...
[3]: https://www.amazon.com/Computations-Hopkins-Studies-Mathemat...