Progress on memory systems is bottlenecked on explanation science
(Originally posted as a comment on someone’s non-public post. I felt like re-posting it on my site because I think the thing I’m trying to communicate is pretty important and wish more people understood it. I’ve made some edits so it reads better as a standalone piece and to remove quoted parts of the post for privacy.)
I think current memory systems suck in large part because most expositors’ understanding of the material they are writing about sucks and because explanations suck. Maybe memory systems aren’t the thing to be working on, and the focus should instead be on how to produce better explanations. Or rather, in order to produce great memory systems, maybe we first need to solve the problem of exposition.
In my own SRS practice I have noticed that when I already have no confusions about a topic, Anki cards are a good way of organizing (but not necessarily remembering!) that knowledge: I can write “good” Anki cards. But the cards are hardly even necessary since my understanding is already so durable. I don’t need (and in fact, do not have) any Anki cards about long multiplication or long division. This is not because I know these procedures to the point of automaticity (in the past I have gone probably five or more years without performing either procedure, and have been unsure how to perform them on at least one occasion). Rather, it’s because (1) I understand the underlying principles (how the decimal system works, the distributive property, what a quotient means, etc.) to such a deep extent that I can rederive the procedures even years later; and (2) I have some faint/foggy but still surprisingly good intuition about “how the answer is supposed to come out”, so if I make a few mistakes I can course-correct.
If Anki cards are not so useful for material I understand well, does it follow that Anki cards work well for material I don’t understand well? Well, it turns out that there are other problems when I don’t understand the material well. Some of these you list in your post. When I am trying to Ankify material I don’t understand, it feels like I am grasping at any footholds at all – to climb high enough, to reach some critical mass of understanding – so that I can reach some ‘state of understanding’, so that from that state I can write better cards to solidify the understanding and eventually discard my initial cards (or maybe all cards!). When I am confused about a topic and trying desperately to understand it and have tons of questions, no amount of prompt writing, no superior skill at phrasing prompts, is going to save me; what I need instead are answers to my questions (i.e. better explanations)!
Ok, so Anki doesn’t work well when the learner has good understanding of the material, and when the prompt author has poor understanding of the material. Can we solve this problem by having people with good understanding write the Anki cards for people who are learning the material? I had the benefit of not having to create Quantum Country, so I got to experience it fresh (I have also been keeping up with my reviews for it, for almost 4 years now). So given my personal experience, my response is:
I am often not receptive to even “good” Anki cards unless I already understand the topic! For many Quantum Country cards, I recognize the prompt and can produce the correct answer, but I no longer (or never did) understand why I was asked to memorize them. Not Quantum Country, but a few times I have had the experience of creating a card, then forgetting I created it, and then when I revisit the topic I make the exact same card. But this time, I am making the card because I actually understand the topic, whereas before I was making it while just kind of going through the motions of “trying to make good cards” – I could “remember” the correct answer to the prompt, but I didn’t even really “remember” its existence. So even if an expert prompt author writes the prompts for a topic I am trying to learn, I may be able to store the prompt and its answer somewhere in my mind, but I won’t be looking at the prompt in the right way; it’s still somehow “disconnected”.
Exposition problems leak into the prompts and infect them. Michael Nielsen is one of the top expositors in the world, has written multiple other expository works in the past about quantum computing, and I trust (but cannot verify firsthand) that he has deep understanding of quantum computing. And yet: I vaguely recall that the text vacillated between “computable” and “efficiently computable”, which confused me a lot; I remember being very suspicious of the bit about the Bell inequality but did not see my suspicions addressed; I have completely forgotten how quantum teleportation and quantum search work (and probably never understood them well in the first place) other than the few bits I am asked about (ah, but was it or ?). Most expositions are far worse (either because the author doesn’t actually understand the topic that well or because they do not know how to write a good explanation). I think memory systems and other learning techniques like the Feynman technique just don’t work that well if the exposition one is basing one’s learning on isn’t already amazingly good.
My tentative conclusion from all of the above: understanding of material is the bottleneck to being able to write and be receptive enough to make productive use of the best cards. Another way of saying it: if you wrote a perfectly good explanation, the reader could easily write the cards themselves. Most expositions just don’t have enough there to Ankify in the first place, which is why you have to do the iterated Ankification thing that Michael Nielsen loves to talk about: in part, that Ankification technique is a symptom of the disease of poor exposition.
So how to make progress on producing better exposition? I wrote up some ideas a while ago here.