你不是没学好,
是被教得太好了
1990 年,哈佛物理系教授 Eric Mazur 碰上了一件难堪的事。
他读到 Halloun 和 Hestenes 的几篇论文,说大学生学完一学期力学,脑子里装的多半还是亚里士多德那一套。他起初不信,觉得说的是别人家的学生,就把他们的概念测验(后来“力学概念测验”的前身)拿来考自己班上。
这类题不用计算,问的都是最朴素的事。比如:一辆重卡车迎面撞上一辆小轿车,谁受的力更大?多数人选卡车。答案是一样大,牛顿第三定律。
结果很难看。这套简单的题,哈佛学生的成绩只比一次他自认为难得多的期中考试好一点。有个学生问他:这些题,是按您教的答,还是按我平时的想法答?
这个问题把病根说穿了。学生脑子里有两套物理,一套用来考试,一套是自己真信的,两套之间没有连线。
这不是哈佛独有的毛病。1998 年,Hake 汇总了 62 门大学物理入门课、6542 名学生的开学和期末测验:
标准化增益的意思是:开学时你离满分差多少,一学期下来补回了其中几成。传统讲授的课平均只补回两成多。讲课、作业、考试走完一整轮,学生对“力”的基本理解,四分之三的缺口原封不动。
费曼 1950 年代初在巴西教书时,见过同样的事。他给一班将来要当物理老师的学生每人发了一条偏振片,问怎样确定它的偏振方向,还提示他们看看窗外海湾的反光。没人吭声。他换个问法:布儒斯特角是什么?学生立刻背出来,反射光在这个角度完全偏振,角的正切等于折射率。可他们没想到,海湾的水面就是那个“有折射率的介质”。直到他让大家拿偏振片对着海湾转一转,他们才惊讶地发现,反射光真是偏振的。费曼的判断很干脆:这些学生什么都背下来了,却不懂任何一样东西的意思。
会背、会算、会考,和会用,是两回事。下面先讲为什么会这样,再讲这样训练出来的人最后去了哪里,最后讲该怎么学。
知识是按线索存的
认知心理学有一条很硬的结论,叫编码特异性原则(encoding specificity),1973 年由 Tulving 和 Thomson 提出:一段记忆能不能被想起来,要看想的时候手里的线索,和存进去时的情境重合了多少。
最出名的演示来自潜水员。1975 年,Godden 和 Baddeley 让潜水员分别在岸上和水下背单词,再分别在岸上和水下回忆。在哪儿背的,回到哪儿就想起得更多,换了地方,回忆量少了三成左右。(这个经典实验 2021 年被 Murre 直接重复过,没能干净地复现,所以别把它当铁证。编码特异性本身另有大量证据支撑。)
放到学习上,后果很直接。你在“第四章、期末考试、选择题第 7 题”这个情境里学会梯度下降,它的检索线索就是这几样。到了工作里,线索长得完全不同:模型的 loss 卡在 2.3 不动了。两组线索没有交集。
知识明明在脑子里,就是调不出来。怀特海 1929 年就给这种东西起过名字,叫“惰性观念”(inert ideas):
1980 年,Gick 和 Holyoak 把“调不出来”量了出来。被试先读一个故事:一位将军要攻打要塞,通往要塞的每条路都埋了地雷,大军一齐走就会引爆。将军把部队拆成小队,从四面八方同时进发,在要塞下会合。
接着解一道医学难题,心理学里叫 Duncker 放射线问题。病人胃里有个肿瘤,射线够强才能杀死它,可这么强的射线会烧坏沿途的健康组织。怎么办?
两道题结构一模一样:分散成几路弱的,同时汇聚到一点。没读过故事的人,想到这个解法的不到一成。读过故事的人呢?在其中一个实验里,故事混在几个不相干的故事中间,没人提醒时只有 20% 的人想到用它。主试多说一句“前面某个故事也许有用”,比例升到 92%。
所以“会做题但解决不了问题”,可以下一个精确的定义:你有答案,但没有通往答案的路。
章节替你做了最难的一步
翻开任何一本教材的课后习题,都暗含四条保证。拿它们和真实的难题对照一下:
Herbert Simon 1973 年把右边这类问题叫作“病构问题”(ill-structured problems)。在他看来,很多被当成良构的问题,其实更该看作病构的。他还说过,解题的力气大多花在给问题搭结构上,搭好之后真正“求解”的部分只占一小块。
再看你是怎么练的。学完 3.2 节,做 3.2 节后面的习题。你根本不用判断这道题该用什么方法,章节号已经说了。可在真实问题里,这一步往往最难。
这个差别可以直接测出来。2007 年,Rohrer 和 Taylor 让大学生学四种几何体的体积公式。一组按类型分块练,练完一种再练下一种;另一组把四类题打乱了混着练。
练习当场,分块组的正确率高得多,看起来学得更好。一周后测试,结果倒了过来:分块组只剩两成,混练组六成多。分块练习制造的,是“我会了”的错觉。
Chi、Feltovich 和 Glaser 1981 年请物理新手和专家给一堆力学题分类。新手按表面分:这是斜面题,那是弹簧题。专家按原理分:这道归能量守恒,那道归牛顿第二定律。按章节编排的课本,天天在训练前一种分法。
道理其实很简单。分块练的时候,章节号就是一个现成的提示:这一节的题,全用这一节的方法。你完全可以不读懂题目,只看它排在第几节,就知道该套哪个公式。练习一题不错,可“看题判断该用什么方法”这项本事,一次也没练到。
这不是说学生偷懒。人脑天生爱抄近路,有近路可抄,就不会去修远路。混着练,近路被堵死了,你只能老老实实读题、判断。一周后的测试考的正是这一步,差距就出在这里。
回头看图 1。课本把察觉、定义、筛选、选工具全替你做完了,答案印在书后。你苦练十几年的,是难题里最容易的那一段。
你像一个过拟合的 AI
做人工智能的人有个词,叫过拟合:模型把练习题背得滚瓜烂熟,一换题就不会。学生身上的毛病,和它一模一样。
Beery 等人 2018 年的论文开头举过一个例子:一个商用图像识别服务,草场上的牛认得很准,牛一站到沙滩和海浪边,就认不出来,或者认得很差。它学到的“牛”,有相当一部分是牛身后的草地。Geirhos 等人 2020 年把这类现象归到一起,叫捷径学习:模型在练习数据里找到一条省事的规则,考的题和练的题长得像,它就百发百中;换个样子,立刻垮掉。
学生学到的捷径如出一辙。这一节的题,用这一节的公式。题目给了几个数,就要用上几个数。答案多半是整数,算出个奇怪的根号就该回头检查。老师反复强调的,一定会考。
每一条在考场上都很灵,因为考试和课后习题长得一样。考试分数只说明你在练过的那类题上表现如何,说明不了换一类题你还行不行。高分不等于学的是捷径,可考试也查不出捷径,因为它从来不出练习范围以外的题。
捷径学得最牢的人,自己往往察觉不到。Rozenblit 和 Keil 2002 年做过一个实验:先请人给自己对拉链、抽水马桶、圆柱锁这类东西的理解打分,再请他们一步一步写出原理,然后重新打分。写完之后,分数明显往下掉。他们管这叫“解释深度的错觉”。
不妨现在就试一下。合上书,把梯度下降画出来。不是写出那行更新公式,是画。
画出来是这样。把要优化的目标想成一片山地,最低点在谷底。梯度下降就是闭着眼睛下山:每一步都朝脚下最陡的方向迈。
山谷又窄又长的时候,最陡的方向几乎横着指向对面谷壁,而不是顺着谷底往前。步子迈大了,就在两侧谷壁之间来回弹,像图里的红线;步子迈小了,不弹,可半天挪不了几步,像灰线;再大一点,直接飞出山谷。
做机器学习的人天天在调的东西,都能在这张图里找到来历。学习率为什么不能太大,看红线。为什么要加“动量”:让小球带着惯性往下滚,左右的来回弹互相抵消,往前的速度越积越多。Adam 这类方法在干什么:给每个方向换一把尺子,把扁长的山谷拉圆。
一张图讲清楚这么多事,那行公式一件也没告诉你。公式是这张图的压缩。只会写公式、画不出图的人,手里拿着一个解不开的压缩包。
基础是最后才有的
看看人类自己是按什么顺序学会微积分的。
微积分被用了一个半世纪,解决了一大批实际问题。这一个半世纪里,没人说得清无穷小是什么。今天课本第一章那套极限的严格定义,到十九世纪下半叶才定型。集合论公理更晚,到了二十世纪。
数学的地基,是最后才浇的。教材把顺序整个倒了过来:先给你最晚发现的东西,再给你最早发现的东西,这一切原本是为了解决什么问题,常常留到最后,或者干脆不提。Lakatos 在《证明与反驳》里管这种写法叫“演绎主义风格”:定义、引理、定理一字排开,看上去天经地义,知识在猜测、反例和修补里长出来的过程,全藏起来了。
更讽刺的证据来自代数课。Koedinger 和 Nathan 2004 年测了 171 名刚学代数的高中生:同一个数量关系,写成文字应用题,正确率 70%;写成纯符号方程,只有 42%。而 Nathan 和 Koedinger 此前调查的几十位高中数学老师和教育研究者,多数预测的是反过来:符号方程更“基础”,应用题更难。研究者管这叫“符号优先观”,持这种看法最多的,是高中老师。Nathan 和 Petrosino 2003 年又发现,数学学得越深的准教师,越容易持这种看法,他们叫它“专家盲点”。
课程设计者眼里的“基础”,和学生实际的认知方向,是反的。
Paul Lockhart 在《一个数学家的叹息》开头写了一个音乐家的噩梦:音乐成了必修课,却只教五线谱、调号和各种记谱规则,孩子们一年年抄写音符,真正去弹、去听,要等到大学,多半是研究生阶段。他说,我们就是这样教数学的。
阿诺尔德讲过一个更短的例子。法国一个小学生被问“2 加 3 等于几”,答“3 加 2,因为加法满足交换律”。他不知道答案是 5,但他知道一条公理。
教育学家 Perkins 给这两种病起了名字。元素病(elementitis):只学零件,从没见过整体。关于病(aboutitis):只学“关于”一件事的知识,从没真正做过这件事。
原理为什么不能先学?打个比方。一条原理像一张会员卡:办卡要先交一笔钱,之后每次消费便宜一点。只来一两次,办卡亏了;来得够多,才回本。
学原理也是这笔账。记住它本身要花力气;它的好处是,之后每见到一个例子,你不用从头死记,只要记下这个例子和原理不一样的那一点。见过的例子越多,原理越省力。可一个例子都没见过,原理就只剩成本。越抽象的原理,会员费越贵,越得先见过足够多的例子才回本。课本把极限的严格定义放在第一章,等于让你一次都没来过,就先交全款。
在你见过它要概括的东西之前,一条原理只是负担。你背下来的不是原理,是一串符号。
螺丝钉的形状
现在可以给螺丝钉下一个精确的定义了:需要别人先把问题定义好、拆好、定好验收标准,才能开始干活的人。
对照图 1。学校十几年训练出来的,就是这个形状。你不是能力不够,是能力长成了螺丝钉的形状。
公司里谁负责出题?架构师、产品负责人、技术负责人、创始人。他们的活,是把一团乱麻的现实变成一张张定义清楚的任务单。剩下的人负责把任务单做完。
招聘把这个闭环焊死了。白板面试和 LeetCode 刷题,测的说到底是你记住了多少“题型到解法”的映射。流传最广的刷题清单只有几十到一两百道,比如 Blind 75、NeetCode 150,刷题就是在扩大一张查找表。学校按考试筛人,公司按刷题筛人,两道关口选出来的是同一种能力。
这种筛法到底准不准?2013 年,时任 Google 人力运营高级副总裁的 Laszlo Bock 接受《纽约时报》采访时说,公司分析过自己的招聘数据:脑筋急转弯式的面试题完全是浪费时间;GPA 和考试分数作为招聘标准几乎没用,只对毕业两三年内的新人有一点点相关,之后就没了。
还有更刺耳的一层。到了 2026 年,求解定义清楚的问题,已经是大模型最擅长、也最便宜的事。给一个清楚的函数签名和一组测试,编程 agent 几分钟就能交卷。如果你全部的本事是“拿到清楚的题目,写出正确的解”,你和一个 AI 工具之间的区别,正在只剩下价格。
螺丝钉正在被更便宜的螺丝钉替换。替换不掉的,是能把乱麻变成题目的人,而这种能力,学校从来不练。
反方向也是坑
网上流行另一种说法:别从基础学起,直接啃最难的东西,碰到不懂的词就一路追问下去,先修课是最蠢的发明。这话说中了一半。只认这一半,会掉进另一个坑。
常见的论据是:孩子学语言从来不是先学语法,一张嘴就是完整的句子。可教育心理学家 Geary 1995 年区分过两类能力。一类是生物原生的,比如说话、认人脸,演化给大脑预备好了专门的机制,泡在环境里就能学会。另一类是生物次生的,比如阅读、代数、编程,是文明的产物,大脑没有为它们预备什么。人类说话少说有几万年,文字只有五千年左右,读写到今天仍然得专门教。
就算学说话,孩子听到的也不是“完整的复杂度”。大人对婴儿说话,会本能地放慢、简化、重复,研究者叫它儿向语言(child-directed speech)。Snow 1972 年就发现,母亲跟两岁孩子说话,比跟十岁孩子说话更简单、更多重复。这本身就是脚手架。
Kirschner、Sweller 和 Clark 2006 年的综述,批评的正是“少给指导、让学生自己发现”的教法:对真正的新手,它不如明确的讲解和样例。Kalyuga 等人 2003 年总结了“专长逆转效应”:对新手有用的详细指导,对行家反而碍事;适合行家的放手不管,会让新手不知所措。
Kapur 的“有成效的失败”(productive failure)给了一个更精细的顺序:先让学生去碰一个超出能力的问题,失败之后再讲。Sinha 和 Kapur 2021 年汇总了 53 项研究,在理解概念、举一反三上,这个顺序稳定地比“先讲后练”好;在熟练计算上,两种顺序差不多;对小学低年级的孩子,反倒是先讲更好。Schwartz 和 Bransford 1998 年发现了类似的事:先让学生分析一组对比案例,再听讲,听讲的收获更大。
所以基础要讲,但不能抢在问题前面讲。
“只有放进大图景里的零件才会被理解”,这句话也说过了头。巴西的街头数学研究是个反例。1985 年,Carraher 夫妇和 Schliemann 考察了巴西累西腓五个在街头卖货的孩子,年龄 9 到 15 岁:
在摊位上,孩子们心算价格几乎全对。同样的运算写成应用题,正确率降了一截;写成学校格式的算式,只剩三成多。样本只有五个孩子,结论要打折扣,方向却很清楚:嵌在情境里的知识,同样被困在了情境里。
出路在 Gick 和 Holyoak 1983 年的后续实验里。只读一个类比故事的人,没人提醒时 21% 想到解法;读两个不同情境的故事、并写下它们的共同点,这个比例升到 45%。抽象不是起点,是比较了几个情境之后浮现出来的东西。
为什么两个故事比一个好?每个故事里,除了真正的道理,还夹着一堆道具:将军、地雷、军队。只看一个故事,你分不清哪些是道理、哪些是道具,只能连道具一起记住。两个故事一比,道具对不上,自然掉了,剩下对得上的那部分才是道理。比的情境越多,混进来的道具越少。
街头的孩子学会的算术进不了教室,也是这个原因:他们只在一个情境里见过它,算术和椰子、摊位、吆喝缠在一起,分不开。课本走了另一个极端,直接把剥干净的“道理”塞给你,一个故事也没给。你没法自己判断这道理说的是什么,只能整个背下来。
Ericsson 研究柏林音乐学院的小提琴学生时发现,最好的那批人,花了大量时间独自做刻意练习,把具体的毛病拆出来反复磨。孤立练习本身没有错。刻意练习针对的是具体的弱点,而弱点只有在完整的演奏里才暴露得出来。没见过整体,就按目录顺序磨零件,才是错的。
只学“高级”、从不回头补基础的人,是另一台机器上的螺丝钉。他们能聊 ChatGPT 背后的 Transformer,会调开源模型,可要他们推一推最基本的那几步数学,或者说清某个设计为什么非这样不可,就卡住了。问题一旦偏离教程,他们手里没有任何可以往下推的东西。
真正的敌人不是基础,是脱嵌。把概念从它要回答的问题里剥出来,在你需要它之前塞给你,再拿它当门槛拦人,这三件事才是病。
该怎么学
把上面的证据拼起来,可以得到一套能直接上手的做法。
- 从一个够不着、但看得懂目标的完整问题出发。不是前沿最难的,是跳一跳能摸到边的。Perkins 的说法是,先玩简化版的整场比赛。
- 碰到不懂的词就往下追问。每次记两件事:这个概念在回答什么问题?没有它会怎样?
- 让基础被问题拉进来,别让课表推进来。看不懂 PCA 的时候去学特征值,它就有了挂钩。
- 每个核心概念,至少在两个不同情境里见过。然后动手比较,把共同点写下来。那个共同点才是你的“基础”。
- 用三个测试检验自己。画出来,讲给外行听,不查资料从零实现。卡在哪一步,哪里就是错觉。
- 去掉章节提示。把不同来源的题混在一起做,逼自己先判断“这是什么问题”。
- 在完整任务里找弱点,再拆开练。练完回到完整任务,看弱点还在不在。
写在最后
考试问你:你记不记得答案。世界问你:你知不知道问题是什么。
学校十几年只训练了前一种能力,为此给你发了文凭。公司为此给你发工资。现在,机器正在用更低的价格提供同一种能力。
这不全怪你。课本、考试、招聘一环扣一环,每一环都在奖励同一件事。可你现在知道了,接下来怎么学,是你自己的事。
基础不是起点,
是摸爬滚打之后,
回头才看清的地基。
你第一次发现“学过的东西调不出来”,是在什么时候?哪一样知识,你是后来在真问题里才真正弄懂的?
- Mazur, E. (1997), Peer Instruction: A User's Manual, Prentice Hall
- Halloun & Hestenes (1985), Am. J. Phys. 53: 1043;Hestenes, Wells & Swackhamer (1992), Phys. Teach. 30: 141
- Hake, R. R. (1998), Am. J. Phys. 66: 64–74
- 费曼《别闹了,费曼先生》,“O Americano, Outra Vez!”一章
- Tulving & Thomson (1973), Psychol. Rev. 80: 352–373
- Godden & Baddeley (1975), Br. J. Psychol. 66: 325–331;Murre (2021), R. Soc. Open Sci. 8: 200724
- Whitehead, A. N. (1929), The Aims of Education and Other Essays
- Gick & Holyoak (1980), Cogn. Psychol. 12: 306–355;(1983), 15: 1–38
- Simon, H. A. (1973), Artificial Intelligence 4: 181–201
- Rohrer & Taylor (2007), Instr. Sci. 35: 481–498
- Chi, Feltovich & Glaser (1981), Cogn. Sci. 5: 121–152
- Beery, Van Horn & Perona (2018), ECCV;Geirhos et al. (2020), Nat. Mach. Intell. 2: 665–673
- Rozenblit & Keil (2002), Cogn. Sci. 26: 521–562
- Lakatos, I. (1976), Proofs and Refutations, Cambridge UP
- Koedinger & Nathan (2004), J. Learn. Sci. 13: 129–164;Nathan & Koedinger (2000), Cogn. Instr. 18: 209–237;Nathan & Petrosino (2003), Am. Educ. Res. J. 40: 905–928
- Lockhart《一个数学家的叹息》(2009);Arnold, V. I. (1998), Russian Math. Surveys 53: 229–236
- Perkins, D. (2009), Making Learning Whole, Jossey-Bass
- Bryant, A. (2013-06-20), In Head-Hunting, Big Data May Not Be Such a Big Deal, NYT
- Geary, D. C. (1995), Am. Psychol. 50: 24–37;Snow, C. E. (1972), Child Dev. 43: 549–565
- Kirschner, Sweller & Clark (2006), Educ. Psychol. 41: 75–86;Kalyuga et al. (2003), Educ. Psychol. 38: 23–31
- Sinha & Kapur (2021), Rev. Educ. Res. 91: 761–798;Schwartz & Bransford (1998), Cogn. Instr. 16: 475–522
- Carraher, Carraher & Schliemann (1985), Br. J. Dev. Psychol. 3: 21–29
- Ericsson, Krampe & Tesch-Römer (1993), Psychol. Rev. 100: 363–406
You Didn't Learn Badly.
You Were Taught Too Well.
In 1990, Eric Mazur, a physics professor at Harvard, ran into something embarrassing.
He had read a few papers by Halloun and Hestenes saying that after a semester of mechanics, most college students still carried Aristotle's physics around in their heads. At first he didn't believe it. Surely they meant other people's students. So he took their concept test (the forerunner of what became the Force Concept Inventory) and gave it to his own class.
Questions like these involve no calculation. They ask about the most basic things. For example: a heavy truck collides head-on with a small car. Which one feels the larger force? Most people choose the truck. The answer is that the forces are equal, by Newton's third law.
The results were ugly. On this simple test, his Harvard students did only slightly better than on a midterm he considered far harder. One student asked him: should I answer these questions according to what you taught us, or according to how I usually think about these things?
That question laid bare the root of the trouble. The students had two versions of physics in their heads, one for exams and one they actually believed, with no wiring between the two.
This is not a problem peculiar to Harvard. In 1998, Hake pooled the start-of-term and end-of-term test results of 6,542 students in 62 introductory college physics courses:
Normalized gain means this: take the gap between your score at the start of term and a perfect score, and ask what fraction of it you closed by the end. Traditionally taught courses closed, on average, only a little over a fifth. After a full cycle of lectures, homework and exams, three quarters of the gap in students' basic understanding of “force” was left untouched.
Feynman saw the same thing when he taught in Brazil in the early 1950s. He gave each student in a class of future physics teachers a strip of polaroid and asked how they could find its direction of polarization, with a hint to look at the light reflected off the bay outside the window. Nobody said a word. He tried another question: what is Brewster's angle? The students recited it at once: light reflected at that angle is completely polarized, and the tangent of the angle equals the index of refraction. Yet it never occurred to them that the water of the bay was just such a “medium with an index of refraction.” Only when he had everyone turn their polaroids while looking at the bay were they amazed to find that the reflected light really was polarized. Feynman's verdict was blunt: the students had memorized everything, but they didn't know what anything meant.
Being able to recite, calculate and pass exams is one thing; being able to use what you know is another. Below I first explain why this happens, then where people trained this way end up, and finally how to learn.
Knowledge is stored by its cues
Cognitive psychology has a solid finding called the encoding specificity principle, proposed by Tulving and Thomson in 1973: whether a memory can be recalled depends on how much the cues you have at the moment of recall overlap with the context in which it was stored.
The most famous demonstration involves divers. In 1975, Godden and Baddeley had divers learn lists of words either on shore or underwater, and then recall them either on shore or underwater. They remembered more when they recalled in the same place they had learned; changing places cut recall by roughly 30%. (Murre ran a direct replication of this classic experiment in 2021 and could not reproduce it cleanly, so don't treat it as ironclad. Encoding specificity itself rests on a large body of other evidence.)
Applied to learning, the consequence is direct. If you learned gradient descent in the context of “Chapter 4, final exam, multiple-choice question 7,” then those are its retrieval cues. At work the cues look completely different: your model's loss is stuck at 2.3 and won't budge. The two sets of cues have nothing in common.
The knowledge is right there in your head; you just can't get it out. Back in 1929, Whitehead gave this kind of thing a name, “inert ideas”:
In 1980, Gick and Holyoak put a number on this failure to retrieve. Participants first read a story: a general wants to capture a fortress, but every road leading to it is mined, and a large army marching together would set off the mines. So the general splits his army into small groups, which set out from every direction at once and meet at the fortress.
Then they tackled a medical puzzle, known in psychology as Duncker's radiation problem. A patient has a tumor in the stomach. Only rays strong enough will destroy it, but rays that strong would also burn the healthy tissue they pass through on the way. What can be done?
The two problems have exactly the same structure: split into several weak forces that converge on one point at the same time. Of the people who had not read the story, fewer than one in ten came up with this solution. And those who had? In one experiment, with the story buried among several unrelated ones, only 20% thought of using it when no one prompted them. When the experimenter added a single hint that one of the earlier stories might help, the figure rose to 92%.
So “good at exercises but unable to solve problems” can be given a precise definition: you have the answers, but no path to them.
The chapter does the hardest step for you
Open any textbook to the exercises at the end of a chapter, and you'll find four unspoken guarantees. Set them against what real hard problems look like:
In 1973, Herbert Simon called problems like those on the right “ill-structured problems.” In his view, many problems treated as well-structured are better seen as ill-structured. He also observed that most of the effort in problem solving goes into giving the problem a structure; once that is done, the actual “solving” is only a small part.
Now look at how you practice. You finish Section 3.2, then do the exercises at the end of Section 3.2. You never have to decide which method a problem calls for, because the section number has already told you. Yet in real problems, that step is often the hardest.
This difference can be measured directly. In 2007, Rohrer and Taylor had college students learn the volume formulas for four kinds of solids. One group practiced in blocks by type, finishing one kind before moving on to the next; the other group practiced all four kinds shuffled together.
During practice, the blocked group scored much higher and seemed to be learning better. On the test a week later, the result flipped: the blocked group was down to about 20%, while the mixed group scored over 60%. What blocked practice produces is the illusion of “I've got this.”
In 1981, Chi, Feltovich and Glaser asked physics novices and experts to sort a pile of mechanics problems. Novices sorted by surface features: this is an inclined-plane problem, that is a spring problem. Experts sorted by principle: this one belongs to conservation of energy, that one to Newton's second law. Textbooks organized by chapter train the first kind of sorting every day.
The reason is actually simple. In blocked practice, the section number is a ready-made hint: every problem in this section uses this section's method. You can skip understanding the problem altogether and know which formula to apply just from the section it appears in. You may get every exercise right, yet the skill of “reading a problem and deciding which method it needs” never gets practiced even once.
This isn't to say students are lazy. The brain loves a shortcut: when one is available, it won't bother building the long road. Mixed practice blocks the shortcut, so you have no choice but to read the problem properly and decide. The test a week later tests exactly that step, and that is where the gap comes from.
Look back at Figure 1. The textbook has done the noticing, defining, filtering and choosing of tools for you, and the answers are printed at the back. What you have drilled for more than a decade is the easiest stretch of a hard problem.
You're like an overfitted AI
People who work in AI have a word for this: overfitting. A model learns its practice problems inside out, then fails the moment the problems change. Students suffer from exactly the same thing.
Beery and colleagues open their 2018 paper with an example: a commercial image-recognition service identified cows in pastures accurately, but once a cow stood on a beach by the waves, it failed to recognize the cow or did so poorly. A good part of what it had learned as “cow” was the grass behind the cow. In 2020, Geirhos and colleagues gathered such phenomena under one name, shortcut learning: a model finds a convenient rule in its training data, and as long as the test questions look like the practice ones, it never misses; change their appearance and it collapses at once.
The shortcuts students learn are just the same. Problems in this section use this section's formula. However many numbers the problem gives you, you must use every one of them. The answer is usually a whole number, so if you get a strange square root, go back and check. Whatever the teacher keeps stressing will be on the exam.
Every one of these works well in the exam hall, because exams look just like the exercises. An exam score tells you only how you do on the kind of problem you have practiced; it can't tell you whether you would still cope with a different kind. A high score doesn't mean you learned shortcuts, but an exam can't catch shortcuts either, because it never asks anything outside the range of what was practiced.
The people who have learned shortcuts most thoroughly are often the last to notice. In 2002, Rozenblit and Keil ran an experiment: they asked people to rate their own understanding of things like zippers, flush toilets and cylinder locks, then asked them to write out, step by step, how each one works, and then had them rate themselves again. After writing, the ratings dropped noticeably. They called this “the illusion of explanatory depth.”
Why not try it now? Close the book and draw gradient descent. Not the one-line update formula: draw it.
Drawn out, it looks like this. Think of the objective you are optimizing as a landscape, with the lowest point at the bottom of a valley. Gradient descent is walking downhill with your eyes closed: at every step, you move in whichever direction is steepest under your feet.
When the valley is long and narrow, the steepest direction points almost straight across at the opposite wall, not along the valley floor. Take steps that are too big and you bounce back and forth between the walls, like the red path in the figure. Take small steps and you don't bounce, but you hardly get anywhere, like the gray path. Make the steps a little bigger still and you fly right out of the valley.
Almost everything people in machine learning tune day after day can be traced back to this picture. Why can't the learning rate be too large? Look at the red path. Why add “momentum”? It lets the ball roll downhill with inertia, so the side-to-side bouncing cancels out while the forward speed keeps building. What do methods like Adam do? They give each direction its own ruler, stretching the flattened valley back into a round one.
One picture explains all of this, and the formula told you none of it. The formula is a compressed version of this picture. Someone who can write the formula but can't draw the picture is holding a zip file they can't unzip.
The foundations came last
Look at the order in which humanity itself learned calculus.
Calculus was in use for a century and a half and solved a great many practical problems. Throughout that century and a half, nobody could say what an infinitesimal was. The rigorous definition of the limit that opens today's textbooks did not take its final shape until the second half of the nineteenth century. The axioms of set theory came later still, in the twentieth.
The foundations of mathematics were poured last. Textbooks turn the order completely around: they give you first what was discovered last, and only then what was discovered first; and what problem all of it was originally meant to solve is often left to the end, or never mentioned at all. In “Proofs and Refutations,” Lakatos calls this way of writing the “deductivist style”: definitions, lemmas and theorems laid out in a row, looking as if they could not be otherwise, while the way knowledge actually grew, through conjectures, counterexamples and repairs, is kept out of sight.
Even more ironic evidence comes from algebra classes. In 2004, Koedinger and Nathan tested 171 high school students who had just begun algebra. When a quantitative relationship was written as a word problem, 70% of answers were correct; when the same relationship was written as a bare symbolic equation, only 42% were. Yet most of the dozens of high school math teachers and education researchers that Nathan and Koedinger had surveyed earlier predicted the reverse: symbolic equations are more “basic,” and word problems are harder. The researchers called this the “symbol precedence view,” and the group that held it most widely was high school teachers. In 2003, Nathan and Petrosino further found that the more mathematics prospective teachers had studied, the more likely they were to hold this view. They called it the “expert blind spot.”
What curriculum designers see as “basic” runs opposite to the direction in which students actually think.
Paul Lockhart opens “A Mathematician's Lament” with a musician's nightmare: music has become a compulsory subject, but all that is taught is notation, staves, key signatures and the rules for writing music down. Children copy out notes year after year, and actually playing and listening has to wait until college, more likely graduate school. That, he says, is how we teach mathematics.
Arnold told an even shorter story. A French schoolchild was asked “What is 2 + 3?” and answered: “3 + 2, since addition is commutative.” He didn't know the answer was 5, but he knew an axiom.
Perkins, an education researcher, gave these two ailments names. Elementitis: learning only the parts, never seeing the whole. Aboutitis: learning only knowledge “about” something, without ever actually doing it.
Why can't principles come first? Here is an analogy. A principle is like a membership card: you pay a fee up front to get it, and after that every visit costs a little less. Come only once or twice and the card loses you money; only if you come often enough does it pay for itself.
Learning a principle works the same way. Memorizing it takes effort in itself. The payoff is that afterwards, each time you meet an example, you don't have to memorize it from scratch: you only note the one way this example differs from the principle. The more examples you have seen, the more effort the principle saves. But if you haven't seen a single example, the principle is pure cost. The more abstract the principle, the pricier the membership, and the more examples you need to see before it pays off. Putting the rigorous definition of the limit in chapter 1 is like making you pay in full before you have visited even once.
Until you have seen the things a principle sums up, it is only a burden. What you have memorized isn't a principle; it is a string of symbols.
The shape of a cog
Now we can give a precise definition of a cog: someone who can only start work after someone else has defined the problem, broken it down and set the acceptance criteria.
Compare Figure 1. This is exactly the shape that more than a decade of school trains people into. It's not that you lack ability. Your ability has grown into the shape of a cog.
Who sets the problems in a company? Architects, product leads, tech leads, founders. Their job is to turn the tangled mess of reality into a stack of clearly defined task tickets. Everyone else's job is to get the tickets done.
Hiring welds the loop shut. Whiteboard interviews and grinding LeetCode problems test, in the end, how many “problem type to solution” mappings you have memorized. The most widely shared practice lists run from a few dozen to one or two hundred problems, such as Blind 75 and NeetCode 150; grinding them is just enlarging a lookup table. Schools filter people by exams, companies filter them by coding drills, and the two gates select for the same ability.
Does this filtering actually work? In 2013, Laszlo Bock, then Google's senior vice president of people operations, told The New York Times that the company had analyzed its own hiring data. Brainteaser-style interview questions turned out to be a complete waste of time. GPAs and test scores were almost worthless as hiring criteria: they correlated only slightly with performance for new graduates in their first two or three years, and not at all after that.
There is a harsher layer still. By 2026, solving well-defined problems is the thing large language models do best and most cheaply. Give a coding agent a clear function signature and a set of tests, and it hands in its work within minutes. If all you can do is “take a clear problem and write a correct solution,” the difference between you and an AI tool is shrinking to price alone.
Cogs are being replaced by cheaper cogs. What can't be replaced are the people who can turn a mess into a problem, and that ability is the one school never trains.
The opposite direction is a trap too
Another view is popular online: don't start with the basics. Dive straight into the hardest material, chase down every unfamiliar word as you go, and treat prerequisites as the dumbest invention ever. That view is half right. Take only that half, and you fall into a different trap.
The usual argument is that children never learn grammar first; they open their mouths and out come whole sentences. But in 1995 the educational psychologist Geary distinguished two kinds of ability. One kind is biologically primary, such as speaking or recognizing faces: evolution has given the brain dedicated machinery for them, and simply being immersed in the environment is enough to learn them. The other kind is biologically secondary, such as reading, algebra and programming: these are products of civilization, and the brain has nothing prepared for them. Humans have been talking for tens of thousands of years at least, but writing is only about five thousand years old, and reading and writing still have to be taught explicitly today.
Even when learning to talk, children don't hear “the full complexity.” Adults instinctively slow down, simplify and repeat themselves when talking to babies, which researchers call child-directed speech. As early as 1972, Snow found that mothers spoke more simply and with more repetition to two-year-olds than to ten-year-olds. That in itself is scaffolding.
A 2006 review by Kirschner, Sweller and Clark took aim at exactly the teaching that gives little guidance and lets students discover things for themselves: for genuine beginners, it does worse than explicit instruction and worked examples. In 2003, Kalyuga and colleagues summarized the “expertise reversal effect”: detailed guidance that helps novices gets in experts' way, while the hands-off approach that suits experts leaves novices lost.
Kapur's “productive failure” offers a more refined order: first let students attempt a problem beyond their ability, and explain only after they have failed. In 2021, Sinha and Kapur pooled 53 studies. For conceptual understanding and transfer to new problems, this order consistently beat “explain first, then practice”; for procedural fluency, the two orders came out about even; and for children in the early grades of primary school, explaining first actually worked better. Schwartz and Bransford found something similar in 1998: when students first analyzed a set of contrasting cases and then heard the lecture, they got more out of the lecture.
So the basics do need to be taught, just not ahead of the problem.
“Parts are only understood once they are placed in the big picture”: that claim goes too far as well. The studies of street mathematics in Brazil are a counterexample. In 1985, the Carrahers and Schliemann studied five children, aged 9 to 15, who sold goods on the streets of Recife, Brazil:
At their stalls, the children worked out prices in their heads and got almost all of them right. When the same operations were written as word problems, accuracy dropped a notch; written as bare sums in school format, it fell to just over a third. With only five children in the sample, the conclusion should be taken with a grain of salt, but the direction is clear: knowledge embedded in a context is also trapped in that context.
The way out lies in a follow-up experiment by Gick and Holyoak in 1983. Of those who read just one analogous story, 21% came up with the solution without a hint; of those who read two stories set in different contexts and wrote down what they had in common, 45% did. Abstraction is not the starting point. It is what emerges after you compare several contexts.
Why are two stories better than one? Besides the real idea, each story carries a pile of props: the general, the mines, the army. With only one story, you can't tell which parts are the idea and which are props, so you have to memorize the props along with it. Put two stories side by side and the props don't match, so they simply fall away; what is left, the part that does match, is the idea. The more contexts you compare, the fewer props slip in.
That is also why the arithmetic the street children learned never made it into the classroom: they had only ever seen it in one context, so it was tangled up with coconuts, stalls and hawkers' cries, impossible to pull apart. Textbooks go to the other extreme and hand you the stripped-down “idea” directly, without a single story. You have no way to judge for yourself what the idea is about, so all you can do is memorize it whole.
When Ericsson studied violin students at the music academy in Berlin, he found that the best of them had spent a great deal of time alone in deliberate practice, isolating specific flaws and working on them over and over. Isolated practice is not wrong in itself. Deliberate practice targets specific weaknesses, and weaknesses only show up in a full performance. What is wrong is grinding away at the parts in table-of-contents order without ever having seen the whole.
People who learn only the “advanced” material and never go back to fill in the basics are cogs in a different machine. They can talk about the Transformer behind ChatGPT and tweak open-source models, but ask them to work through the most basic steps of the math, or explain why some design has to be the way it is, and they get stuck. Once a problem strays from the tutorial, they have nothing in hand to push forward with.
The real enemy isn't the basics. It's taking them out of context. Stripping a concept out of the problem it was meant to answer, handing it to you before you need it, and then using it as a gate to keep people out: those three things are the disease.
How to learn
Put the evidence above together, and you get a method you can start using right away.
- Start from a whole problem that is out of your reach but whose goal you can understand. Not the hardest thing at the frontier, but something you can just touch the edge of if you stretch. In Perkins's words, first play a junior version of the whole game.
- When you hit a word you don't understand, chase it down. Each time, write down two things: what question is this concept answering? What would happen without it?
- Let the problem pull the basics in; don't let the syllabus push them in. Learn eigenvalues when PCA stops making sense to you, and they will have a hook to hang on.
- Meet every core concept in at least two different contexts. Then compare them yourself and write down what they have in common. That common part is your real “foundation.”
- Test yourself three ways. Draw it, explain it to a layperson, and implement it from scratch without looking anything up. Wherever you get stuck is where the illusion lies.
- Remove the chapter cues. Mix problems from different sources, and make yourself decide first: “what kind of problem is this?”
- Find your weak spots inside the whole task, then practice them separately. Afterwards, go back to the whole task and see whether the weakness is still there.
In closing
An exam asks you: do you remember the answer? The world asks you: do you know what the problem is?
For more than a decade, school trained only the first ability, and gave you a diploma for it. Companies pay you a salary for it. Now machines are offering the same ability at a lower price.
Not all of this is your fault. Textbooks, exams and hiring lock into one another, and every link rewards the same thing. But now you know, and how you learn from here on is up to you.
The basics are not the starting point.
They are the foundation you see clearly
only looking back, after the scrapes and falls.
When did you first notice that something you had learned just wouldn't come back to you? Which piece of knowledge did you only truly understand later, inside a real problem?
- Mazur, E. (1997), Peer Instruction: A User's Manual, Prentice Hall
- Halloun & Hestenes (1985), Am. J. Phys. 53: 1043; Hestenes, Wells & Swackhamer (1992), Phys. Teach. 30: 141
- Hake, R. R. (1998), Am. J. Phys. 66: 64–74
- Feynman, “Surely You're Joking, Mr. Feynman!”, chapter “O Americano, Outra Vez!”
- Tulving & Thomson (1973), Psychol. Rev. 80: 352–373
- Godden & Baddeley (1975), Br. J. Psychol. 66: 325–331; Murre (2021), R. Soc. Open Sci. 8: 200724
- Whitehead, A. N. (1929), The Aims of Education and Other Essays
- Gick & Holyoak (1980), Cogn. Psychol. 12: 306–355; (1983), 15: 1–38
- Simon, H. A. (1973), Artificial Intelligence 4: 181–201
- Rohrer & Taylor (2007), Instr. Sci. 35: 481–498
- Chi, Feltovich & Glaser (1981), Cogn. Sci. 5: 121–152
- Beery, Van Horn & Perona (2018), ECCV; Geirhos et al. (2020), Nat. Mach. Intell. 2: 665–673
- Rozenblit & Keil (2002), Cogn. Sci. 26: 521–562
- Lakatos, I. (1976), Proofs and Refutations, Cambridge UP
- Koedinger & Nathan (2004), J. Learn. Sci. 13: 129–164; Nathan & Koedinger (2000), Cogn. Instr. 18: 209–237; Nathan & Petrosino (2003), Am. Educ. Res. J. 40: 905–928
- Lockhart, “A Mathematician's Lament” (2009); Arnold, V. I. (1998), Russian Math. Surveys 53: 229–236
- Perkins, D. (2009), Making Learning Whole, Jossey-Bass
- Bryant, A. (2013-06-20), In Head-Hunting, Big Data May Not Be Such a Big Deal, NYT
- Geary, D. C. (1995), Am. Psychol. 50: 24–37; Snow, C. E. (1972), Child Dev. 43: 549–565
- Kirschner, Sweller & Clark (2006), Educ. Psychol. 41: 75–86; Kalyuga et al. (2003), Educ. Psychol. 38: 23–31
- Sinha & Kapur (2021), Rev. Educ. Res. 91: 761–798; Schwartz & Bransford (1998), Cogn. Instr. 16: 475–522
- Carraher, Carraher & Schliemann (1985), Br. J. Dev. Psychol. 3: 21–29
- Ericsson, Krampe & Tesch-Römer (1993), Psychol. Rev. 100: 363–406