Essays北极甜虾 · No. 17

会考试和会解决问题,是两种能力

你不是没学好,
是被教得太好了

考试高手为什么解不了真问题
十几年的学校教育,练的是难题里最容易的那一段。这篇把认知科学几十年的实验摆出来,说清三件事:会考试的人为什么一碰真问题就卡住;为什么这样的人最后多半成了螺丝钉;反过来“别学基础”为什么也是坑。最后给一套能直接上手的学法。
一个真实的难题,要走完六步察觉哪里不对劲定义问题到底是什么筛选哪些信息有用选工具该用哪套方法求解把方法用对验收怎样才算好课本里,出题人替你做完了学校练的,全在这一格答案印在书后
图 1一个真实的难题要走完六步。课本习题只留了第五步给你,其余五步,出题人和书后答案替你做了。示意。

1990 年,哈佛物理系教授 Eric Mazur 碰上了一件难堪的事。

他读到 Halloun 和 Hestenes 的几篇论文,说大学生学完一学期力学,脑子里装的多半还是亚里士多德那一套。他起初不信,觉得说的是别人家的学生,就把他们的概念测验(后来“力学概念测验”的前身)拿来考自己班上。

这类题不用计算,问的都是最朴素的事。比如:一辆重卡车迎面撞上一辆小轿车,谁受的力更大?多数人选卡车。答案是一样大,牛顿第三定律。

结果很难看。这套简单的题,哈佛学生的成绩只比一次他自认为难得多的期中考试好一点。有个学生问他:这些题,是按您教的答,还是按我平时的想法答?

这个问题把病根说穿了。学生脑子里有两套物理,一套用来考试,一套是自己真信的,两套之间没有连线。

这不是哈佛独有的毛病。1998 年,Hake 汇总了 62 门大学物理入门课、6542 名学生的开学和期末测验:

6542名学生62 门入门课
0.23传统讲授标准化增益
0.48互动参与标准化增益

标准化增益的意思是:开学时你离满分差多少,一学期下来补回了其中几成。传统讲授的课平均只补回两成多。讲课、作业、考试走完一整轮,学生对“力”的基本理解,四分之三的缺口原封不动。

费曼 1950 年代初在巴西教书时,见过同样的事。他给一班将来要当物理老师的学生每人发了一条偏振片,问怎样确定它的偏振方向,还提示他们看看窗外海湾的反光。没人吭声。他换个问法:布儒斯特角是什么?学生立刻背出来,反射光在这个角度完全偏振,角的正切等于折射率。可他们没想到,海湾的水面就是那个“有折射率的介质”。直到他让大家拿偏振片对着海湾转一转,他们才惊讶地发现,反射光真是偏振的。费曼的判断很干脆:这些学生什么都背下来了,却不懂任何一样东西的意思。

会背、会算、会考,和会用,是两回事。下面先讲为什么会这样,再讲这样训练出来的人最后去了哪里,最后讲该怎么学。

知识是按线索存的

认知心理学有一条很硬的结论,叫编码特异性原则(encoding specificity),1973 年由 Tulving 和 Thomson 提出:一段记忆能不能被想起来,要看想的时候手里的线索,和存进去时的情境重合了多少。

最出名的演示来自潜水员。1975 年,Godden 和 Baddeley 让潜水员分别在岸上和水下背单词,再分别在岸上和水下回忆。在哪儿背的,回到哪儿就想起得更多,换了地方,回忆量少了三成左右。(这个经典实验 2021 年被 Murre 直接重复过,没能干净地复现,所以别把它当铁证。编码特异性本身另有大量证据支撑。)

放到学习上,后果很直接。你在“第四章、期末考试、选择题第 7 题”这个情境里学会梯度下降,它的检索线索就是这几样。到了工作里,线索长得完全不同:模型的 loss 卡在 2.3 不动了。两组线索没有交集。

课堂上存进去的线索工作里冒出来的线索第四章期末考试选择题第 7 题梯度下降loss 卡在 2.3×训练忽快忽慢×线索对不上,知识就调不出来
图 2知识跟着存进去时的线索一起存。工作里冒出来的线索,和课堂上的线索搭不上。示意。

知识明明在脑子里,就是调不出来。怀特海 1929 年就给这种东西起过名字,叫“惰性观念”(inert ideas):

只是被接收进头脑,却从未被使用、检验,也从未被放进新的组合里的观念。怀特海《教育的目的》(1929)

1980 年,Gick 和 Holyoak 把“调不出来”量了出来。被试先读一个故事:一位将军要攻打要塞,通往要塞的每条路都埋了地雷,大军一齐走就会引爆。将军把部队拆成小队,从四面八方同时进发,在要塞下会合。

接着解一道医学难题,心理学里叫 Duncker 放射线问题。病人胃里有个肿瘤,射线够强才能杀死它,可这么强的射线会烧坏沿途的健康组织。怎么办?

两道题结构一模一样:分散成几路弱的,同时汇聚到一点。没读过故事的人,想到这个解法的不到一成。读过故事的人呢?在其中一个实验里,故事混在几个不相干的故事中间,没人提醒时只有 20% 的人想到用它。主试多说一句“前面某个故事也许有用”,比例升到 92%。

所以“会做题但解决不了问题”,可以下一个精确的定义:你有答案,但没有通往答案的路。

章节替你做了最难的一步

翻开任何一本教材的课后习题,都暗含四条保证。拿它们和真实的难题对照一下:

课后习题的保证真实难题的样子
问题已经定义好了题干写明了求什么
没人告诉你问题是什么,很多时候连“哪里出了问题”都要自己发现
信息不多不少每个条件都要用上
一半缺失,一半是噪音;缺的那部分要花钱、花时间去查
有唯一标准答案书后对得上就算对
几个方案各有代价,“好”的标准得你自己定,还得说服别人
方法就在本章刚学什么就用什么
工具散落在整个知识体系里,也可能还没人发明出来
图 3左栏是课本给你的四条保证,右栏是它们在真实问题里的样子。

Herbert Simon 1973 年把右边这类问题叫作“病构问题”(ill-structured problems)。在他看来,很多被当成良构的问题,其实更该看作病构的。他还说过,解题的力气大多花在给问题搭结构上,搭好之后真正“求解”的部分只占一小块。

再看你是怎么练的。学完 3.2 节,做 3.2 节后面的习题。你根本不用判断这道题该用什么方法,章节号已经说了。可在真实问题里,这一步往往最难。

这个差别可以直接测出来。2007 年,Rohrer 和 Taylor 让大学生学四种几何体的体积公式。一组按类型分块练,练完一种再练下一种;另一组把四类题打乱了混着练。

分块练:一类练完再练下一类混着练0%25%50%75%100%89%60%练习当场20%63%一周后测试同一批材料、同样多的练习题,只是顺序不同
图 4按 Rohrer & Taylor(2007)实验 2 报告的平均正确率重画。练习当场分块组领先,一周后倒了过来。

练习当场,分块组的正确率高得多,看起来学得更好。一周后测试,结果倒了过来:分块组只剩两成,混练组六成多。分块练习制造的,是“我会了”的错觉。

Chi、Feltovich 和 Glaser 1981 年请物理新手和专家给一堆力学题分类。新手按表面分:这是斜面题,那是弹簧题。专家按原理分:这道归能量守恒,那道归牛顿第二定律。按章节编排的课本,天天在训练前一种分法。

道理其实很简单。分块练的时候,章节号就是一个现成的提示:这一节的题,全用这一节的方法。你完全可以不读懂题目,只看它排在第几节,就知道该套哪个公式。练习一题不错,可“看题判断该用什么方法”这项本事,一次也没练到。

这不是说学生偷懒。人脑天生爱抄近路,有近路可抄,就不会去修远路。混着练,近路被堵死了,你只能老老实实读题、判断。一周后的测试考的正是这一步,差距就出在这里。

回头看图 1。课本把察觉、定义、筛选、选工具全替你做完了,答案印在书后。你苦练十几年的,是难题里最容易的那一段。

你像一个过拟合的 AI

做人工智能的人有个词,叫过拟合:模型把练习题背得滚瓜烂熟,一换题就不会。学生身上的毛病,和它一模一样。

Beery 等人 2018 年的论文开头举过一个例子:一个商用图像识别服务,草场上的牛认得很准,牛一站到沙滩和海浪边,就认不出来,或者认得很差。它学到的“牛”,有相当一部分是牛身后的草地。Geirhos 等人 2020 年把这类现象归到一起,叫捷径学习:模型在练习数据里找到一条省事的规则,考的题和练的题长得像,它就百发百中;换个样子,立刻垮掉。

学生学到的捷径如出一辙。这一节的题,用这一节的公式。题目给了几个数,就要用上几个数。答案多半是整数,算出个奇怪的根号就该回头检查。老师反复强调的,一定会考。

每一条在考场上都很灵,因为考试和课后习题长得一样。考试分数只说明你在练过的那类题上表现如何,说明不了换一类题你还行不行。高分不等于学的是捷径,可考试也查不出捷径,因为它从来不出练习范围以外的题。

捷径学得最牢的人,自己往往察觉不到。Rozenblit 和 Keil 2002 年做过一个实验:先请人给自己对拉链、抽水马桶、圆柱锁这类东西的理解打分,再请他们一步一步写出原理,然后重新打分。写完之后,分数明显往下掉。他们管这叫“解释深度的错觉”。

不妨现在就试一下。合上书,把梯度下降画出来。不是写出那行更新公式,是画。

起点最低点大步走:在谷壁之间来回弹小步走:不弹,但走得慢
图 5一个又窄又长的山谷,一圈圈是等高线。红线每步迈得大,在两侧谷壁之间来回弹;灰线每步迈得小,不弹,但走得慢。两条路都走了 16 步,由程序算出。

画出来是这样。把要优化的目标想成一片山地,最低点在谷底。梯度下降就是闭着眼睛下山:每一步都朝脚下最陡的方向迈。

山谷又窄又长的时候,最陡的方向几乎横着指向对面谷壁,而不是顺着谷底往前。步子迈大了,就在两侧谷壁之间来回弹,像图里的红线;步子迈小了,不弹,可半天挪不了几步,像灰线;再大一点,直接飞出山谷。

做机器学习的人天天在调的东西,都能在这张图里找到来历。学习率为什么不能太大,看红线。为什么要加“动量”:让小球带着惯性往下滚,左右的来回弹互相抵消,往前的速度越积越多。Adam 这类方法在干什么:给每个方向换一把尺子,把扁长的山谷拉圆。

一张图讲清楚这么多事,那行公式一件也没告诉你。公式是这张图的压缩。只会写公式、画不出图的人,手里拿着一个解不开的压缩包。

基础是最后才有的

看看人类自己是按什么顺序学会微积分的。

1660s
牛顿、莱布尼茨各自发明微积分,用“无穷小”来算。无穷小到底是什么,谁也说不清。
1700s
欧拉、伯努利家族、拉格朗日拿它解决了力学、天文、振动里的大批问题。1734 年,贝克莱主教写《分析学家》,讥讽无穷小是“消逝量的鬼魂”。数学家照用不误。
1821
柯西出版《分析教程》,把极限放到中心。
1861
魏尔斯特拉斯在柏林讲课,今天课本里极限的严格定义(ε-δ)写进了讲义。
1908
策梅洛发表集合论公理。1922 年,Fraenkel 和 Skolem 各自补上替换公理,才有了今天 ZF 的样子。
今天
教材的顺序:先讲集合和极限的严格定义,再求导,最后(也许)讲应用。
图 6人类学微积分的顺序,和教材教微积分的顺序,方向相反。

微积分被用了一个半世纪,解决了一大批实际问题。这一个半世纪里,没人说得清无穷小是什么。今天课本第一章那套极限的严格定义,到十九世纪下半叶才定型。集合论公理更晚,到了二十世纪。

数学的地基,是最后才浇的。教材把顺序整个倒了过来:先给你最晚发现的东西,再给你最早发现的东西,这一切原本是为了解决什么问题,常常留到最后,或者干脆不提。Lakatos 在《证明与反驳》里管这种写法叫“演绎主义风格”:定义、引理、定理一字排开,看上去天经地义,知识在猜测、反例和修补里长出来的过程,全藏起来了。

更讽刺的证据来自代数课。Koedinger 和 Nathan 2004 年测了 171 名刚学代数的高中生:同一个数量关系,写成文字应用题,正确率 70%;写成纯符号方程,只有 42%。而 Nathan 和 Koedinger 此前调查的几十位高中数学老师和教育研究者,多数预测的是反过来:符号方程更“基础”,应用题更难。研究者管这叫“符号优先观”,持这种看法最多的,是高中老师。Nathan 和 Petrosino 2003 年又发现,数学学得越深的准教师,越容易持这种看法,他们叫它“专家盲点”。

课程设计者眼里的“基础”,和学生实际的认知方向,是反的。

Paul Lockhart 在《一个数学家的叹息》开头写了一个音乐家的噩梦:音乐成了必修课,却只教五线谱、调号和各种记谱规则,孩子们一年年抄写音符,真正去弹、去听,要等到大学,多半是研究生阶段。他说,我们就是这样教数学的。

阿诺尔德讲过一个更短的例子。法国一个小学生被问“2 加 3 等于几”,答“3 加 2,因为加法满足交换律”。他不知道答案是 5,但他知道一条公理。

教育学家 Perkins 给这两种病起了名字。元素病(elementitis):只学零件,从没见过整体。关于病(aboutitis):只学“关于”一件事的知识,从没真正做过这件事。

原理为什么不能先学?打个比方。一条原理像一张会员卡:办卡要先交一笔钱,之后每次消费便宜一点。只来一两次,办卡亏了;来得够多,才回本。

脑子里要记的东西见过的例子数 →回本点还没回本开始省力一个例子都没见过就先学原理:先交全款不学原理,每个例子都死记先学原理,每个例子只记不一样的那点
图 7会员卡的账。交点左边,原理还没回本;交点右边,它开始帮你省力。数字随手取,示意。

学原理也是这笔账。记住它本身要花力气;它的好处是,之后每见到一个例子,你不用从头死记,只要记下这个例子和原理不一样的那一点。见过的例子越多,原理越省力。可一个例子都没见过,原理就只剩成本。越抽象的原理,会员费越贵,越得先见过足够多的例子才回本。课本把极限的严格定义放在第一章,等于让你一次都没来过,就先交全款。

在你见过它要概括的东西之前,一条原理只是负担。你背下来的不是原理,是一串符号。

螺丝钉的形状

现在可以给螺丝钉下一个精确的定义了:需要别人先把问题定义好、拆好、定好验收标准,才能开始干活的人。

对照图 1。学校十几年训练出来的,就是这个形状。你不是能力不够,是能力长成了螺丝钉的形状。

公司里谁负责出题?架构师、产品负责人、技术负责人、创始人。他们的活,是把一团乱麻的现实变成一张张定义清楚的任务单。剩下的人负责把任务单做完。

一团乱麻的现实出题的人定义问题 · 拆解 · 定验收标准学校几乎不练任务单 1任务单 2任务单 3执行的人按单交付学校练了十几年编程 agent几分钟交卷,按用量计价更快更便宜
图 8出题的人把乱麻变成任务单,执行的人按单交付。学校只训练下半截,而下半截现在有了更便宜的执行者。示意。

招聘把这个闭环焊死了。白板面试和 LeetCode 刷题,测的说到底是你记住了多少“题型到解法”的映射。流传最广的刷题清单只有几十到一两百道,比如 Blind 75、NeetCode 150,刷题就是在扩大一张查找表。学校按考试筛人,公司按刷题筛人,两道关口选出来的是同一种能力。

这种筛法到底准不准?2013 年,时任 Google 人力运营高级副总裁的 Laszlo Bock 接受《纽约时报》采访时说,公司分析过自己的招聘数据:脑筋急转弯式的面试题完全是浪费时间;GPA 和考试分数作为招聘标准几乎没用,只对毕业两三年内的新人有一点点相关,之后就没了。

还有更刺耳的一层。到了 2026 年,求解定义清楚的问题,已经是大模型最擅长、也最便宜的事。给一个清楚的函数签名和一组测试,编程 agent 几分钟就能交卷。如果你全部的本事是“拿到清楚的题目,写出正确的解”,你和一个 AI 工具之间的区别,正在只剩下价格。

螺丝钉正在被更便宜的螺丝钉替换。替换不掉的,是能把乱麻变成题目的人,而这种能力,学校从来不练。

反方向也是坑

网上流行另一种说法:别从基础学起,直接啃最难的东西,碰到不懂的词就一路追问下去,先修课是最蠢的发明。这话说中了一半。只认这一半,会掉进另一个坑。

学说话和学微积分不一样

常见的论据是:孩子学语言从来不是先学语法,一张嘴就是完整的句子。可教育心理学家 Geary 1995 年区分过两类能力。一类是生物原生的,比如说话、认人脸,演化给大脑预备好了专门的机制,泡在环境里就能学会。另一类是生物次生的,比如阅读、代数、编程,是文明的产物,大脑没有为它们预备什么。人类说话少说有几万年,文字只有五千年左右,读写到今天仍然得专门教。

就算学说话,孩子听到的也不是“完整的复杂度”。大人对婴儿说话,会本能地放慢、简化、重复,研究者叫它儿向语言(child-directed speech)。Snow 1972 年就发现,母亲跟两岁孩子说话,比跟十岁孩子说话更简单、更多重复。这本身就是脚手架。

新手直接下深水会淹

Kirschner、Sweller 和 Clark 2006 年的综述,批评的正是“少给指导、让学生自己发现”的教法:对真正的新手,它不如明确的讲解和样例。Kalyuga 等人 2003 年总结了“专长逆转效应”:对新手有用的详细指导,对行家反而碍事;适合行家的放手不管,会让新手不知所措。

证据支持“先问题,后讲解”

Kapur 的“有成效的失败”(productive failure)给了一个更精细的顺序:先让学生去碰一个超出能力的问题,失败之后再讲。Sinha 和 Kapur 2021 年汇总了 53 项研究,在理解概念、举一反三上,这个顺序稳定地比“先讲后练”好;在熟练计算上,两种顺序差不多;对小学低年级的孩子,反倒是先讲更好。Schwartz 和 Bransford 1998 年发现了类似的事:先让学生分析一组对比案例,再听讲,听讲的收获更大。

所以基础要讲,但不能抢在问题前面讲。

只在一个情境里学,也带不走

“只有放进大图景里的零件才会被理解”,这句话也说过了头。巴西的街头数学研究是个反例。1985 年,Carraher 夫妇和 Schliemann 考察了巴西累西腓五个在街头卖货的孩子,年龄 9 到 15 岁:

正确率摊位上,口头问98.2%写成应用题73.7%写成算式36.8%
图 9同一批孩子、同样的运算,换一种问法。按 Carraher 等(1985)报告的正确率重画。

在摊位上,孩子们心算价格几乎全对。同样的运算写成应用题,正确率降了一截;写成学校格式的算式,只剩三成多。样本只有五个孩子,结论要打折扣,方向却很清楚:嵌在情境里的知识,同样被困在了情境里。

出路在 Gick 和 Holyoak 1983 年的后续实验里。只读一个类比故事的人,没人提醒时 21% 想到解法;读两个不同情境的故事、并写下它们的共同点,这个比例升到 45%。抽象不是起点,是比较了几个情境之后浮现出来的东西。

故事:将军攻城问题:射线治肿瘤将军要塞地雷军队医生肿瘤健康组织射线分散成几路弱的力量同时汇聚到一点重叠的部分,才是带得走的原理
图 10两个故事表面毫不相干,重叠的部分才是可以带走的原理。只看一个故事,你分不清哪些是本质,哪些是道具。示意。

为什么两个故事比一个好?每个故事里,除了真正的道理,还夹着一堆道具:将军、地雷、军队。只看一个故事,你分不清哪些是道理、哪些是道具,只能连道具一起记住。两个故事一比,道具对不上,自然掉了,剩下对得上的那部分才是道理。比的情境越多,混进来的道具越少。

街头的孩子学会的算术进不了教室,也是这个原因:他们只在一个情境里见过它,算术和椰子、摊位、吆喝缠在一起,分不开。课本走了另一个极端,直接把剥干净的“道理”塞给你,一个故事也没给。你没法自己判断这道理说的是什么,只能整个背下来。

孤立练习没错,错在顺序

Ericsson 研究柏林音乐学院的小提琴学生时发现,最好的那批人,花了大量时间独自做刻意练习,把具体的毛病拆出来反复磨。孤立练习本身没有错。刻意练习针对的是具体的弱点,而弱点只有在完整的演奏里才暴露得出来。没见过整体,就按目录顺序磨零件,才是错的。

另一种螺丝钉

只学“高级”、从不回头补基础的人,是另一台机器上的螺丝钉。他们能聊 ChatGPT 背后的 Transformer,会调开源模型,可要他们推一推最基本的那几步数学,或者说清某个设计为什么非这样不可,就卡住了。问题一旦偏离教程,他们手里没有任何可以往下推的东西。

真正的敌人不是基础,是脱嵌。把概念从它要回答的问题里剥出来,在你需要它之前塞给你,再拿它当门槛拦人,这三件事才是病。

该怎么学

把上面的证据拼起来,可以得到一套能直接上手的做法。

七步学法
  1. 从一个够不着、但看得懂目标的完整问题出发。不是前沿最难的,是跳一跳能摸到边的。Perkins 的说法是,先玩简化版的整场比赛。
  2. 碰到不懂的词就往下追问。每次记两件事:这个概念在回答什么问题?没有它会怎样?
  3. 让基础被问题拉进来,别让课表推进来。看不懂 PCA 的时候去学特征值,它就有了挂钩。
  4. 每个核心概念,至少在两个不同情境里见过。然后动手比较,把共同点写下来。那个共同点才是你的“基础”。
  5. 用三个测试检验自己。画出来,讲给外行听,不查资料从零实现。卡在哪一步,哪里就是错觉。
  6. 去掉章节提示。把不同来源的题混在一起做,逼自己先判断“这是什么问题”。
  7. 在完整任务里找弱点,再拆开练。练完回到完整任务,看弱点还在不在。
完整的问题跳一跳够得着卡住了往下追问拉进基础拆开来练换个情境比较,抽出共同点
图 11从整体出发,在整体里卡住,把基础拉进来拆开练,换个情境比较,再回到整体。示意。

写在最后

考试问你:你记不记得答案。世界问你:你知不知道问题是什么。

学校十几年只训练了前一种能力,为此给你发了文凭。公司为此给你发工资。现在,机器正在用更低的价格提供同一种能力。

这不全怪你。课本、考试、招聘一环扣一环,每一环都在奖励同一件事。可你现在知道了,接下来怎么学,是你自己的事。

基础不是起点,
是摸爬滚打之后,
回头才看清的地基。
留言区聊聊

你第一次发现“学过的东西调不出来”,是在什么时候?哪一样知识,你是后来在真问题里才真正弄懂的?

参考资料
  1. Mazur, E. (1997), Peer Instruction: A User's Manual, Prentice Hall
  2. Halloun & Hestenes (1985), Am. J. Phys. 53: 1043;Hestenes, Wells & Swackhamer (1992), Phys. Teach. 30: 141
  3. Hake, R. R. (1998), Am. J. Phys. 66: 64–74
  4. 费曼《别闹了,费曼先生》,“O Americano, Outra Vez!”一章
  5. Tulving & Thomson (1973), Psychol. Rev. 80: 352–373
  6. Godden & Baddeley (1975), Br. J. Psychol. 66: 325–331;Murre (2021), R. Soc. Open Sci. 8: 200724
  7. Whitehead, A. N. (1929), The Aims of Education and Other Essays
  8. Gick & Holyoak (1980), Cogn. Psychol. 12: 306–355;(1983), 15: 1–38
  9. Simon, H. A. (1973), Artificial Intelligence 4: 181–201
  10. Rohrer & Taylor (2007), Instr. Sci. 35: 481–498
  11. Chi, Feltovich & Glaser (1981), Cogn. Sci. 5: 121–152
  12. Beery, Van Horn & Perona (2018), ECCV;Geirhos et al. (2020), Nat. Mach. Intell. 2: 665–673
  13. Rozenblit & Keil (2002), Cogn. Sci. 26: 521–562
  14. Lakatos, I. (1976), Proofs and Refutations, Cambridge UP
  15. Koedinger & Nathan (2004), J. Learn. Sci. 13: 129–164;Nathan & Koedinger (2000), Cogn. Instr. 18: 209–237;Nathan & Petrosino (2003), Am. Educ. Res. J. 40: 905–928
  16. Lockhart《一个数学家的叹息》(2009);Arnold, V. I. (1998), Russian Math. Surveys 53: 229–236
  17. Perkins, D. (2009), Making Learning Whole, Jossey-Bass
  18. Bryant, A. (2013-06-20), In Head-Hunting, Big Data May Not Be Such a Big Deal, NYT
  19. Geary, D. C. (1995), Am. Psychol. 50: 24–37;Snow, C. E. (1972), Child Dev. 43: 549–565
  20. Kirschner, Sweller & Clark (2006), Educ. Psychol. 41: 75–86;Kalyuga et al. (2003), Educ. Psychol. 38: 23–31
  21. Sinha & Kapur (2021), Rev. Educ. Res. 91: 761–798;Schwartz & Bransford (1998), Cogn. Instr. 16: 475–522
  22. Carraher, Carraher & Schliemann (1985), Br. J. Dev. Psychol. 3: 21–29
  23. Ericsson, Krampe & Tesch-Römer (1993), Psychol. Rev. 100: 363–406
注:插图均为本文自制示意。图 4、图 9 按原论文报告的正确率重画;图 5 的路径由程序算出;其余为示意,非原始数据。
犹剑在心
北极甜虾(剑心犹在!)
— 全文完 —
Translated from the Chinese original, “你不是没学好,是被教得太好了”.
Passing exams and solving problems are two different skills

You Didn't Learn Badly.
You Were Taught Too Well.

Why exam aces can't solve real problems
More than a decade of schooling trains the easiest stretch of a hard problem. Drawing on decades of experiments in cognitive science, this piece explains three things: why people who are good at exams get stuck the moment they meet a real problem; why such people mostly end up as cogs; and why the opposite advice, “skip the basics,” is a trap as well. It ends with a way of learning you can put to use right away.
A real hard problem takes six stepsNoticesomething's offDefinewhat the problem isFilterwhich facts matterChoosewhich method fitsSolveapply it correctlyJudgewhat counts as goodIn a textbook,the authordid these for youSchool drillsonly this boxanswers at the back
Figure 1A real hard problem takes six steps. Textbook exercises leave you only the fifth; the other five are done for you by the problem-setter and the answers at the back. Schematic.

In 1990, Eric Mazur, a physics professor at Harvard, ran into something embarrassing.

He had read a few papers by Halloun and Hestenes saying that after a semester of mechanics, most college students still carried Aristotle's physics around in their heads. At first he didn't believe it. Surely they meant other people's students. So he took their concept test (the forerunner of what became the Force Concept Inventory) and gave it to his own class.

Questions like these involve no calculation. They ask about the most basic things. For example: a heavy truck collides head-on with a small car. Which one feels the larger force? Most people choose the truck. The answer is that the forces are equal, by Newton's third law.

The results were ugly. On this simple test, his Harvard students did only slightly better than on a midterm he considered far harder. One student asked him: should I answer these questions according to what you taught us, or according to how I usually think about these things?

That question laid bare the root of the trouble. The students had two versions of physics in their heads, one for exams and one they actually believed, with no wiring between the two.

This is not a problem peculiar to Harvard. In 1998, Hake pooled the start-of-term and end-of-term test results of 6,542 students in 62 introductory college physics courses:

6542students62 intro courses
0.23traditional lecturenormalized gain
0.48interactive engagementnormalized gain

Normalized gain means this: take the gap between your score at the start of term and a perfect score, and ask what fraction of it you closed by the end. Traditionally taught courses closed, on average, only a little over a fifth. After a full cycle of lectures, homework and exams, three quarters of the gap in students' basic understanding of “force” was left untouched.

Feynman saw the same thing when he taught in Brazil in the early 1950s. He gave each student in a class of future physics teachers a strip of polaroid and asked how they could find its direction of polarization, with a hint to look at the light reflected off the bay outside the window. Nobody said a word. He tried another question: what is Brewster's angle? The students recited it at once: light reflected at that angle is completely polarized, and the tangent of the angle equals the index of refraction. Yet it never occurred to them that the water of the bay was just such a “medium with an index of refraction.” Only when he had everyone turn their polaroids while looking at the bay were they amazed to find that the reflected light really was polarized. Feynman's verdict was blunt: the students had memorized everything, but they didn't know what anything meant.

Being able to recite, calculate and pass exams is one thing; being able to use what you know is another. Below I first explain why this happens, then where people trained this way end up, and finally how to learn.

Knowledge is stored by its cues

Cognitive psychology has a solid finding called the encoding specificity principle, proposed by Tulving and Thomson in 1973: whether a memory can be recalled depends on how much the cues you have at the moment of recall overlap with the context in which it was stored.

The most famous demonstration involves divers. In 1975, Godden and Baddeley had divers learn lists of words either on shore or underwater, and then recall them either on shore or underwater. They remembered more when they recalled in the same place they had learned; changing places cut recall by roughly 30%. (Murre ran a direct replication of this classic experiment in 2021 and could not reproduce it cleanly, so don't treat it as ironclad. Encoding specificity itself rests on a large body of other evidence.)

Applied to learning, the consequence is direct. If you learned gradient descent in the context of “Chapter 4, final exam, multiple-choice question 7,” then those are its retrieval cues. At work the cues look completely different: your model's loss is stuck at 2.3 and won't budge. The two sets of cues have nothing in common.

Cues from classCues at workChapter 4Final examMC question 7GDloss stuck at 2.3×erratic training×The cues don't match, so the knowledge won't come back
Figure 2Knowledge is stored together with the cues present when it went in. The cues that come up at work don't connect with the ones from class. Schematic.

The knowledge is right there in your head; you just can't get it out. Back in 1929, Whitehead gave this kind of thing a name, “inert ideas”:

Ideas that are merely received into the mind without being utilised, or tested, or thrown into fresh combinations.Whitehead, “The Aims of Education” (1929)

In 1980, Gick and Holyoak put a number on this failure to retrieve. Participants first read a story: a general wants to capture a fortress, but every road leading to it is mined, and a large army marching together would set off the mines. So the general splits his army into small groups, which set out from every direction at once and meet at the fortress.

Then they tackled a medical puzzle, known in psychology as Duncker's radiation problem. A patient has a tumor in the stomach. Only rays strong enough will destroy it, but rays that strong would also burn the healthy tissue they pass through on the way. What can be done?

The two problems have exactly the same structure: split into several weak forces that converge on one point at the same time. Of the people who had not read the story, fewer than one in ten came up with this solution. And those who had? In one experiment, with the story buried among several unrelated ones, only 20% thought of using it when no one prompted them. When the experimenter added a single hint that one of the earlier stories might help, the figure rose to 92%.

So “good at exercises but unable to solve problems” can be given a precise definition: you have the answers, but no path to them.

The chapter does the hardest step for you

Open any textbook to the exercises at the end of a chapter, and you'll find four unspoken guarantees. Set them against what real hard problems look like:

What exercises guaranteeWhat real problems look like
The problem is already definedThe question states what to find
No one tells you what the problem is; often you have to discover for yourself even that something has gone wrong
Exactly the right informationEvery given gets used
Half of it is missing and half of it is noise; finding the missing part costs money and time
A single correct answerIf it matches the back of the book, it's right
Several options, each with its own costs; you have to set the standard for “good” yourself, and then convince other people
The method is in this chapterUse whatever you just learned
The tools are scattered across the whole body of knowledge, or may not have been invented yet
Figure 3The left column lists the four guarantees a textbook gives you; the right column shows what they look like in real problems.

In 1973, Herbert Simon called problems like those on the right “ill-structured problems.” In his view, many problems treated as well-structured are better seen as ill-structured. He also observed that most of the effort in problem solving goes into giving the problem a structure; once that is done, the actual “solving” is only a small part.

Now look at how you practice. You finish Section 3.2, then do the exercises at the end of Section 3.2. You never have to decide which method a problem calls for, because the section number has already told you. Yet in real problems, that step is often the hardest.

This difference can be measured directly. In 2007, Rohrer and Taylor had college students learn the volume formulas for four kinds of solids. One group practiced in blocks by type, finishing one kind before moving on to the next; the other group practiced all four kinds shuffled together.

Blocked: one type, then the nextMixed0%25%50%75%100%89%60%During practice20%63%Test a week laterSame problems, same amount; only the order differs
Figure 4Redrawn from the mean accuracies reported for Experiment 2 in Rohrer & Taylor (2007). The blocked group is ahead during practice; a week later the positions are reversed.

During practice, the blocked group scored much higher and seemed to be learning better. On the test a week later, the result flipped: the blocked group was down to about 20%, while the mixed group scored over 60%. What blocked practice produces is the illusion of “I've got this.”

In 1981, Chi, Feltovich and Glaser asked physics novices and experts to sort a pile of mechanics problems. Novices sorted by surface features: this is an inclined-plane problem, that is a spring problem. Experts sorted by principle: this one belongs to conservation of energy, that one to Newton's second law. Textbooks organized by chapter train the first kind of sorting every day.

The reason is actually simple. In blocked practice, the section number is a ready-made hint: every problem in this section uses this section's method. You can skip understanding the problem altogether and know which formula to apply just from the section it appears in. You may get every exercise right, yet the skill of “reading a problem and deciding which method it needs” never gets practiced even once.

This isn't to say students are lazy. The brain loves a shortcut: when one is available, it won't bother building the long road. Mixed practice blocks the shortcut, so you have no choice but to read the problem properly and decide. The test a week later tests exactly that step, and that is where the gap comes from.

Look back at Figure 1. The textbook has done the noticing, defining, filtering and choosing of tools for you, and the answers are printed at the back. What you have drilled for more than a decade is the easiest stretch of a hard problem.

You're like an overfitted AI

People who work in AI have a word for this: overfitting. A model learns its practice problems inside out, then fails the moment the problems change. Students suffer from exactly the same thing.

Beery and colleagues open their 2018 paper with an example: a commercial image-recognition service identified cows in pastures accurately, but once a cow stood on a beach by the waves, it failed to recognize the cow or did so poorly. A good part of what it had learned as “cow” was the grass behind the cow. In 2020, Geirhos and colleagues gathered such phenomena under one name, shortcut learning: a model finds a convenient rule in its training data, and as long as the test questions look like the practice ones, it never misses; change their appearance and it collapses at once.

The shortcuts students learn are just the same. Problems in this section use this section's formula. However many numbers the problem gives you, you must use every one of them. The answer is usually a whole number, so if you get a strange square root, go back and check. Whatever the teacher keeps stressing will be on the exam.

Every one of these works well in the exam hall, because exams look just like the exercises. An exam score tells you only how you do on the kind of problem you have practiced; it can't tell you whether you would still cope with a different kind. A high score doesn't mean you learned shortcuts, but an exam can't catch shortcuts either, because it never asks anything outside the range of what was practiced.

The people who have learned shortcuts most thoroughly are often the last to notice. In 2002, Rozenblit and Keil ran an experiment: they asked people to rate their own understanding of things like zippers, flush toilets and cylinder locks, then asked them to write out, step by step, how each one works, and then had them rate themselves again. After writing, the ratings dropped noticeably. They called this “the illusion of explanatory depth.”

Why not try it now? Close the book and draw gradient descent. Not the one-line update formula: draw it.

StartMinimumBig steps: bouncing between the valley wallsSmall steps: no bouncing, but slow
Figure 5A long, narrow valley; the rings are contour lines. The red path takes big steps and bounces back and forth between the two walls; the gray path takes small steps and doesn't bounce, but moves slowly. Both paths take 16 steps, computed by a program.

Drawn out, it looks like this. Think of the objective you are optimizing as a landscape, with the lowest point at the bottom of a valley. Gradient descent is walking downhill with your eyes closed: at every step, you move in whichever direction is steepest under your feet.

When the valley is long and narrow, the steepest direction points almost straight across at the opposite wall, not along the valley floor. Take steps that are too big and you bounce back and forth between the walls, like the red path in the figure. Take small steps and you don't bounce, but you hardly get anywhere, like the gray path. Make the steps a little bigger still and you fly right out of the valley.

Almost everything people in machine learning tune day after day can be traced back to this picture. Why can't the learning rate be too large? Look at the red path. Why add “momentum”? It lets the ball roll downhill with inertia, so the side-to-side bouncing cancels out while the forward speed keeps building. What do methods like Adam do? They give each direction its own ruler, stretching the flattened valley back into a round one.

One picture explains all of this, and the formula told you none of it. The formula is a compressed version of this picture. Someone who can write the formula but can't draw the picture is holding a zip file they can't unzip.

The foundations came last

Look at the order in which humanity itself learned calculus.

1660s
Newton and Leibniz independently invented calculus and computed with “infinitesimals.” What an infinitesimal actually was, nobody could say.
1700s
Euler, the Bernoulli family and Lagrange used it to solve a host of problems in mechanics, astronomy and vibration. In 1734, Bishop Berkeley wrote “The Analyst,” mocking infinitesimals as “the ghosts of departed quantities.” Mathematicians kept using them all the same.
1821
Cauchy published his “Cours d'analyse,” putting the limit at the center.
1861
Weierstrass lectured in Berlin; the rigorous definition of the limit used in today's textbooks (ε-δ) was written into his lecture notes.
1908
Zermelo published axioms for set theory. In 1922, Fraenkel and Skolem independently added the axiom of replacement, giving ZF the form it has today.
Today
The textbook order: first sets and the rigorous definition of the limit, then derivatives, and finally (maybe) applications.
Figure 6The order in which humanity learned calculus runs opposite to the order in which textbooks teach it.

Calculus was in use for a century and a half and solved a great many practical problems. Throughout that century and a half, nobody could say what an infinitesimal was. The rigorous definition of the limit that opens today's textbooks did not take its final shape until the second half of the nineteenth century. The axioms of set theory came later still, in the twentieth.

The foundations of mathematics were poured last. Textbooks turn the order completely around: they give you first what was discovered last, and only then what was discovered first; and what problem all of it was originally meant to solve is often left to the end, or never mentioned at all. In “Proofs and Refutations,” Lakatos calls this way of writing the “deductivist style”: definitions, lemmas and theorems laid out in a row, looking as if they could not be otherwise, while the way knowledge actually grew, through conjectures, counterexamples and repairs, is kept out of sight.

Even more ironic evidence comes from algebra classes. In 2004, Koedinger and Nathan tested 171 high school students who had just begun algebra. When a quantitative relationship was written as a word problem, 70% of answers were correct; when the same relationship was written as a bare symbolic equation, only 42% were. Yet most of the dozens of high school math teachers and education researchers that Nathan and Koedinger had surveyed earlier predicted the reverse: symbolic equations are more “basic,” and word problems are harder. The researchers called this the “symbol precedence view,” and the group that held it most widely was high school teachers. In 2003, Nathan and Petrosino further found that the more mathematics prospective teachers had studied, the more likely they were to hold this view. They called it the “expert blind spot.”

What curriculum designers see as “basic” runs opposite to the direction in which students actually think.

Paul Lockhart opens “A Mathematician's Lament” with a musician's nightmare: music has become a compulsory subject, but all that is taught is notation, staves, key signatures and the rules for writing music down. Children copy out notes year after year, and actually playing and listening has to wait until college, more likely graduate school. That, he says, is how we teach mathematics.

Arnold told an even shorter story. A French schoolchild was asked “What is 2 + 3?” and answered: “3 + 2, since addition is commutative.” He didn't know the answer was 5, but he knew an axiom.

Perkins, an education researcher, gave these two ailments names. Elementitis: learning only the parts, never seeing the whole. Aboutitis: learning only knowledge “about” something, without ever actually doing it.

Why can't principles come first? Here is an analogy. A principle is like a membership card: you pay a fee up front to get it, and after that every visit costs a little less. Come only once or twice and the card loses you money; only if you come often enough does it pay for itself.

What you have to rememberExamples seen →Break-evenStill a lossSaving effortPrinciple before any example:pay in full up frontNo principle: memorize every examplePrinciple first: note only how each differs
Figure 7The membership-card math. Left of the crossing point, the principle hasn't paid for itself yet; right of it, it starts saving you effort. The numbers are made up, for illustration only.

Learning a principle works the same way. Memorizing it takes effort in itself. The payoff is that afterwards, each time you meet an example, you don't have to memorize it from scratch: you only note the one way this example differs from the principle. The more examples you have seen, the more effort the principle saves. But if you haven't seen a single example, the principle is pure cost. The more abstract the principle, the pricier the membership, and the more examples you need to see before it pays off. Putting the rigorous definition of the limit in chapter 1 is like making you pay in full before you have visited even once.

Until you have seen the things a principle sums up, it is only a burden. What you have memorized isn't a principle; it is a string of symbols.

The shape of a cog

Now we can give a precise definition of a cog: someone who can only start work after someone else has defined the problem, broken it down and set the acceptance criteria.

Compare Figure 1. This is exactly the shape that more than a decade of school trains people into. It's not that you lack ability. Your ability has grown into the shape of a cog.

Who sets the problems in a company? Architects, product leads, tech leads, founders. Their job is to turn the tangled mess of reality into a stack of clearly defined task tickets. Everyone else's job is to get the tickets done.

Messy realityProblem-settersdefine · break down · set criteriaSchoolbarely trainsTicket 1Ticket 2Ticket 3Implementersdeliver to the ticketSchool trainsfor 10+ yearsCoding agentdone in minutes, billed by usageFasterCheaper
Figure 8Problem-setters turn the mess into task tickets; implementers deliver to the ticket. School trains only the lower half, and the lower half now has a cheaper implementer. Schematic.

Hiring welds the loop shut. Whiteboard interviews and grinding LeetCode problems test, in the end, how many “problem type to solution” mappings you have memorized. The most widely shared practice lists run from a few dozen to one or two hundred problems, such as Blind 75 and NeetCode 150; grinding them is just enlarging a lookup table. Schools filter people by exams, companies filter them by coding drills, and the two gates select for the same ability.

Does this filtering actually work? In 2013, Laszlo Bock, then Google's senior vice president of people operations, told The New York Times that the company had analyzed its own hiring data. Brainteaser-style interview questions turned out to be a complete waste of time. GPAs and test scores were almost worthless as hiring criteria: they correlated only slightly with performance for new graduates in their first two or three years, and not at all after that.

There is a harsher layer still. By 2026, solving well-defined problems is the thing large language models do best and most cheaply. Give a coding agent a clear function signature and a set of tests, and it hands in its work within minutes. If all you can do is “take a clear problem and write a correct solution,” the difference between you and an AI tool is shrinking to price alone.

Cogs are being replaced by cheaper cogs. What can't be replaced are the people who can turn a mess into a problem, and that ability is the one school never trains.

The opposite direction is a trap too

Another view is popular online: don't start with the basics. Dive straight into the hardest material, chase down every unfamiliar word as you go, and treat prerequisites as the dumbest invention ever. That view is half right. Take only that half, and you fall into a different trap.

Learning to talk is not like learning calculus

The usual argument is that children never learn grammar first; they open their mouths and out come whole sentences. But in 1995 the educational psychologist Geary distinguished two kinds of ability. One kind is biologically primary, such as speaking or recognizing faces: evolution has given the brain dedicated machinery for them, and simply being immersed in the environment is enough to learn them. The other kind is biologically secondary, such as reading, algebra and programming: these are products of civilization, and the brain has nothing prepared for them. Humans have been talking for tens of thousands of years at least, but writing is only about five thousand years old, and reading and writing still have to be taught explicitly today.

Even when learning to talk, children don't hear “the full complexity.” Adults instinctively slow down, simplify and repeat themselves when talking to babies, which researchers call child-directed speech. As early as 1972, Snow found that mothers spoke more simply and with more repetition to two-year-olds than to ten-year-olds. That in itself is scaffolding.

Beginners thrown in the deep end drown

A 2006 review by Kirschner, Sweller and Clark took aim at exactly the teaching that gives little guidance and lets students discover things for themselves: for genuine beginners, it does worse than explicit instruction and worked examples. In 2003, Kalyuga and colleagues summarized the “expertise reversal effect”: detailed guidance that helps novices gets in experts' way, while the hands-off approach that suits experts leaves novices lost.

The evidence favors “problem first, explanation after”

Kapur's “productive failure” offers a more refined order: first let students attempt a problem beyond their ability, and explain only after they have failed. In 2021, Sinha and Kapur pooled 53 studies. For conceptual understanding and transfer to new problems, this order consistently beat “explain first, then practice”; for procedural fluency, the two orders came out about even; and for children in the early grades of primary school, explaining first actually worked better. Schwartz and Bransford found something similar in 1998: when students first analyzed a set of contrasting cases and then heard the lecture, they got more out of the lecture.

So the basics do need to be taught, just not ahead of the problem.

Learned in one context only, it won't travel

“Parts are only understood once they are placed in the big picture”: that claim goes too far as well. The studies of street mathematics in Brazil are a counterexample. In 1985, the Carrahers and Schliemann studied five children, aged 9 to 15, who sold goods on the streets of Recife, Brazil:

AccuracyOral, at the stall98.2%Word problems73.7%Bare sums36.8%
Figure 9The same children, the same operations, asked in different ways. Redrawn from the accuracies reported by Carraher et al. (1985).

At their stalls, the children worked out prices in their heads and got almost all of them right. When the same operations were written as word problems, accuracy dropped a notch; written as bare sums in school format, it fell to just over a third. With only five children in the sample, the conclusion should be taken with a grain of salt, but the direction is clear: knowledge embedded in a context is also trapped in that context.

The way out lies in a follow-up experiment by Gick and Holyoak in 1983. Of those who read just one analogous story, 21% came up with the solution without a hint; of those who read two stories set in different contexts and wrote down what they had in common, 45% did. Abstraction is not the starting point. It is what emerges after you compare several contexts.

Story: the siegeProblem: the tumorGeneralFortressMinesArmyDoctorTumorHealthy tissueRaysSplit intoweak forces,all convergingon one pointOnly the overlap is a principle you can take with you
Figure 10On the surface the two stories have nothing to do with each other; only the overlap is a principle you can take with you. With just one story, you can't tell what is essential and what is a prop. Schematic.

Why are two stories better than one? Besides the real idea, each story carries a pile of props: the general, the mines, the army. With only one story, you can't tell which parts are the idea and which are props, so you have to memorize the props along with it. Put two stories side by side and the props don't match, so they simply fall away; what is left, the part that does match, is the idea. The more contexts you compare, the fewer props slip in.

That is also why the arithmetic the street children learned never made it into the classroom: they had only ever seen it in one context, so it was tangled up with coconuts, stalls and hawkers' cries, impossible to pull apart. Textbooks go to the other extreme and hand you the stripped-down “idea” directly, without a single story. You have no way to judge for yourself what the idea is about, so all you can do is memorize it whole.

Isolated practice isn't wrong; the order is

When Ericsson studied violin students at the music academy in Berlin, he found that the best of them had spent a great deal of time alone in deliberate practice, isolating specific flaws and working on them over and over. Isolated practice is not wrong in itself. Deliberate practice targets specific weaknesses, and weaknesses only show up in a full performance. What is wrong is grinding away at the parts in table-of-contents order without ever having seen the whole.

Another kind of cog

People who learn only the “advanced” material and never go back to fill in the basics are cogs in a different machine. They can talk about the Transformer behind ChatGPT and tweak open-source models, but ask them to work through the most basic steps of the math, or explain why some design has to be the way it is, and they get stuck. Once a problem strays from the tutorial, they have nothing in hand to push forward with.

The real enemy isn't the basics. It's taking them out of context. Stripping a concept out of the problem it was meant to answer, handing it to you before you need it, and then using it as a gate to keep people out: those three things are the disease.

How to learn

Put the evidence above together, and you get a method you can start using right away.

Learning in seven steps
  1. Start from a whole problem that is out of your reach but whose goal you can understand. Not the hardest thing at the frontier, but something you can just touch the edge of if you stretch. In Perkins's words, first play a junior version of the whole game.
  2. When you hit a word you don't understand, chase it down. Each time, write down two things: what question is this concept answering? What would happen without it?
  3. Let the problem pull the basics in; don't let the syllabus push them in. Learn eigenvalues when PCA stops making sense to you, and they will have a hook to hang on.
  4. Meet every core concept in at least two different contexts. Then compare them yourself and write down what they have in common. That common part is your real “foundation.”
  5. Test yourself three ways. Draw it, explain it to a layperson, and implement it from scratch without looking anything up. Wherever you get stuck is where the illusion lies.
  6. Remove the chapter cues. Mix problems from different sources, and make yourself decide first: “what kind of problem is this?”
  7. Find your weak spots inside the whole task, then practice them separately. Afterwards, go back to the whole task and see whether the weakness is still there.
Whole problemjust within reachStuckchase it downPull in basicsdrill the partsSwitch contextcompare, find overlap
Figure 11Start from the whole, get stuck inside it, pull in the basics and drill them separately, compare in a new context, and come back to the whole. Schematic.

In closing

An exam asks you: do you remember the answer? The world asks you: do you know what the problem is?

For more than a decade, school trained only the first ability, and gave you a diploma for it. Companies pay you a salary for it. Now machines are offering the same ability at a lower price.

Not all of this is your fault. Textbooks, exams and hiring lock into one another, and every link rewards the same thing. But now you know, and how you learn from here on is up to you.

The basics are not the starting point.
They are the foundation you see clearly
only looking back, after the scrapes and falls.
Let's talk in the comments

When did you first notice that something you had learned just wouldn't come back to you? Which piece of knowledge did you only truly understand later, inside a real problem?

References
  1. Mazur, E. (1997), Peer Instruction: A User's Manual, Prentice Hall
  2. Halloun & Hestenes (1985), Am. J. Phys. 53: 1043; Hestenes, Wells & Swackhamer (1992), Phys. Teach. 30: 141
  3. Hake, R. R. (1998), Am. J. Phys. 66: 64–74
  4. Feynman, “Surely You're Joking, Mr. Feynman!”, chapter “O Americano, Outra Vez!”
  5. Tulving & Thomson (1973), Psychol. Rev. 80: 352–373
  6. Godden & Baddeley (1975), Br. J. Psychol. 66: 325–331; Murre (2021), R. Soc. Open Sci. 8: 200724
  7. Whitehead, A. N. (1929), The Aims of Education and Other Essays
  8. Gick & Holyoak (1980), Cogn. Psychol. 12: 306–355; (1983), 15: 1–38
  9. Simon, H. A. (1973), Artificial Intelligence 4: 181–201
  10. Rohrer & Taylor (2007), Instr. Sci. 35: 481–498
  11. Chi, Feltovich & Glaser (1981), Cogn. Sci. 5: 121–152
  12. Beery, Van Horn & Perona (2018), ECCV; Geirhos et al. (2020), Nat. Mach. Intell. 2: 665–673
  13. Rozenblit & Keil (2002), Cogn. Sci. 26: 521–562
  14. Lakatos, I. (1976), Proofs and Refutations, Cambridge UP
  15. Koedinger & Nathan (2004), J. Learn. Sci. 13: 129–164; Nathan & Koedinger (2000), Cogn. Instr. 18: 209–237; Nathan & Petrosino (2003), Am. Educ. Res. J. 40: 905–928
  16. Lockhart, “A Mathematician's Lament” (2009); Arnold, V. I. (1998), Russian Math. Surveys 53: 229–236
  17. Perkins, D. (2009), Making Learning Whole, Jossey-Bass
  18. Bryant, A. (2013-06-20), In Head-Hunting, Big Data May Not Be Such a Big Deal, NYT
  19. Geary, D. C. (1995), Am. Psychol. 50: 24–37; Snow, C. E. (1972), Child Dev. 43: 549–565
  20. Kirschner, Sweller & Clark (2006), Educ. Psychol. 41: 75–86; Kalyuga et al. (2003), Educ. Psychol. 38: 23–31
  21. Sinha & Kapur (2021), Rev. Educ. Res. 91: 761–798; Schwartz & Bransford (1998), Cogn. Instr. 16: 475–522
  22. Carraher, Carraher & Schliemann (1985), Br. J. Dev. Psychol. 3: 21–29
  23. Ericsson, Krampe & Tesch-Römer (1993), Psychol. Rev. 100: 363–406
Note: All figures are schematics made by the author. Figures 4 and 9 are redrawn from the accuracies reported in the original papers, and the paths in Figure 5 were computed by a program; the rest are schematic, not original data.
犹剑在心
北极甜虾(剑心犹在!)
— The End —

← All essays