I’ve spoken before about the effects of motivation on test performance. This is displayed in a fascinating study by researchers at the Educational Testing Service, who gave one of their widely-used tests (the ETS Proficiency Profile, short form, plus essay) to 757 students from three institutions: a research university, a master's institution and a community college. Here’s the good bit: students were randomly assigned to groups, each given a different consent form. In the control condition, students were told: “Your answers on the tests and the survey will be used only for research purposes and will not be disclosed to anyone except the research team.” In the “Institutional” condition, the rider was added: “However, your test scores will be averaged with all other students taking the test at your college.” While in the “Personal” condition, they were told instead: “However, your test scores may be released to faculty in your college or to potential employers to evaluate your academic ability.”

No prizes for guessing which of these was more motivating!

Students in the “personal” group performed significantly and consistently better than those in the control group at all three institutions. On the multi-choice part of the test, the personal group performed on average .41 of the standard deviation higher than the control group, and the institutional group performed on average .26 SD higher than the controls. The largest difference was .68 SD. On the essay, the largest effect size was .59 SD. (The reason for the results being reported this way is because the focus of the study was on the use of such tests to assess and compare learning gains by colleges.)

The effect is perhaps less dramatic at the individual level, with the average sophomore score on the multichoice test being 460, compared to 458 and 455, for personal, institutional, and control groups, respectively. Interestingly, this effect was greater at the senior level: 469 vs 466 vs 460. For the essay question, however, the effect was larger: 4.55 vs 4.35 vs 4.21 (sophomore); 4.75 vs 4.37 vs 4.37 (senior). (Note that these scores have been adjusted by college admission scores).

Students also reported on motivation level, and this was found to be a significant predictor of test performance, after controlling for SAT or placement scores.

Student participants had received at least one year of college, or (for community colleges) taken at least three courses.

The findings confirm recently expressed concern that students don’t put their best efforts into low-stakes tests, and that, when such tests are used to make judgments about institutional performance (how much value they add), they may well be significantly misleading, if different institutions are providing different levels of motivation.

On a personal level, of course, the findings may be taken as further confirmation of the importance of non-academic factors in academic achievement. Something looked at more directly in the next study.

Motivation, study habits—not IQ—determine growth in math achievement

Data from a large German longitudinal study assessing math ability in adolescents found that, although intelligence was strongly linked to students' math achievement, this was only in the initial development of competence. The significant predictors of growth in math achievement, however, were motivation and study skills.

Specifically (and excitingly for me, since it supports some of my recurring themes!), at the end of Grade 5, perceived control was a significant positive predictor for growth, and surface learning strategies were a significant negative predictor. ‘Perceived control’ reflects the student’s belief that their grades are under their control, that their efforts matter. ‘Surface learning strategies’ reflect the use of rote memorization/rehearsal strategies rather than ones that encourage understanding. (This is not to say, of course, that these strategies don’t have their place — but they need to be used appropriately).

At the end of Grade 7, however, a slightly different pattern emerged, with intrinsic motivation and deep learning strategies the significant positive predictors of growth, while perceived control and surface learning strategies were no longer significant.

In other words, while intelligence didn’t predict growth at either point, the particular motivational and strategy variables that affected growth were different at different points in time, reflecting, presumably, developmental changes and/or changes in academic demands.

Note that this is not to say that intelligence doesn’t affect math achievement! It is, indeed, a strong predictor — but through its effect on getting the student off to a good start (lifting the starting point) rather than having an ongoing benefit.

There was, sadly but unfortunately consistent with other research, an overall decline in motivation from grade 5 to 7. There was also a smaller decline in strategy use (any strategy! — presumably reflecting the declining motivation).

It’s also worth noting that (also sadly but unsurprisingly) the difference between school types increased over time, with those in the higher track schools making more progress than those in the lowest track.

The last point I want to emphasize is that extrinsic motivation only affected initial levels, not growth. The idea that extrinsic motivation (e.g., wanting good grades) is of only short-term benefit, while intrinsic motivation (e.g., being interested in the subject) is far more durable, is one I have made before, and one that all parents and teachers should pay attention to.

The study involved 3,520 students, following them from grades 5 to 10. The math achievement test was given at the end of each grade, while intelligence and self-reported motivation and strategy use were assessed at the end of grades 5 and 7. Intelligence was assessed using the nonverbal reasoning subtest of Thorndike’s Cognitive Abilities Test (German version). The 42 schools in the study were spread among the three school types: lower-track (Hauptschule), intermediate-track (Realschule), and higher-track (Gymnasium). These school types differ in entrance standards and academic demands.