Traditionally, an athlete assessment reflects a single point in time; an isolated act.  A testing day is secured and each athlete undergoes a barrage of tests (e.g. a sprint; max bench press or squat; a balance test). Assessors explain the task and the athlete is often given several trials.  The best trial or an average of all trials is recorded in the assessment. What if an athlete was assessed on a day that did not accurately reflect their current state of fitness?  For example, they had a rough week at school, were mentally distracted and did not give their all in the assessment task. Our perhaps the athlete was not given the proper warm-up needed to perform up to their skill level.  What if the athlete didn’t understand the test or was afraid to give their all?  An individual snapshot such as this would not accurately represent the athlete’s actual capabilities but rather a glimpse of what they could do on that particular day. This happens at competitions when the athlete makes an uncharacteristic mistake (Kurt Browning landing a beautiful triple axel in the Olympics and then ‘popping’ an ‘easy’ double later in the same exact program comes to mind!).  According to the research literature, athlete development programs fail because of the lack of validity of and overreliance such programs place on snapshot assessments to predict athletes’ future performance (Phillips, Keith et al. 2010, MacNamara and Collins 2012, Renshaw, Davids et al. 2012, Weissensteiner 2017).

Above: Traditional athlete assessments test young children’s physical abilities.  How accurately do you think we could judge my son Kellen’s future sport performance if we were testing his abilities in this moment pictured?  What if a child harbors great physical potential but lacks the perceived competence (Collins and MacNamara 2017) in that potential?  They hold back as a result and this conveys to the assessor a lack of physical ability. Yet the real issue is that the child needs to develop confidence in their abilities first.

Davies and Wavering (1999) argue that a complete photo album tells a much richer story than a single snapshot.  One isolated assessment is not much use.  However, if multiple snapshot assessments are strung together then the ‘big picture’ of development should be clearer.  The sequential use of assessments over time are meant to measure rate of progress (Vaeyens, Lenoir et al. 2008)[In academics literature, this form of assessment is also called learning gain (McGrath, Guerin et al. 2015, Fearon, Nachmias et al. 2017)].  Rate of progress emphasizes the rate of learning an athlete displays from one assessment to another which can be explained through the attainment of higher scores for individual criterion and the acquisition of new skills and other performance milestones.

Rate of progress assessment appears to be well-received in the research literature.  However, we have our own concerns.  We realize the rate of learning concept is important, and more beneficial than using an assessment as an isolated act.  One of our biggest issues with rate of learning assessment is the concern that coaches (and parents) will ‘push’ athletes along through whatever means necessary to maintain a fluid pace of progress. Traditionally, progress reflects skills acquired (e.g. a double axel), tests passed (e.g. moves in the field), or gains in physical abilities from one assessment to the next.  This perspective makes the (misguided) assumption that learning and development are ideally linear and fluid.  In other words, did the athlete who had a fluid and linear, but otherwise uneventful, pattern of progress learn more or display greater talent or potential for the future than the athlete who ran into a few major obstacles (e.g. struggle with a new skill; an injury; a poor competitive season; an issue with school) and did not progress?  Is the assessed who makes more mistakes within a given period not learning as effectively as the assessed who does not make mistakes?

Research in both academics (McCombs and Vakili 2005, Koopmans 2014)and motor learning (I Schollhorn, Hegen et al. 2012, Chow, Davids et al. 2015) suggests that learning is in fact a nonlinear process.  Practice conditions that attempt to facilitate performance do not facilitate long-term learning (Schmidt and Lee 2013) and individuals who make the most mistakes in practice might even be learning the most (Soderstrom and Bjork 2015, Bjork 2017).  This means that if we designed practices to be simple and repetitive in efforts to reduce challenge, our athletes will not learn as much compared to practices designed to induce more mistakes; to be more challenging.  Why?  When learners are forced to practice in different ways they experience the ‘material’ from new perspectives which allows them to perceive details they might not have recognized before (see our blog Be Mindful of Repetition for more).  There is a distinction, in other words, between learning conditions (those that induce the most errors) and performance conditions (those that reduce errors) (Schmidt and Lee 2005, Kantak and Winstein 2012) and, of course, we will discuss this in greater detail in future blogs.

On a grander “life event” scale, researchers argue that the experience of major obstacles, such as those provided above, when progress comes to a screeching halt or even seems to regress, are vital to athletes reaching their optimal long-term potential (Collins and MacNamara 2012, Collins, MacNamara et al. 2016, Collins and MacNamara 2017).  They even speculate that athletes who do not have such experiences, may progress more smoothly at first, but seem to plateau or peak before they reach the highest levels [see: (Collins, MacNamara et al. 2016) for specific discussion].  Researchers presume that these major obstacles trigger deep reflection, new perspectives, greater focus and/or commitment that would otherwise not manifest within the athlete.

Unfortunately, the common practice of coaches, parents, and athletes is to judge the quality of a practice session based on the quality of the athlete’s performance within that practice (my child fell a lot = my child had a bad practice and didn’t learn anything). Greater practice performance (e.g. landing all your jumps) means the practice was more effective.  However, this could not be farther from the truth. Keeping this distinction in mind, our asssessments should measure progress but they also need to consider the context through which that progress developed.  The measurement of ‘progress’ must be broadened to include these life event experiences and other learning events, and not just performance alone.

Questions for parents

Please take time to deeply think about the question(s) below and provide sincere answers.  The questions do not have right or wrong answers but are meant to trigger your thinking about the assessment process.

Please send your responses to: Garrett@acskating.com and Sean@acskating.com 

1. How could mistake-laden periods of training and major obstacles be expected as necessary or vital features of an athlete’s development, yet progress-centered assessment doesn’t take these events into consideration?

References

Bjork, R. (2017). “Creating Desirable Difficulties to Enhance Learning.” Progress.

Chow, J. Y., K. Davids, C. Button and I. Renshaw (2015). Nonlinear pedagogy in skill acquisition: An introduction, Routledge.

Collins, D. and A. MacNamara (2017). Talent development: a practitioner guide, Routledge.

Collins, D. and Á. MacNamara (2012). “The rocky road to the top.” Sports Medicine42(11): 907-914.

Collins, D. and Á. MacNamara (2017). “A smooth sea never made a skilled sailor: Optimizing and exploiting the rocky road in talent development.”

Collins, D., Á. MacNamara and N. McCarthy (2016). “Super champions, champions, and almosts: important differences and commonalities on the rocky road.” Frontiers in psychology6: 2009.

Davies, M. A. and M. Wavering (1999). “Alternative assessment: New directions in teaching and learning.” Contemporary Education71(1): 39.

Fearon, C., S. Nachmias, H. McLaughlin and N. Leatherbarrow (2017). “Capturing (self-determined) learning gain using personality measures and core trait self-evaluation: a new “learner profile matrix”.”

I Schollhorn, W., P. Hegen and K. Davids (2012). “The nonlinear nature of learning-A differential learning approach.” The Open Sports Sciences Journal5(1).

Kantak, S. S. and C. J. Winstein (2012). “Learning–performance distinction and memory processes for motor skills: A focused review and perspective.” Behavioural brain research228(1): 219-231.

Koopmans, M. (2014). “Nonlinear change and the black box problem in educational research.” Nonlinear dynamics, psychology, and life sciences18(1): 5-22.

MacNamara, A. and D. Collins (2012). Building talent development systems on mechanistic principles: making them better at what makes them good. Talent Identification and Development in Sport. J. Baker, S. Cobley and J. Schorer. New York, NY, Routledge.

McCombs, B. L. and D. Vakili (2005). “A learner-centered framework for e-learning.” Teachers college record107(8): 1582.

McGrath, C. H., B. Guerin, E. Harte, M. Frearson and C. Manville (2015). “Learning gain in higher education.” Santa Monica, CA: RAND Corporation.

Phillips, E., D. Keith, I. Renshaw and M. Portus (2010). “Expert performance in sport and the dynamics of talent development.” Sports Medicine40(4): 271-283.

Renshaw, I., K. Davids, E. Phillips and H. Kerherve (2012). Developing talent in athletes as complex neurobiological systems. Talent Identification and Development in Sport. J. Baker, S. Cobley and J. Schorer. New York, NY, Routledge.

Schmidt, R. and T. Lee (2013). Motor Learning and performance, 5E with web study guide: from principles to application, Human Kinetics.

Schmidt, R. A. and T. D. Lee (2005). “Motor control and learning: A behavioral emphasis.”

Soderstrom, N. C. and R. A. Bjork (2015). “Learning versus performance: An integrative review.” Perspectives on Psychological Science10(2): 176-199.

Vaeyens, R., M. Lenoir, A. M. Williams and R. M. Philippaerts (2008). “Talent identification and development programmes in sport.” Sports medicine38(9): 703-714.

Weissensteiner, J. R. (2017). How contemporary international perspectives have consolidated a best-practice approach for identifying and developing sport talent. Routledge Handbook of Talent Identification and Development in Sport. J. Baker, S. Cobley, J. Schorer and N. Wattie. London, Routledge.