Formal athlete assessments tend to target a combination of physical abilities (e.g. maximum strength) and anthropometrics (e.g. morphology).  More recently, they tend to include cognitive abilities (e.g. creativity) (Memmert 2011), psychological profiles (e.g. motivations) (MacNamara, Button et al. 2010, MacNamara and Collins 2015), and sport specific technique(Glazier 2017).  Athlete assessments have long since been in practice, most notably in the former Soviet Union and East Germany.  They were used as a form of prediction to assess which children would be more suitable to train in different sports.  Though some argue that advancements in research and technology have greatly progressed to make athlete assessment more effective (Albert, Glickman et al. 2017), much research claims that athlete assessment practices, including the Soviet Union, have extremely low success rates (Phillips, Davids et al. 2010, MacNamara and Collins 2012, Renshaw, Davids et al. 2012, Gullich and Cobley 2017).

Assessing the Issues with Traditional Assessment

Next, we will highlight some issues with traditional assessment as presented in both academics and sport science literature.

The Standard of Successful Assessment

In its most traditional form, an assessment compares the performance of an individual to some sort of standard- be it a competition protocol, moves in the field test sheet, or physical ability test.  The purpose of a standard is to reflect the quality of performance expected in a profession or discipline (Joughin 2009).

There’s been a long-heated debate in the literature on the topic of standards.  Some experts claim that standards should be resisted (Kim 2011); that standards suppress individuality and creativity and promote conformity and linearity (Robinson and Aronica 2016).  Others caution that, once a standard is established, it becomes etched in stone (Goodyear and Markauskaite 2018); a blueprint for success.  Anyone taking part in learning must follow that pathway.

Let’s use the IJS competition protocol as an example of standards.  IJS protocols are those sheets of paper given to skaters after competitions that reflect judges’ scores for their performance.  Skaters can see which skills they were credited for performing (e.g. a level 4 combination spin, the hardest type of spin to achieve) and the quality of their performance (say, for example, the judges averaged awarding that spin a +3 grade of execution, which is great!).  The quality of performance is informed by guidelines in the IJS grade of execution documents.  So, in the case of the level 4 combination spin, to achieve a +3, the performance had to ‘tick’ three boxes out of six options in the grade of execution marking guidelines provided to the judges (and available for us all to read). Those options include: 1. Good speed and/or acceleration during the spin 2. Good controlled, clear positions 3. Effortless throughout 4. Maintaining a centered spin 5. Creativity and originality 6. Element matches the music.

There certainly are issues with this form of assessment and they will be discussed in other blogs as examples.  However, IJS protocols are still very helpful for coaches and skaters to adapt training and program content.

So, then what is the issue with standards?

The concept of standards becomes problematic when standards limit the ways (we think) quality or success can be achieved.  For example, if a standard is provided then everyone will simply copy that standard to make sure they don’t do something that might be more creative (even IF creativity is encouraged) but won’t be considered ‘quality’ work.  This applies not only to a single performance but also an entire developmental trajectory. The promotion of a standard pathway to success is a monumental concern in athlete development research (Cobley, Schorer et al. 2012, Baker, Cobley et al. 2013, Bergeron, Mountjoy et al. 2015, Baker, Cobley et al. 2017, Cobley and Till 2017, Davids, Gullich et al. 2017).  Ideas such as all athletes must train in a particular way, participate in some magical combination of activities, progress through a standard set of developmental stages, use a specific type of equipment, or win medals in their youth by a certain age all come to mind.  Another standard, is the notion that all athletes must accumulate 10,000 hours of practice (a standard) to achieve expertise, which is glorified, inaccurately, in ‘pop science’ books [(Colvin 2008, Gladwell 2008, Coyle 2009, Shenk 2010, Syed 2010)].

There’s even concern in motor skill acquisition research regarding the widespread belief that a standard movement pattern exists for all athletes to copy to perfect a skill (Davids, Araújo et al. 2013, Chow, Davids et al. 2015).  If a great skater lands a quadruple jump, for example, we might look to this performance as a high-quality standard to achieve and start to teach all of our skaters to copy the exact same movements.  The skater certainly proved to us all that this is certainly one way to nail a quadruple jump.  However, the movements that work for that athlete might not be the exact movements that work for other skaters.  In this type of example, it certainly is beneficial to acknowledge this great skater’s quadruple jump and technique as a high-quality standard.  However, the value in this acknowledgement is not for everyone to copy that standard but to use it as one that informs the development of their own way (see also our blog series on video analysis).

What to do about standards?

We came to one powerful realization not long ago: standards are an absolute necessity of the learning process just not in the traditional ‘carved in stone way’.  In the past, to avoid the promotion of rigid, fixed, standards to our students, we took the warnings of the research literature on standards.  The goal was to promote creativity, adaptability, and to acknowledge the uniqueness of each athlete and context.  However, the reality is that, without some sort of standard for a learner to observe, analyze, and compare their work to it is impossible for them to know if they are heading in the right direction.  Learning certainly is a nonlinear process but learning still needs a goal, a target, a level of quality to strive for and perhaps bulldoze over to establish a new level.  Learners need to experience different degrees of quality and different ways of achieving those different degrees.  They need to learn the level of quality expected by their coaches, judges, within their region, section, the country, internationally.  They need learn how to adapt their work accordingly and most importantly, they must develop their own dynamic conception of standards.

Questions for parents

Please take time to deeply think about the question(s) below and provide sincere answers.  The questions do not have right or wrong answers but are meant to trigger your thinking about the assessment process.

Please send your responses to: Garrett@acskating.comand Sean@acskating.com

1. How could an assessment be made without a standard to compare the assessment to?

2. Do you assess your child’s (or your own) skating progress in comparison to a common standard?

References

Albert, J., M. E. Glickman, T. B. Swartz and R. H. Koning (2017). Handbook of Statistical Methods and Analyses in Sports, CRC Press.

Baker, J., S. Cobley and J. Schorer (2013). Talent identification and development in sport: international perspectives, Routledge.

Baker, J., S. Cobley and J. Schorer (2017). Routledge Handbook of Talent Identification and Development in Sport, Taylor & Francis.

Bergeron, M. F., M. Mountjoy, N. Armstrong, M. Chia, J. Côté, C. A. Emery, A. Faigenbaum, G. Hall, S. Kriemler and M. Léglise (2015). “International Olympic Committee consensus statement on youth athletic development.” Br J Sports Med49(13): 843-851.

Chow, J. Y., K. Davids, C. Button and I. Renshaw (2015). Nonlinear pedagogy in skill acquisition: An introduction, Routledge.

Cobley, S., J. Schorer, J. Baker, J. Baker, S. Cobley and J. Schorer (2012). “Identification and development of sport talent: A brief introduction to a growing field of research and practice.” Talent identification and development in sport. International perspectives: 1-10.

Cobley, S. and K. Till (2017). Longitudinal Studies of Athlete Development: Their importance, methods and future considerations Routledge Handbook of Talent Identification and Development in Sport. J. Baker, S. Cobley, J. Schorer and N. Wattie. New York, NY, Routledge.

Colvin, G. (2008). Talent is overrated, Penguin Books.

Coyle, D. (2009). The Talent Code: Greatest Isn’t Born, It’s Grown, Here’s how, Bantam.

Davids, K., D. Araújo, L. Vilar, I. Renshaw and R. Pinder (2013). “An ecological dynamics approach to skill acquisition: Implications for development of talent in sport.” Talent Development & Excellence5(1): 21-34.

Davids, K., A. Gullich, R. Shuttleworth and D. Araujo (2017). Understanding environmental and task constraints on talent development. Routledge Handbook of Talent Identification and Development in Sport. J. Baker, S. Cobley, J. Schorer and N. Wattie. New York, NY, Routledge.

Gladwell, M. (2008). Outliers: The story of success, Hachette UK.

Glazier, P. S. (2017). Could sports biomechanics provide the missing pieces to the talent identification and development puzzle? Routledge Handbook of Talent Identification and Development in Youth Sport. J. Baker, S. Cobley, J. Schorer and N. Wattie. New York, NY, Routledge.

Goodyear, P. and L. Markauskaite (2018). Epistemic resourcefulness and the development of evaluative judgement. Developing Evaluative Judgement in Higher Education, Routledge: 44-54.

Gullich, A. and S. Cobley (2017). On the efficacy of talent identification and talent development programmes. Routledge Handbook of Talent Identification and Development in Sport. J. Baker, S. Cobley, J. Schorer and N. Wattie. New York, NY, Routledge.

Joughin, G. (2009). Assessment, learning and judgement in higher education: A critical review. Assessment, learning and judgement in higher education, Springer: 1-15.

Kim, K. H. (2011). “The creativity crisis: The decrease in creative thinking scores on the Torrance Tests of Creative Thinking.” Creativity Research Journal23(4): 285-295.

MacNamara, Á., A. Button and D. Collins (2010). “The role of psychological characteristics in facilitating the pathway to elite performance part 1: Identifying mental skills and behaviors.” The Sport Psychologist24(1): 52-73.

MacNamara, A. and D. Collins (2012). “Building talent development systems on mechanistic principles: Making them better at what makes them good.”

MacNamara, Á. and D. Collins (2015). “Profiling, exploiting, and countering psychological characteristics in talent identification and development.” The Sport Psychologist29(1): 73-81.

Memmert, D. (2011). “Sports and creativity.” Encyclopedia of creativity2: 373-378.

Phillips, E., K. Davids, I. Renshaw and M. Portus (2010). “Expert performance in sport and the dynamics of talent development.” Sports Medicine40(4): 271-283.

Renshaw, I., K. Davids, E. Phillips and H. Kerhervé (2012). “Developing talent in athletes as complex neurobiological systems.”

Robinson, K. and L. Aronica (2016). Creative Schools: The grassroots revolution that’s transforming education, Penguin Books.

Shenk, D. (2010). The genius in all of us: why everything you’ve been told about genetics, talent, and IQ is wrong, Random House LLC.

Syed, M. (2010). Bounce: Mozart, Federer, Picasso, Beckham, and the science of success, Harper Collins.